#infiniband

1 blog post.

Blog posts

Past one node the network becomes the machine. InfiniBand vs RoCE, gang scheduling, FSDP and Megatron, and why a 16k-GPU cluster fails every three hours.

Related tags

#ai #ai-gateway #ai-tooling #akamai #anime #audit-logs #automation #aws