Back to tags
#infrastructure
the parts of the stack you only think about when they break.
5 blog posts.
Blog posts
Past one node the network becomes the machine. InfiniBand vs RoCE, gang scheduling, FSDP and Megatron, and why a 16k-GPU cluster fails every three hours.
Eight GPUs in one server behave like a small network. NVLink vs PCIe, reading nvidia-smi topo -m, NCCL transports, the ACS trap, and fitting a 70B model.
A GPU pod sits on a dozen layers from silicon to scheduler, and each one fails its own way. Drivers, the container toolkit, MIG, DCGM, and the metrics…
Our AWS bill hit $50k/month. Here's exactly how we reduced it to $20k without sacrificing performance or reliability.
Learning Terraform the hard way. Here are the mistakes that cost me sleep, money, and a bit of my sanity.