#routing

1 blog post.

Blog posts

The cheapest speedup is your load balancer

Same GPUs, same model, same replica count. Swap round-robin for prefix-cache-aware routing and the fleet gets 2.3x faster. The router was throwing the…

Related tags

#ai #ai-gateway #ai-tooling #akamai #anime #audit-logs #automation #autoscaling