#kserve

1 blog post.

Blog posts

Your model isn't crashing, your probe is

A model server that takes five minutes to load and a liveness probe that gives it ten seconds is a crash loop waiting to happen. Probes, drains, and safe…

Related tags

#gpu #kubernetes #vllm #reliability