The llm-d project announced version 1.0 on October 20, an open-source release aimed at making large-scale AI inference more efficient and production-ready.

On October 20, 2026, llm-d contributors announced version 1.0 at PyTorch Conference North America, consolidating an open-source, Kubernetes-native stack for distributed large-language-model inference. IBM, Red Hat, Google Cloud, CoreWeave and NVIDIA are among the project’s contributors. The release adds priority- and fairness-aware controls for sharing capacity across tenants, router hot standby and rolling upgrades, coordinated updates for separate prompt-processing and response-generation replicas, and a stable v1 configuration API. Session-aware routing and turn-priority scheduling aim to reuse context across agent conversations rather than repeatedly processing it. In a reported test, llm-d served up to 3,000 concurrent coding agents with zero preemptions and peaked above 6.6 million output tokens per minute.