AIBrix
Kubernetes control plane that scales vLLM serving with LLM-aware routing, autoscaling, and KV cache reuse.
About
Born inside ByteDance and donated to the vLLM project in 2025, AIBrix is a cloud-native control plane that turns a Kubernetes cluster into scalable LLM serving infrastructure around the vLLM engine. The stack, written mostly in Go, adds an LLM-aware gateway that routes requests intelligently, an application-tailored autoscaler, high-density LoRA management for serving many adapters cheaply, a distributed KV cache for cross-engine reuse, heterogeneous GPU serving that mixes card types to cut cost, and GPU hardware failure detection, with a unified runtime sidecar handling metrics and model management. Deployment is Kubernetes-native: install the control plane with kubectl or kustomize manifests, then declare models as custom resources. The project is Apache-2.0 licensed, shipped v0.7.0 in June 2026, has passed 5,000 GitHub stars, and keeps its documentation on Read the Docs. Platform teams running vLLM fleets at scale use it as the orchestration layer that plain Kubernetes primitives do not provide.
Should you use AIBrix?
Pick it when
Pick AIBrix when a platform team already runs vLLM on Kubernetes and needs autoscaling, LLM-aware routing, and dense LoRA adapter serving for many models or tenants, especially across a mix of GPU types to control cost.
Look elsewhere when
Skip it if you have no Kubernetes cluster or serve one model on a few GPUs; plain vLLM or GPUStack is far less to operate. It is built around vLLM, so fleets that also run SGLang or TensorRT-LLM workers fit NVIDIA Dynamo better.
Alternatives to AIBrix
- llm-d
Similar Kubernetes plus vLLM stack with CNCF governance and prefill/decode disaggregation, but less focus on LoRA density and mixed GPU types.
- NVIDIA Dynamo
Coordinates vLLM, SGLang, and TensorRT-LLM workers and installs from PyPI without Kubernetes, but is a larger multi-node framework to learn.
- GPUStack
Far simpler to stand up on machines you already own with no Kubernetes needed, but it is a general cluster manager, not a vLLM control plane.
- LoRAX
Serves thousands of LoRA adapters on one GPU from a single docker run, lighter if adapter density is the only need, with no fleet autoscaling.
Reviews (0)
Leave a Review
No reviews yet. Be the first to review!
Details
- Category
- LLM Inference & Serving
- Price
- Free
- Platform
- Local/Desktop
- Difficulty
- Advanced (4/5)
- License
- Apache-2.0
- Added
- Jul 29, 2026