vLLM
High-throughput LLM serving engine with PagedAttention
About
vLLM is a library for high-throughput LLM inference and serving, originally developed at UC Berkeley's Sky Computing Lab. Its PagedAttention algorithm manages attention key and value memory efficiently to raise throughput and reduce waste. It supports more than 200 model architectures from Hugging Face, continuous batching, quantization, tensor parallelism, and an OpenAI-compatible server. Apache 2.0 licensed and maintained by a large contributor community.
Reviews (0)
Leave a Review
No reviews yet. Be the first to review!
Details
- Category
- LLM Inference & Serving
- Price
- Free
- Platform
- Local/Desktop
- Difficulty
- Intermediate (3/5)
- License
- Apache-2.0
- Minimum VRAM
- 16 GB
- Added
- Jan 29, 2026
Related Tools
Open-source ChatGPT alternative that runs 100% offline on your computer.
Port of Meta's LLaMA model in C/C++ for efficient CPU inference
Fast LLM inference on consumer GPUs using neuron-aware sparse computation.
Easy-to-use local AI inference with built-in web UI and API.
High-performance LLM inference engine forked from vLLM with extra features.
Minimalist machine learning framework for Rust focused on performance and serverless inference.
Mentioned in
The Best Self-Hosted AI Stack for Small Teams in 2026
An opinionated reference architecture for self-hosted team AI in 2026: vLLM, Open WebUI, LiteLLM, Qdrant, and...
Billy C
Best Local LLM Setups by GPU Budget: 8GB, 16GB, 24GB, Multi-GPU
Exact model, quant, and engine picks for every VRAM tier in 2026, from an 8GB laptop GPU to a dual-3090...
Billy C
Running LLMs Without Nvidia in 2026: ROCm and Apple Silicon
The 2026 state of local LLMs off Nvidia: which runtimes actually work on AMD ROCm and Apple Silicon, real...
Billy C