ExLlamaV2

Turboderp's ExLlamaV2 was for a long stretch one of the fastest ways to run a quantized large language model on a single consumer NVIDIA card. Its hand-written CUDA kernels handled GPTQ weights and its own EXL2 format, which mixes quantization levels inside a model so you can target an exact average bits per weight and fit a given VRAM budget, and its later dynamic generator added batching, prompt caching and speculative decoding. The final release, v0.3.2, shipped in July 2025, and in March 2026 the author marked the project archived and sent users to ExLlamaV3. The ecosystem followed: text-generation-webui dropped its ExLlamaV2 loader in March 2026 and TabbyAPI removed the backend in July 2026. Existing EXL2 quants on Hugging Face still load with the last published wheels on a matching PyTorch and CUDA combination, or through an older pinned TabbyAPI build, but expect no fixes and no support for new model architectures. The code remains MIT. For new setups, use ExLlamaV3 with the EXL3 format and current TabbyAPI, or llama.cpp with GGUF files if you also need CPU, Apple or AMD hardware.