Nunchaku

SVDQuant, an ICLR 2025 Spotlight paper from MIT HAN Lab, quantizes both weights and activations of diffusion models to 4 bits while absorbing outliers through a low-rank branch, and Nunchaku is the inference engine built to run those models fast. On a 16 GB laptop RTX 4090 it delivers a 3.6x memory reduction and an 8.7x speedup over the BF16 FLUX.1-dev baseline, and with asynchronous CPU offload the engine can operate in as little as 3 GiB of VRAM. Prequantized models cover the FLUX.1 family including Kontext and Krea-dev, Qwen-Image and Qwen-Image-Edit, Z-Image-Turbo, SANA, and PixArt, with LoRA and ControlNet support layered on top. Distribution is through pip with detailed install guides and a ComfyUI integration that drives much of its adoption, and documentation lives at nunchaku.tech; NVIDIA GPUs from the 20 series through the 50 series are supported. The code is Apache-2.0 licensed with about 3,900 stars, and the 1.2 line shipped in January 2026 under the project's renamed nunchaku-ai GitHub organization, followed by 1.3 development builds.