Nunchaku
Inference engine that runs 4-bit SVDQuant diffusion models with large speed and memory gains.
About
SVDQuant, an ICLR 2025 Spotlight paper from MIT HAN Lab, quantizes both weights and activations of diffusion models to 4 bits while absorbing outliers through a low-rank branch, and Nunchaku is the inference engine built to run those models fast. On a 16 GB laptop RTX 4090 it delivers a 3.6x memory reduction and an 8.7x speedup over the BF16 FLUX.1-dev baseline, and with asynchronous CPU offload the engine can operate in as little as 3 GiB of VRAM. Prequantized models cover the FLUX.1 family including Kontext and Krea-dev, Qwen-Image and Qwen-Image-Edit, Z-Image-Turbo, SANA, and PixArt, with LoRA and ControlNet support layered on top. Distribution is through pip with detailed install guides and a ComfyUI integration that drives much of its adoption, and documentation lives at nunchaku.tech; NVIDIA GPUs from the 20 series through the 50 series are supported. The code is Apache-2.0 licensed with about 3,900 stars, and the 1.2 line shipped in January 2026 under the project's renamed nunchaku-ai GitHub organization, followed by 1.3 development builds.
Should you use Nunchaku?
Pick it when
Pick Nunchaku when you run FLUX.1, Qwen-Image, or Z-Image-Turbo on an NVIDIA 20 to 50 series card and want both lower VRAM and much faster generation, ideally through its ComfyUI integration, with LoRA and ControlNet intact.
Look elsewhere when
Skip it if your model has no SVDQuant release and you do not want to quantize it yourself with DeepCompressor, or if you are on AMD, Apple, or CPU only, where stable-diffusion.cpp or MFLUX fit.
Alternatives to Nunchaku
- ComfyUI-GGUF
Wider choice of quantization levels and community GGUF files including SD 3.5 Large and T5-XXL, but aimed at fitting in memory rather than the speedups reported here.
- TeaCache
Training-free speedup that keeps your existing weights and covers video models, but saves no VRAM and trades some fidelity through its threshold.
- stable-diffusion.cpp
Runs quantized GGUF models on AMD, Apple Metal, Vulkan, or plain CPU as well as NVIDIA with no Python, but without SVDQuant's 4-bit kernels.
- xDiT
Cuts latency by spreading one generation across multiple GPUs instead of shrinking the model, suited to servers rather than a single consumer card.
Reviews (0)
Leave a Review
No reviews yet. Be the first to review!
Details
- Category
- Diffusion Model Tools & UIs
- Price
- Free
- Platform
- Local/Desktop
- Difficulty
- Intermediate (3/5)
- License
- Apache-2.0
- Minimum VRAM
- 4 GB
- Added
- Jul 29, 2026
Tags
Related Tools
Browse all Diffusion Model Tools & UIs toolsMentioned in
The Best Open-Source Image Generation Stack in 2026
Model picks per use case, the UI that actually won, a working upscaling and editing chain, and an honest read...
Max P
From ComfyUI to Production: Serving Diffusion Models in 2026
A working guide to moving diffusion pipelines from ComfyUI prototype to production: environment locking,...
Max P
Instruction Image Editing in 2026: Kontext, Qwen, Step1X
A working comparison of FLUX.1 Kontext, Qwen-Image-Edit, Step1X-Edit, MagicQuill and SUPIR on license, VRAM,...
Max P