LLM Compressor

Quantized checkpoints that load straight into vLLM are the output of LLM Compressor, the vLLM project's official compression library developed with Red Hat AI and descended from Neural Magic's tooling after the Red Hat acquisition. It applies post-training quantization and sparsity to weights, activations, KV cache, and attention, covering W8A8 in both int8 and fp8, microscaling formats like NVFP4 and MXFP4, and algorithms including GPTQ, AWQ, SmoothQuant, AutoRound, and rotation-based methods, plus REAP expert pruning that shrinks mixture-of-experts models by structurally removing less relevant experts. Models save in the compressed-tensors format that vLLM reads natively, so the compress-then-serve loop needs no conversion step. Integration with Hugging Face Transformers keeps most recipes to a few lines of Python, with multi-GPU and disk offloading support for models too large for one card. Installation is pip install llmcompressor, docs sit under docs.vllm.ai, and the Apache-2.0 codebase, at about 3.6k GitHub stars, produces the quantized checkpoints Red Hat publishes on Hugging Face.