TRL

Post-training, the stage that turns a pretrained language model into an instruction follower or a reasoning model, is what Hugging Face's TRL library packages up. It ships trainers for supervised fine-tuning, reward modeling, DPO and KTO preference tuning, online reinforcement learning methods such as GRPO and RLOO, and distillation, each a thin layer over the Transformers Trainer so Hub models and datasets plug straight in. Scaling goes through Accelerate, with DDP or DeepSpeed taking a run from one GPU to a multi-node cluster, and PEFT support means LoRA or QLoRA adapters let mid-sized models train on a single card with modest memory. Online methods can hand generation to a vLLM server for faster sampling, and Unsloth and Liger kernel integrations cut memory further. A trl command line tool starts SFT, DPO or KTO jobs from flags or a YAML config without writing Python. Install with pip; in practice an NVIDIA GPU is needed beyond toy models. TRL is Apache 2.0 and fine for commercial work, though your base model's license still applies. Built for ML engineers and researchers aligning or specializing open models.