LitGPT

Recipes rather than abstractions drive LitGPT, the Lightning AI library that reimplements more than 20 language models from scratch, including Llama 3, Qwen 2.5, Phi 4, Gemma 2, Mistral, Falcon 3, and Code Llama, each kept readable with no abstraction layers on top of PyTorch. The command line covers the full lifecycle, litgpt pretrain, finetune, evaluate, and serve, with parameter-efficient LoRA, QLoRA, and Adapter methods alongside full fine-tuning, 4-bit and 8-bit quantization, Flash Attention 2, and multi-GPU scaling through FSDP. Configs ship as validated YAML files with tested hyperparameters, so a fine-tune can start with a single command after pip install litgpt[extra]. Serious training needs an NVIDIA GPU, while quantized runs of small models fit on modest hardware. Everything is Apache-2.0 licensed, and the project has about 13,600 GitHub stars. Several open pretraining efforts have used it as their base implementation, and it comes from the team that maintains PyTorch Lightning, which shows in the emphasis on reproducible, debuggable single-file model code.