Megatron-LM
NVIDIA framework for training multi-billion parameter transformer models.
About
Megatron-LM by NVIDIA is a GPU-optimized framework for training transformer models at very large scale. It ships Megatron Core, a composable library of transformer building blocks with tensor, pipeline, data, expert, and context parallelism plus mixed precision, alongside reference training scripts. It has been used to train many of the largest open models. Released under a custom NVIDIA open-source license.
Reviews (0)
Leave a Review
No reviews yet. Be the first to review!
Details
- Category
- AI Frameworks & Libraries
- Price
- Free
- Platform
- Local/Desktop
- Difficulty
- Expert (5/5)
- License
- Custom
- Minimum VRAM
- 16 GB
- Added
- Apr 3, 2026
Related Tools
Unified system for large-scale distributed training and inference.
Guidance language for controlling LLM generation with constraints, grammars, and JSON schemas.
Library for easily accessing and processing ML datasets.
JAX neural network library with PyTorch-like module system.
High-level deep learning API supporting JAX, TensorFlow, and PyTorch backends.
Gradient boosting library by Yandex with native categorical feature handling.