OpenELM

OpenELM stands out as a rare fully open release from Apple: language models at 270M, 450M, 1.1B, and 3B parameters, each in pretrained and instruction-tuned form, published in April 2024 together with the complete CoreNet training framework, YAML-based recipes, and checkpoints. The models use a layer-wise scaling strategy that allocates parameters non-uniformly across transformer layers for better accuracy per parameter, and were pretrained on roughly 1.8T tokens drawn from RefinedWeb, deduplicated PILE, and subsets of RedPajama and Dolma v1.6. CoreNet, at about 7k GitHub stars, includes MLX examples for running OpenELM efficiently on Apple Silicon, which signals the on-device intent behind the sizes. Weights are distributed on Hugging Face under the Apple Sample Code License rather than a standard open-source license, so commercial terms need direct review, though research use is straightforward. The release matters mostly for reproducibility: training code, data composition, and evaluation are all documented, making it a solid base for studying small-model training end to end.