Tulu 3

Tulu 3 is Ai2's fully open post-training suite: not just model weights but the complete recipe, decontaminated datasets, and training code needed to turn a base model into a competitive instruct model. The open-instruct repository, Apache-2.0 with about 3.8k stars, implements supervised fine-tuning, DPO, PPO, and RLVR, reinforcement learning with verifiable rewards, the technique the Tulu 3 paper introduced for math and precise instruction following, now trained through a fast GRPO implementation. Released in November 2024, the Tulu 3 checkpoints at 8B and 70B built on Llama 3.1 outperformed the corresponding Llama 3.1 Instruct models in Ai2's evaluations, and the pipeline also supports OLMo 2 and OLMo 3 bases. All SFT and preference datasets are published with decontamination against evaluation suites, which made the release the reference open post-training stack cited across subsequent work. Reproducing the full pipeline requires multi-GPU training infrastructure, but the datasets and individual recipes are routinely reused piecemeal for smaller fine-tuning projects, which is where most of its practical adoption happens.