Synthetic Data Kit

Fine-tuning data preparation is the problem Synthetic Data Kit attacks: it is Meta's official command line tool for turning raw documents into training examples for large language models. The workflow runs in four stages, ingest, create, curate, and save-as. Ingest parses PDF, HTML, DOCX, PPTX, TXT, and YouTube transcript sources; create generates question-answer pairs, chain-of-thought reasoning traces, or summaries using a local LLM backend, typically served through vLLM; curate filters the output with a Llama model acting as judge to keep only high quality examples; save-as exports the survivors to Alpaca, ChatML, or plain JSONL formats ready for fine-tuning pipelines. Installation is a single pip install of the synthetic-data-kit package on Python 3.10 or newer, with behavior controlled through a YAML config file. The code is MIT licensed, free for commercial use, and maintained under the official meta-llama GitHub organization, which positions it as the data preparation step in Llama fine-tuning workflows; the repository holds around 1,600 stars.