micrograd

Andrej Karpathy wrote micrograd as a teaching artifact: an automatic differentiation engine where every number is a Value object that remembers how it was produced, so calling backward() walks the resulting graph in reverse and fills in gradients with the chain rule. The engine file is roughly a hundred lines and the companion nn module, which defines neurons, layers and a multilayer perceptron, is about half that. Because it works on single scalars rather than tensors it is far too slow for real workloads, yet small enough to read in one sitting and check by hand. The repository includes a notebook that trains a two-layer classifier on the moons toy dataset with a max-margin loss and plain SGD, another that draws the computation graph with Graphviz, and unit tests that compare gradients against PyTorch. Install with pip install micrograd or copy the two files. The code has been essentially frozen since 2020 by design; a 2026 update points readers to microgpt, a single-file GPT built on a faster take on the idea. MIT licensed. Ideal for students who want to understand backpropagation before relying on a full framework.