01 · Abstract
We present micrograd, a minimal automatic differentiation library implementing scalar-valued backpropagation. The library contains an approximately 150-line implementation of an autograd engine supporting arithmetic operations plus tanh, relu, and exp. A small neural network library built on top allows training MLPs with stochastic gradient descent. We demonstrate convergence on a 2D moon-shaped binary classification dataset, reducing loss from ~5 to near 0 over 100 iterations.