Autograd: Scalar Reverse-Mode

1 / 5

Value node

  • data, grad
  • _prev parents
  • _op label
  • _backward closure

Forward pass builds a DAG.

2 / 5

Local derivatives

  • Define gradient flow when you create out
  • Use += for accumulation
3 / 5

Backward pass

  1. Build topo order with DFS
  2. Set self.grad = 1.0
  3. Traverse in reverse and call _backward
4 / 5

Gradients accumulate

Zero grads before each new backward pass.

5 / 5
Use arrow keys or click edges to navigate. Press H to toggle help, F for fullscreen.