Autograd Checkpoint

1. In a computational graph, the backward pass traverses nodes in:

Reverse topological order Forward topological order Random order Breadth-first order

2. Why do we use += instead of = when accumulating gradients?

A value might be used multiple times It's faster It prevents NaN It's required by Python

3. The derivative of tanh(x) is:

1 - tanh(x)^2 tanh(x)^2 1/cosh(x) sech(x)

4. What is the gradient of x at x=2 for the expression x^3?

5. Before calling backward(), we should:

Zero all gradients Initialize weights Compute forward pass Apply dropout