Neural Network Abstractions

1 / 5

Module contract

  • __call__ -> forward
  • auto-register parameters
  • recursive nesting
  • train/eval flag
2 / 5

Auto-registration

__setattr__ registers:

  • Value -> _parameters
  • Module -> _modules
  • lists/tuples -> recursive
3 / 5

Linear layer

y = xW^T + b

  • Xavier init: k = 1/sqrt(in_features)
4 / 5

Sequential + activations

Sequential(Linear, Tanh, Linear, ...)

Activations are modules with no params.

5 / 5
Use arrow keys or click edges to navigate. Press H to toggle help, F for fullscreen.