Neural Network Abstractions

1 / 7

Why Module?

We want layers that:

  • Are callable (__call__ -> forward)
  • Track parameters automatically
  • Nest cleanly
2 / 7

Auto-registration

__setattr__ registers:

  • Value -> _parameters
  • Module -> _modules
  • list/tuple -> recursive register
3 / 7

Parameters API

  • parameters() returns each trainable scalar identity once, including aliased children
  • named_parameters() uses the first traversal path for each unique identity
  • train() / eval() toggle modes; they do not disable gradients
4 / 7

Linear layer

y = xW^T + b

  • weight shape: (out_features, in_features)
  • bias shape: (out_features,)
  • Fan-in uniform init (not Glorot/Xavier): k = 1/sqrt(in_features)
5 / 7

Sequential

model = Sequential(
    Linear(2, 16), Tanh(),
    Linear(16, 1)
)

Feeds output of each module into the next.

6 / 7

Activations

  • Tanh, ReLU, Sigmoid
  • No parameters, still Modules
7 / 7
←/→ or click edges to navigate · ? help · N notes · F fullscreen