Neural Network Abstractions

Why Module?

We want layers that:

• Are callable (__call__ -> forward)
• Track parameters automatically
• Nest cleanly

Auto-registration

__setattr__ registers:

• Value -> _parameters
• Module -> _modules
• list/tuple -> recursive register

Parameters API

• parameters() returns all Values recursively
• named_parameters() yields hierarchical names
• train() / eval() toggle modes

Linear layer

y = xW^T + b

• weight shape: (out_features, in_features)
• bias shape: (out_features,)
• Xavier init: k = 1/sqrt(in_features)

Sequential

model = Sequential(
    Linear(2, 16), Tanh(),
    Linear(16, 1)
)

Feeds output of each module into the next.

Activations

• Tanh, ReLU, Sigmoid
• No parameters, still Modules