MLP Language Model Checkpoint

Apply the module contracts to concrete traces and diagnose failures.

Get every question correct to pass this checkpoint. You can review feedback, revise your answers, and submit again.

1. An MLP uses T=3, D=2, hidden width 4, vocabulary 5. What are the first weight matrix and output logit shapes?

Select one answer.

(4,6) weights and (3,5) logits (4,6) weights and (5,) logits (4,2) weights and (5,) logits (6,4) weights and (3,5) logits

2. Why can a concatenating MLP distinguish reversed contexts without a positional embedding table?

Select one answer.

Token IDs automatically encode position Mean cross-entropy preserves input order by itself Different concatenation slots use different first-layer weight columns The embedding table allocates a new row per occurrence

3. block_size=3 but the generation seed has only one ID. What does this pack require?

Select one answer.

Pad with token zero without reporting it Repeat the ID until the window is full Change the first linear layer to accept one token Reject the seed because no padding convention is defined

4. For B MLP contexts, why are target IDs shaped (B,) instead of (B,T)?

Select one answer.

There is one target per hidden feature Each complete context predicts one subsequent token The model predicts the whole vocabulary as labels Only the first context is trained

Auto-advance on pass

Open access learning · Go · Python · C · SQL · No external dependencies

Transparency · Support this project

Join Discord