DSA Studio

Transformer Block Checkpoint

Test your understanding of transformer architecture.

Get every question correct to pass this checkpoint. You can review feedback, revise your answers, and submit again.

1. In a pre-norm residual attention block, which expression matches the intended order?

Select one answer.

2. The v1 block accepts num_heads but its exercise uses one SelfAttention head. What should a learner infer?

Select one answer.

3. A residual stream has width 6 and an FFN expands to 24. What output width must it return before residual addition?

Select one answer.

4. For y=x+f(x), the direct skip contributes what multiplier to dy/dx?
5. At inference, replacing a token at a future position changes an earlier output in a causal decoder. What does this indicate?

Select one answer.

Join Discord