DSA Studio

Multi-Head Attention and Transformer Blocks Checkpoint

Apply the module contracts to concrete traces and diagnose failures.

Get every question correct to pass this checkpoint. You can review feedback, revise your answers, and submit again.

1. For D=12 and H=3, what is each head score denominator?

Select one answer.

2. Why project full input vectors before splitting the projected feature dimension?

Select one answer.

3. Every parameter in attention and FFN branches is zero. What should a pre-norm block return?

Select one answer.

4. LayerNorm forward values match, but its input derivatives fail finite differences. What is a likely cause?

Select one answer.

Join Discord