Self-Attention Checkpoint

Trace score/value mixing, exact causal exclusion, stability, and gradient dependencies. Select one answer per multiple-choice question.

Get every question correct to pass this checkpoint. You can review feedback, revise your answers, and submit again.

1. At zero-based query position 2 in a causal sequence of length 5, which keys may receive nonzero weight?

Select one answer.

Only 2 0, 1, and 2 2, 3, and 4 All five

2. Allowed attention scores are [0,0] with values [2,6]. What scalar output follows?

Select one answer.

2 6 8 4

3. A future score is 10^12 and a mask adds -10^9. What is the masking defect?

Select one answer.

The finite penalty can leave the future score dominant The future weight is guaranteed zero The query dimension becomes negative Softmax will always return equal weights

4. Dense attention stores T by T scores. Doubling T multiplies the score count by what factor? Enter an integer.

5. If head dimension is 9, what divisor does scaled dot-product attention use?

Select one answer.

1/9 9 81 3

6. A causal row may use only its first of two positions. Full-row softmax gave [0.5,0.5], and values are [2,6]. A learner zeroes the second weight without renormalizing. What output do they get, and what should it be?

Select one answer.

They get 2; it should be 4 They get 1; it should be 2 They get 4; it should be 6 They get 0; it should be 2

7. Why does Value(math.exp(score.data-m)) break the softmax backward pass?

Select one answer.

The maximum shift is too small Exponentials must be negative It normalizes across queries It creates a new leaf instead of preserving the score-to-exponential computation graph

8. All scalar projections are identity. For inputs [1,10^12], backpropagate only from the first causal output. What input gradients are required?

Select one answer.

[1,0] [0,1] [0.5,0.5] [1,1]

Auto-advance on pass

Join Discord