DSA Studio

Bigram Language Model Checkpoint

Apply the module contracts to concrete traces and diagnose failures.

Get every question correct to pass this checkpoint. You can review feedback, revise your answers, and submit again.

1. A zero logit row for two tokens trains on target 1 with learning rate 0.1. What is the new row after one SGD step?

Select one answer.

2. Logits are [1000,-1000], target 1. Which computation preserves the correct loss near 2000?

Select one answer.

3. Duplicating every transition in a batch with mean cross-entropy should do what to its gradient?

Select one answer.

4. The data contains abx and cby. Why cannot more bigram training learn different predictions after those two b tokens?

Select one answer.

Join Discord