Mini-GPT Integration Checkpoint

Check target alignment, loss stability, causal dependencies, ownership, and generation. Select one answer per multiple-choice question.

Get every question correct to pass this checkpoint. You can review feedback, revise your answers, and submit again.

1. Source token IDs are [2,0,1,2]. Which is a valid length-three next-token input/target pair?

Select one answer.

Inputs [2,0,1], targets [2,0,1] Inputs [2,0,1], targets [0,1,2] Inputs [2,0,1], targets [0,1,1] Inputs [0,1,2], targets [2,0,1]

2. Logits are [1000,-1000] and the target is index 1. What is the cross-entropy loss to the nearest integer?

3. A block is reachable through a list and an alias. Gradients are correct, but parameters() returns its scalars twice. What happens in a plain SGD loop?

Select one answer.

Each duplicated scalar receives the update twice The duplicate values cancel their gradients Only parameter names are affected The computation graph automatically removes optimizer duplicates

4. The first token and its position stay fixed while future token IDs change. Which property should a causal MiniGPT satisfy?

Select one answer.

The first-position logits may change due to position embeddings Only the output matrix dimensions must match Every position must produce identical logits The first-position logits stay unchanged

5. V=3, D=4, M=5, one block. The block has 244 parameters. Embeddings add 12+20, final norm adds 8, and the output projection adds 12. How many unique parameters are there? Enter an integer.

6. Maximum context is 3 and generated IDs are [0,1,2,3,4]. Which IDs go into the next forward call, and which logit row supplies the next distribution?

Select one answer.

All five IDs; the first row [0,1,2]; the final row [2,3,4]; the final row [2,3,4]; all rows sampled together

7. A model reduces training loss on a short repeated string. What does this establish by itself?

Select one answer.

It generalizes to held-out text Its optimization can fit that training case; causal and gradient properties still need independent checks Every parameter is registered once Its generated language is reliable

8. The v1 constructor receives num_heads=2. What attention architecture does this exercise promise?

Select one answer.

Two independently learned heads Two raw feature slices Two transformer blocks A single attention head; the argument is retained for compatibility

Auto-advance on pass

Join Discord