Trace the count matrix Train: red red blue | blue green | empty. Sorted columns: blue, green, red. Rows: [1,0,2], [1,1,0], [0,0,0]. Whitespace splitting preserves punctuation. 2 / 6
Separate TF from DF TF divides occurrence count by total document tokens. DF counts documents containing the token, once per document. In the example N=3 and DF=[2,1,1]. 3 / 6
Use the exact IDF rule IDF=[log(3/2),log(3),log(3)]. First row TF-IDF: [0.135155,0,0.732408]. An ubiquitous term gets log(1)=0. This exercise uses no final L2 normalization. 4 / 6
Transform a new document red gold with the saved vocabulary: BoW [0,0,1]. TF [0,0,0.5]: unknown gold still counts in length. TF-IDF [0,0,0.549306]. Do not refit on held-out documents. 5 / 6
Transfer to learned representations Reconstruct the sun/moon trace before revealing its answer. Test empty, repeated, ubiquitous, and unknown tokens. Counts lose order; learned embeddings introduce a new training signal. 6 / 6