Vectors and Distance

One pair, three comparisons

For [1,2] and [4,6]:

• L2 distance 5; L1 distance 7.
• Cosine approximately 0.9923.
• Same direction can coexist with different magnitude.

Scaling changes the question

Query [0,0], A=[1,100], B=[2,0].

• Raw L2 chooses B.
• Divide feature 2 by 100: A becomes [1,1], so A wins.
• Cosine also changes under coordinate-specific scaling.

Zero is a policy boundary

• Mathematical cosine needs nonzero norms.
• Exercise fallback: zero similarity, distance one.
• normalize(zero) stays zero.
• 1-cosine is not generally a mathematical metric.

Choose neighbors, then vote

Query [1.5,1.5], training [0,0],[1,1],[2,2],[3,3], labels 0,0,1,1.

• Sort (distance,label), then take k=3.
• Selected labels [0,1,0], so class 0 wins.
• Vote ties use the first occurrence in the sorted list.

Transfer without the worked table

• Construct cosine-one vectors with nonzero L2 distance.
• Change a feature scale and predict the winner.
• Verify the tie policy on a fresh example before coding.