Loss Functions

What is Loss?

• L(y_true, y_pred) → scalar
• Measures prediction error
• Training minimizes loss

MSE (Regression)

• MSE = (1/n) Σ(y - ŷ)²
• Penalizes large errors heavily
• Sensitive to outliers

MAE (Regression)

• MAE = (1/n) Σ|y - ŷ|
• Robust to outliers
• Same units as target

Binary Cross-Entropy

• BCE = -[y log(ŷ) + (1-y) log(1-ŷ)]
• For binary classification
• Punishes confident wrong predictions

Categorical Cross-Entropy

• CCE = -Σ yₖ log(ŷₖ)
• For multi-class classification
• Use with softmax outputs

Softmax

• Converts logits to probabilities
• All outputs in (0, 1), sum to 1
• softmax(x)ᵢ = exp(xᵢ) / Σexp(xⱼ)

Choosing Loss

• Regression: MSE, MAE
• Binary: Binary cross-entropy
• Multi-class: Categorical cross-entropy
• Loss encodes your goals