Classification

Types

• Binary: 2 classes
• Multi-class: K classes
• Multi-label: multiple labels per sample

Decision Boundary

• Separates classes in feature space
• Linear: hyperplane
• Non-linear: curves, complex shapes

Linear Classifier

• score = w · x + b
• If score > 0: class 1
• Weights define boundary

Sigmoid (Binary)

• σ(z) = 1 / (1 + e^(-z))
• Converts score to P(class=1)
• Range: (0, 1)

Logistic Regression

• P = sigmoid(w · x + b)
• Loss: binary cross-entropy
• Gradient: (p - y) × x

Softmax (Multi-class)

• softmax(z)ᵢ = exp(zᵢ) / Σexp(zⱼ)
• Outputs sum to 1
• Use with cross-entropy loss

Class Imbalance

• Majority class dominates
• Use resampling or class weights
• Evaluate with F1, not accuracy