Calculus for ML

Derivatives

• Rate of change: f'(x) = df/dx
• Positive = increasing, negative = decreasing
• Zero = potential min/max

Common Derivatives

• x^n → n*x^(n-1)
• e^x → e^x
• ln(x) → 1/x

Chain Rule

• (f(g(x)))' = f'(g(x)) * g'(x)
• Foundation of backpropagation
• Gradients flow backward

Partial Derivatives

• Derivative w.r.t. one variable
• Others held constant
• ∂f/∂x, ∂f/∂y

Gradient

• ∇f = [∂f/∂x₁, ∂f/∂x₂, ...]
• Points toward steepest increase
• Negate for descent

Gradient Descent

• x = x - lr * ∇f(x)
• Learning rate controls step size
• Repeat until convergence

Numerical Gradients

• (f(x+h) - f(x-h)) / 2h
• Slow but useful for debugging
• Always verify analytical gradients