PCA

The Goal

  • Reduce dimensions
  • Preserve maximum variance
  • Find natural axes of data

Algorithm Steps

  1. Center data (subtract mean)
  2. Compute covariance matrix
  3. Eigendecomposition
  4. Sort by eigenvalue
  5. Project to top K components

Principal Components

  • Eigenvectors of covariance matrix
  • Orthogonal to each other
  • Ordered by variance captured

Explained Variance

  • eigenvalue / sum(eigenvalues)
  • Keep 95% of variance typically
  • Plot cumulative to choose K

Projection

  • X_new = X_centered @ components
  • Reconstruction: X_approx = X_new @ components.T

Important Notes

  • Standardize first!
  • Only linear relationships
  • Sensitive to outliers

When to Use

  • Visualization (K=2 or 3)
  • Noise reduction
  • Speed up training
  • Decorrelate features
1 / 1
Use arrow keys or click edges to navigate. Press H to toggle help, F for fullscreen.