It earns a place when
- Use PCA for visualisation, compression, denoising and reducing correlated numeric features.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 3 · Lesson 23
Principal component analysis
Use PCA for variance-aware linear compression and understand what its components mean.
Lesson goal
Use PCA for variance-aware linear compression and understand what its components mean.
The core idea
Rotate the coordinate system so the first axis points along the greatest remaining spread, then keep the first few axes.
Mental model
Picture it this way. Rotate the coordinate system so the first axis points along the greatest remaining spread, then keep the first few axes. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
PCA finds eigenvectors of the covariance matrix, or singular vectors of centred data. The explained-variance ratio describes how much variance each component captures.
Worked example
Compress hundreds of correlated sensor readings into ten components before visualising or modelling. The components are mixtures of original features, not automatically meaningful causes.
When to use it
Do not make it the default when
- Do not fit PCA before the split, use it without scaling when units differ, or assume maximum variance equals maximum predictive value.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Components can be hard to interpret. PCA is linear and unsupervised, so low reconstruction error does not guarantee downstream usefulness.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Standardise a feature matrix, fit PCA inside a pipeline, plot cumulative explained variance and compare a classifier before and after compression.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does the first principal component maximise?
2. Why fit PCA inside a pipeline?
3. What is a PCA component?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.