Compare two representation methods that seek different structures from the same data.
This is one focused step in the 60-lesson course. Use the retrieval check before moving on.
Lesson goal
Compare two representation methods that seek different structures from the same data.
The core idea
PCA finds directions of variance. ICA looks for statistically independent sources. NMF builds additive parts using non-negative quantities.
Mental model
Picture it this way. PCA finds directions of variance. ICA looks for statistically independent sources. NMF builds additive parts using non-negative quantities. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
ICA seeks a transformation whose components are as independent as possible. NMF approximates X as W H with W and H constrained to be non-negative.
Worked example
For mixed audio, ICA can seek separate source signals. For document-term counts, NMF can produce additive topic-like parts that are easier to inspect than signed components.
When to use it
It earns a place when
Use ICA when independent sources are plausible, and NMF for non-negative counts, images or additive parts-based representations.
You can evaluate it against a credible baseline.
Its output fits the decision and data constraints.
Do not make it the default when
Do not treat components as unique truths. Sign, scale, ordering and local optima can make interpretations unstable.
A simpler model has not been tested.
The data or target definition is still unclear.
Failure modes
Watch for this. Noise, preprocessing and the chosen component count affect the result. NMF can get stuck in local minima.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Run FastICA on synthetic mixed signals and NMF on a small non-negative document matrix. Compare reconstruction and interpretability.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does ICA seek?
2. Why is NMF suited to count data?
3. Why are components not automatically unique truths?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.