It earns a place when
- Use GMMs for soft assignments, density estimation and elliptical clusters with enough data to estimate covariance.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 3 · Lesson 20
Gaussian mixture models
Understand soft clustering and density estimation with Gaussian mixtures.
Lesson goal
Understand soft clustering and density estimation with Gaussian mixtures.
The core idea
Each data point can belong partly to several bell-shaped components. The model explains the data as a mixture rather than forcing a hard label.
Mental model
Picture it this way. Each data point can belong partly to several bell-shaped components. The model explains the data as a mixture rather than forcing a hard label. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
A mixture density is p(x) = sum_k pi_k N(x | mu_k, Sigma_k). Expectation-maximisation alternates between membership responsibilities and parameter updates.
Worked example
Customer behaviour may lie between two segments. A mixture model can assign 0.7 membership to one component and 0.3 to another, which is more honest than a hard boundary.
When to use it
Do not make it the default when
- Do not trust a high likelihood with too many components, or use full covariance in a tiny dataset without regularisation.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Components can collapse or become non-identifiable. Different initialisations can find different local optima.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Fit GaussianMixture with several component counts and covariance types. Inspect responsibilities, not just the most likely component.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What makes GMM clustering soft?
2. What is EM doing?
3. What can too many components cause?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Probabilistic Machine Learning. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.