It earns a place when
- Use ensembles when complementary errors and the operational gain justify extra complexity.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 2 · Lesson 18
Ensembles, calibration and interpretation
Combine models responsibly and explain what their outputs do and do not mean.
Lesson goal
Combine models responsibly and explain what their outputs do and do not mean.
The core idea
An ensemble is useful when its members make different mistakes. Averaging similar mistakes adds cost without much new information.
Mental model
Picture it this way. An ensemble is useful when its members make different mistakes. Averaging similar mistakes adds cost without much new information. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Bagging averages models. Boosting adds sequential corrections. Stacking learns a second-level combiner, which must be trained without leaking base-model predictions.
Worked example
A risk system can average a calibrated logistic model with a tree ensemble, then choose a threshold using review capacity. Interpretation should inspect errors and local behaviour, not only one global ranking.
When to use it
Do not make it the default when
- Do not average incompatible scores without calibration, and do not present feature importance as a causal story.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Stacking leaks if the meta-model sees in-sample base predictions. Correlated models can give an illusion of diversity.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Compare two models' error sets. Calculate their disagreement rate, calibrate their probabilities, then decide whether an average helps.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. When does an ensemble usually help?
2. Why calibrate before combining probabilities?
3. How should feature importance be treated?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.