It earns a place when
- Use a simple baseline, a strong conventional model and a specialised model only when the evidence justifies the progression.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 7 · Lesson 59
Choosing the simplest model that works
Make model selection a trade-off between predictive value, evidence, risk, cost and maintainability.
Lesson goal
Make model selection a trade-off between predictive value, evidence, risk, cost and maintainability.
The core idea
Complexity is a budget. Spend it only when a simpler model fails an important requirement and the extra complexity pays for itself.
Mental model
Picture it this way. Complexity is a budget. Spend it only when a simpler model fails an important requirement and the extra complexity pays for itself. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Compare models using a predeclared metric and uncertainty estimate, then include non-score costs such as training time, inference latency, explanation, calibration and monitoring burden.
Worked example
A boosted-tree model may match a neural network on a small table while being easier to validate and cheaper to serve. A neural model earns its place when it can learn signal the tree cannot.
When to use it
Do not make it the default when
- Do not choose by benchmark prestige or a tiny score difference that is smaller than validation noise.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Optimising one metric can hide operational costs, group failures, maintenance effort or a fragile data dependency.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Write a one-page model decision record comparing three candidates across score, error slices, calibration, latency, cost, interpretability and rollback risk.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. When should complexity increase?
2. Why compare validation uncertainty?
3. What is part of a model's real cost?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.