It earns a place when
- Use trees for nonlinear tabular relationships, mixed feature types and explanation through example paths.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 2 · Lesson 14
Decision trees
Read a decision tree as a sequence of feature-based splits and understand its strengths and limits.
Lesson goal
Read a decision tree as a sequence of feature-based splits and understand its strengths and limits.
The core idea
A tree asks a sequence of yes-or-no questions. Each split tries to make the remaining groups more alike in their target.
Mental model
Picture it this way. A tree asks a sequence of yes-or-no questions. Each split tries to make the remaining groups more alike in their target. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
A classification split can reduce impurity such as Gini impurity or entropy. Regression splits reduce within-node squared error.
Worked example
A credit-risk tree might first split on missed payments, then on income. The path is easy to describe, but a small data change can alter the chosen split.
When to use it
Do not make it the default when
- Do not let an unconstrained tree grow until every leaf contains one or two rows, and do not assume a readable path proves fairness.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Deep trees overfit. Small changes in data can produce a different structure, and high-cardinality identifiers can create spurious splits.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Fit shallow and deep DecisionTreeClassifier models. Compare train and validation curves, then inspect the exported tree.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does a tree split try to do?
2. Why can a deep tree overfit?
3. What is a danger of an identifier feature?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.