Handle rare positive classes without confusing resampling tricks with better evidence.
This is one focused step in the 60-lesson course. Use the retrieval check before moving on.
Lesson goal
Handle rare positive classes without confusing resampling tricks with better evidence.
The core idea
When positives are rare, a model can look accurate by ignoring them. The learning and evaluation design must make the scarce event visible.
Mental model
Picture it this way. When positives are rare, a model can look accurate by ignoring them. The learning and evaluation design must make the scarce event visible. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Class-weighted losses multiply errors by class-specific weights. Resampling changes the training distribution; evaluation should usually reflect the real deployment prevalence.
Worked example
For a one-percent defect rate, report precision-recall behaviour at the inspection capacity the factory can support, not just overall accuracy.
When to use it
It earns a place when
Use stratified or time-aware splits, appropriate metrics, class weights or carefully designed sampling, and threshold selection tied to cost.
You can evaluate it against a credible baseline.
Its output fits the decision and data constraints.
Do not make it the default when
Do not oversample before splitting, or assume a balanced training set means predicted probabilities remain calibrated for production prevalence.
A simpler model has not been tested.
The data or target definition is still unclear.
Failure modes
Watch for this. Synthetic minority examples can be unrealistic. Distribution shift and label delay can matter more than the imbalance method.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Train a baseline with class weights and one with a sampling strategy. Compare recall at a fixed alert volume and recalibrate if probabilities are used.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. Why can accuracy fail on rare events?
2. What does class weighting change?
3. Why must resampling happen inside training folds?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.