It earns a place when
- Use domain knowledge to create stable, available and auditable features, then compare them against a simple raw-data baseline.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 7 · Lesson 54
Feature engineering and data leakage
Create useful inputs without allowing future or target information to enter training improperly.
Lesson goal
Create useful inputs without allowing future or target information to enter training improperly.
The core idea
Feature engineering is deciding what evidence the model gets. Leakage is giving it evidence that would not exist at decision time.
Mental model
Picture it this way. Feature engineering is deciding what evidence the model gets. Leakage is giving it evidence that would not exist at decision time. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Transformations such as logs, ratios, lags, counts and interactions change the representation. The transformation must be fitted and timed inside the same evaluation design as the model.
Worked example
A churn model can use the number of support tickets in the prior 30 days. It must not use the cancellation reason entered after churn.
When to use it
Do not make it the default when
- Do not engineer a feature from the full dataset before splitting, and do not silently use post-outcome fields because they improve validation.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Leakage can survive random cross-validation, especially through aggregates, duplicates, target encoding and user or account history.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. For a prediction time you choose, draw a cutoff line. Audit every feature by asking when it becomes available and who writes it.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What is leakage?
2. Why can a rolling feature leak?
3. What should every feature audit ask?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.