It earns a place when
- Use regularised linear models when features are numerous, correlated or noisy and you want a fast, transparent model.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 2 · Lesson 10
Ridge, Lasso and Elastic Net
Use regularisation to control coefficient size, multicollinearity and overfitting.
Lesson goal
Use regularisation to control coefficient size, multicollinearity and overfitting.
The core idea
Regularisation charges rent for large weights. Ridge spreads weight across correlated features; Lasso can set some weights exactly to zero.
Mental model
Picture it this way. Regularisation charges rent for large weights. Ridge spreads weight across correlated features; Lasso can set some weights exactly to zero. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Ridge minimises squared error plus alpha times the sum of squared coefficients. Lasso uses the sum of absolute coefficients. Elastic Net combines both penalties.
Worked example
With hundreds of correlated text or marketing features, ordinary regression can produce unstable weights. Ridge often stabilises them; Lasso can produce a sparse shortlist.
When to use it
Do not make it the default when
- Do not interpret a zero Lasso coefficient as proof that a variable has no relationship. Scaling and penalty selection matter.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. A penalty chosen on the test set leaks information. An unscaled feature can dominate the penalty for purely numerical reasons.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Standardise numeric features, compare Ridge, Lasso and ElasticNet with cross-validation, and inspect how the coefficients change as alpha varies.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does Ridge add to the objective?
2. What is Lasso known for?
3. Why standardise before regularisation?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.