It earns a place when
- Use it for a fast, interpretable baseline when additive effects are plausible and the target is continuous.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 2 · Lesson 9
Linear regression
Understand linear regression as a weighted additive model and a baseline for continuous targets.
Lesson goal
Understand linear regression as a weighted additive model and a baseline for continuous targets.
The core idea
Each feature contributes a signed amount. The model is a spreadsheet of weights plus an intercept.
Mental model
Picture it this way. Each feature contributes a signed amount. The model is a spreadsheet of weights plus an intercept. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
The prediction is y_hat = beta_0 + beta_1 x_1 + ... + beta_p x_p. Ordinary least squares chooses coefficients that minimise squared residuals.
Worked example
Predict rent from floor area, bedrooms and distance to a station. Each coefficient describes an average change while the other included features are held constant in the model.
When to use it
Do not make it the default when
- Do not expect it to discover sharp thresholds or complex interactions without engineered features.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Correlated features make coefficients unstable. Outliers can dominate squared loss, and extrapolation beyond the data can be dangerous.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Fit LinearRegression to a small table, inspect coefficients, and plot residuals against predictions. Look for curvature or unequal spread.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does a linear coefficient represent?
2. What does ordinary least squares minimise?
3. What can residual curvature suggest?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
An Introduction to Statistical Learning. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.