It earns a place when
- Use it as a strong, interpretable classification baseline, especially with sparse or high-dimensional features.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 2 · Lesson 11
Logistic regression and generalised linear models
Understand logistic regression as a probability model for classification and see how GLMs extend linear predictors.
Lesson goal
Understand logistic regression as a probability model for classification and see how GLMs extend linear predictors.
The core idea
The model still adds weighted evidence, then passes it through a link function that turns an unconstrained score into a valid probability.
Mental model
Picture it this way. The model still adds weighted evidence, then passes it through a link function that turns an unconstrained score into a valid probability. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Logistic regression uses logit(p) = beta_0 + beta^T x, so p = 1/(1 + exp(-z)). The coefficients act on log-odds, not directly on probability.
Worked example
Predict whether a transaction is fraudulent from amount, merchant category and velocity features. The model outputs a probability before a business threshold decides whether to review it.
When to use it
Do not make it the default when
- Do not assume a linear decision boundary is enough for complex interactions, and do not interpret odds ratios as causal effects without a causal design.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Complete separation can make coefficients unstable. Poor scaling, leakage and a default threshold can hide real operational problems.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Fit LogisticRegression in a pipeline. Compare its probability calibration and precision-recall curve with its default 0.5 decisions.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does logistic regression model naturally?
2. What does changing the threshold change?
3. Why can a coefficient be hard to interpret causally?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
An Introduction to Statistical Learning. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.