It earns a place when
- Use Bayesian models when uncertainty, prior knowledge, small samples or sequential updating matter.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 3 · Lesson 26
Bayesian modelling
Use priors, likelihoods and posteriors to represent uncertainty and update beliefs with evidence.
Lesson goal
Use priors, likelihoods and posteriors to represent uncertainty and update beliefs with evidence.
The core idea
Bayesian inference starts with a belief about plausible parameters, then reweights those possibilities according to how well they explain the data.
Mental model
Picture it this way. Bayesian inference starts with a belief about plausible parameters, then reweights those possibilities according to how well they explain the data. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Posterior is proportional to likelihood times prior. A posterior predictive distribution integrates over parameter uncertainty rather than plugging in one estimate.
Worked example
A small hospital estimates a treatment effect. A weakly informative prior can prevent extreme estimates while the data accumulates, and the posterior shows remaining uncertainty.
When to use it
Do not make it the default when
- Do not treat a prior as a way to force a preferred answer, or confuse a credible interval with a repeated-sampling confidence statement.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Poor priors can dominate scarce data. Complex posteriors can be hard to sample or approximate correctly.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Write a Beta-Binomial update for a conversion rate. Compare a prior with 2 successes and 2 failures against 200 new observations.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does a prior represent?
2. What does the posterior combine?
3. What does a posterior predictive include?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Probabilistic Machine Learning. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.