It earns a place when
- Use them for small datasets, smooth functions, active learning and uncertainty-aware regression.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 3 · Lesson 27
Gaussian processes
Understand Gaussian processes as distributions over functions with uncertainty that varies across the input space.
Lesson goal
Understand Gaussian processes as distributions over functions with uncertainty that varies across the input space.
The core idea
Instead of choosing one fixed curve, place a probability distribution over possible curves. Nearby inputs can be correlated through a kernel.
Mental model
Picture it this way. Instead of choosing one fixed curve, place a probability distribution over possible curves. Nearby inputs can be correlated through a kernel. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
A Gaussian process is defined by a mean function and covariance kernel. The kernel determines how observations at x and x-prime co-vary.
Worked example
For expensive laboratory measurements, a Gaussian process can predict where a response is likely high and show wider uncertainty where observations are sparse.
When to use it
Do not make it the default when
- Do not expect standard Gaussian processes to scale naively to millions of rows or represent every sharp discontinuity well.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Kernel choice and noise assumptions drive results. Poorly estimated uncertainty is as dangerous as poor mean prediction.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Fit a GaussianProcessRegressor to a noisy one-dimensional function. Plot the mean and 95 percent interval, then sample points where uncertainty is high.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What is a Gaussian process a distribution over?
2. What does a kernel control?
3. Where is GP uncertainty often wider?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Probabilistic Machine Learning. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.