It earns a place when
- Use structured models when output dependencies and valid configurations are central.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 6 · Lesson 53
Structured prediction and conditional random fields
Understand predictions where outputs depend on one another, such as sequences of labels.
Lesson goal
Understand predictions where outputs depend on one another, such as sequences of labels.
The core idea
In structured prediction, the answer is a configuration, not a bag of independent labels. The model scores both local evidence and compatibility between neighbouring outputs.
Mental model
Picture it this way. In structured prediction, the answer is a configuration, not a bag of independent labels. The model scores both local evidence and compatibility between neighbouring outputs. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
A linear-chain CRF scores a label sequence using feature weights for observations and transitions, then normalises over all sequences. Dynamic programming computes marginals and the best path.
Worked example
Named-entity recognition labels a sentence as a sequence of person, organisation and location tags. A transition model can discourage impossible I-ORG tags after O.
When to use it
Do not make it the default when
- Do not predict each position independently if neighbouring labels impose important constraints, and do not assume a CRF solves poor tokenisation or annotation.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Label bias, long-range dependencies, exposure to inconsistent labels and approximate inference can cause errors.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Write a toy sequence with three tokens and two labels. Enumerate all label sequences, assign local and transition scores, and find the best path.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What makes a prediction structured?
2. What does a CRF transition feature represent?
3. Why use dynamic programming in a linear-chain CRF?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Probabilistic Machine Learning. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.