It earns a place when
- Use layered evaluation before and after deployment, with a clear harm model and representative test cases.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 5 · Lesson 45
Foundation-model evaluation and safety
Evaluate broad models across capabilities, risks, reliability and the actual use case.
Lesson goal
Evaluate broad models across capabilities, risks, reliability and the actual use case.
The core idea
A foundation model is not one score. It is a bundle of abilities and failure modes whose importance depends on where it is used.
Mental model
Picture it this way. A foundation model is not one score. It is a bundle of abilities and failure modes whose importance depends on where it is used. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Evaluation can combine task metrics, calibration, robustness tests, human preference, red-team cases and risk-weighted operational measures. A benchmark is a sample, not the whole distribution.
Worked example
A support model may score well on general QA but fail on refusal boundaries, private-data handling, current policy retrieval or escalation to a human.
When to use it
Do not make it the default when
- Do not select a model from one leaderboard, and do not treat safety filters as a substitute for access controls and product design.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Benchmark contamination, narrow test sets, evaluator disagreement, distribution shift and unknown unknowns can hide serious failures.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Write an evaluation matrix for one use case: capability, risk, metric, test data, acceptable threshold, owner and response if it fails.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. Why is one benchmark score insufficient?
2. What is red teaming for?
3. Why are access controls still needed?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Stanford HELM. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.