Distinguish candidate generation, ranking and recommendation metrics.
This is one focused step in the 60-lesson course. Use the retrieval check before moving on.
Lesson goal
Distinguish candidate generation, ranking and recommendation metrics.
The core idea
A recommender has two jobs: find a manageable set of plausible candidates, then order them for a particular user and context.
Mental model
Picture it this way. A recommender has two jobs: find a manageable set of plausible candidates, then order them for a particular user and context. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Collaborative filtering uses user-item interactions. Matrix factorisation approximates R as U V^T. Ranking losses focus on relative order rather than an exact rating.
Worked example
A shop retrieves a few hundred products from a catalogue, ranks them using user history and context, then applies availability, safety and diversity rules.
When to use it
It earns a place when
Use recommendation when interaction history and choice order matter, and measure both offline ranking quality and online outcomes.
You can evaluate it against a credible baseline.
Its output fits the decision and data constraints.
Do not make it the default when
Do not optimise clicks alone if the product goal is retention, satisfaction or safe discovery. Do not leak future interactions into training.
A simpler model has not been tested.
The data or target definition is still unclear.
Failure modes
Watch for this. Popularity bias, cold start, feedback loops, exposure bias and offline-online metric gaps can make a recommender worse while its metric rises.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Build a popularity baseline and a simple item-item or matrix-factorisation model. Evaluate Recall@K and inspect recommendations for novelty and coverage.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What is candidate generation for?
2. What does ranking care about?
3. What is cold start?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Recommenders. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.