Understand an LLM as a next-token prediction system with learned representations and emergent capabilities.
This is one focused step in the 60-lesson course. Use the retrieval check before moving on.
Lesson goal
Understand an LLM as a next-token prediction system with learned representations and emergent capabilities.
The core idea
An LLM predicts the next token from context. Scale, data and training objectives can make that simple task support many downstream behaviours, but not guaranteed truth.
Mental model
Picture it this way. An LLM predicts the next token from context. Scale, data and training objectives can make that simple task support many downstream behaviours, but not guaranteed truth. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Given tokens x_1 to x_t, training maximises the sum of log P(x_t | x_<t). Tokenisation, context length, parameters and decoding all affect behaviour.
Worked example
A language model can complete a support reply, summarise a document or write code because these behaviours are represented in its learned distribution, not because it has a separate built-in module for each task.
When to use it
It earns a place when
Use LLMs for language generation, transformation, extraction and tool-mediated workflows when evaluation and safeguards match the risk.
You can evaluate it against a credible baseline.
Its output fits the decision and data constraints.
Do not make it the default when
Do not treat fluent output as evidence of correctness, grounding or understanding. Do not expose confidential data without a clear data path.
A simpler model has not been tested.
The data or target definition is still unclear.
Failure modes
Watch for this. Hallucination, prompt sensitivity, memorisation, bias, context limits and distribution shift can all appear despite high benchmark scores.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Use a small local or hosted model to compare greedy, temperature and top-p decoding on the same prompt. Record changes in factuality and diversity.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What is the core pretraining task of a causal LLM?
2. Why is fluency not proof of truth?
3. What does decoding affect?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Hugging Face LLM Course. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.