Machine learning models · Lesson 1

What a machine-learning model is

A model is not a magic brain. It is a mathematical structure fitted to examples so it can produce useful outputs for new cases.

Lesson goal

After this lesson, you should be able to name the main parts of a machine-learning system, distinguish the model from the training algorithm, classify the four broad learning settings, and explain why unseen data is the real test.

Course rule: we will learn model families by the decisions they support, not by memorising a list of fashionable names.

The core idea

A machine-learning system uses examples to find a pattern that is useful beyond the examples it saw during training.

A useful first approximation is:

prediction = model(input; learned parameters)

In mathematical shorthand, this is often written as ŷ = f_θ(x). The input is x, the model is f, the learned parameter values are θ, and the prediction is ŷ.

For a spam filter, the input might include the sender, words, links and message metadata. The output might be a probability such as “92% spam”. A decision rule can then turn that probability into an action.

Probabilistic models may output a whole distribution rather than one answer. The central idea is unchanged: input goes in, a fitted mathematical object produces an output.

The system's pieces

Term	Meaning	Spam example
Dataset	Examples used for learning.	A collection of emails.
Feature	An input signal available to the model.	Number of links or sender domain.
Target	The answer we want to predict.	Spam or not spam.
Model	The mathematical form used to produce predictions.	A classifier.
Parameter	A value learned from the training data.	How strongly a feature affects the result.
Hyperparameter	A setting chosen by the developer.	Tree depth or learning rate.
Loss	A score describing how wrong a prediction is.	A penalty for a wrong classification.
Algorithm	The procedure used to fit the model.	Gradient descent or tree splitting.

Important distinction: a decision tree is a model. The procedure used to construct the tree is an algorithm. A neural network is a model. Backpropagation and gradient descent are training algorithms.

Four learning settings

Supervised learning

The examples include target answers. Typical tasks include predicting prices, classifying images and detecting fraud.

Unsupervised learning

No target answer is supplied. The model looks for groups, structure, compressed representations or unusual cases.

Self-supervised learning

The system creates a prediction task from unlabelled data, such as hiding a word and learning to predict it.

Reinforcement learning

An agent interacts with an environment and learns which actions tend to produce better future rewards.

Self-supervised learning still uses a target. The difference is that the target is generated from the data rather than manually supplied by a person.

Model families are different assumptions

Every model makes a bet about what useful patterns look like. This built-in preference is often called an inductive bias.

Family	Underlying bet	Useful first situation
Linear models	Effects combine in a relatively simple way.	Small or medium tabular problems and interpretable baselines.
Trees and ensembles	Useful decisions can be represented as feature splits.	Tabular data with nonlinear interactions.
Nearest neighbours	Similar cases should have similar outputs.	Problems where similarity has a meaningful definition.
Probabilistic models	Uncertainty and hidden relationships should be represented explicitly.	Limited data, sequences and uncertain decisions.
Neural networks	Layered transformations can learn useful representations.	Images, audio, text and high-dimensional data.

A more complex model is not automatically a better model. A boosted-tree model may beat a neural network on ordinary business data while being cheaper and easier to explain.

Training and inference

Training

Training is the fitting stage. The model makes predictions, a loss function measures error, and an optimisation procedure adjusts the parameters. This repeats across many examples.

Inference

Inference is using the fitted model on new input. In Python, a library might expose this as:

model.fit(X_train, y_train)
prediction = model.predict(X_new)

The first line represents training. The second represents inference. A model can perform well during training and still fail during inference if it has memorised the training examples rather than learned a general pattern.

Generalisation and failure modes

Generalisation means performing well on new data from the same kind of environment as the training data. This is the real purpose of machine learning.

Overfitting happens when a model learns the training data too closely, including noise and accidental details. The usual pattern is strong training performance and weak test performance.

Underfitting happens when a model is too limited to capture the useful pattern. Training performance and test performance are both poor.

Data leakage happens when information enters training that would not be available at the time of the real prediction. For example, predicting cancellation using a field recorded only after a customer has cancelled produces an impressive but unusable model.

Test-set principle: training performance tells you how well the model fits what it has seen. Test performance gives evidence about how it may behave on new cases.

How to choose a model

Start with the problem, not the fashionable architecture.

What output do you need: a number, category, probability, ranking, generated object or action?
What kind of data do you have: table, text, image, audio, time series or graph?
What is the cost of each type of error?
What constraints matter: latency, cost, privacy, interpretability or reliability?
What is the simplest credible baseline?

A sensible progression is often:

simple rule → linear model → tree-based model → neural network → specialised model

Move towards the right only when the simpler approach is not good enough for the actual requirement.

Practice

Choose one prediction problem from your work or life. Write down the input features, target, prediction, loss and one reason a simple baseline might be better than a large neural network.

Do this. Explain your choice in five sentences without using a model name as the explanation.

Retrieval check

Answer these without reopening the lesson if you can. The point is recall, not recognition.

1. Which statement best distinguishes a model from an algorithm?

2. Predicting a labelled house price is primarily which setting?

3. A model scores well on training data but poorly on test data. What is the likely diagnosis?

4. Why might a tree-based model be a sensible first baseline for tabular customer data?

Sources

Google Machine Learning Crash Course, especially the introductory model, loss, generalisation and overfitting modules.
scikit-learn User Guide, especially supervised learning, unsupervised learning, model selection and common pitfalls.
Deep Learning by Goodfellow, Bengio and Courville, especially chapters 1 and 5 for the broader map.

Next lesson: data, targets, loss functions and why evaluation is harder than it first appears.