It earns a place when
- Use MLPs when nonlinear interactions matter and you have enough data and tuning capacity.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 4 · Lesson 29
Perceptrons and multilayer perceptrons
Build the mental model from one linear threshold unit to a multilayer perceptron.
Lesson goal
Build the mental model from one linear threshold unit to a multilayer perceptron.
The core idea
A perceptron is a weighted vote. Layers of such units can transform the representation before the final prediction.
Mental model
Picture it this way. A perceptron is a weighted vote. Layers of such units can transform the representation before the final prediction. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
A unit computes activation(w^T x + b). With no nonlinear activation, stacked layers collapse into one linear transformation; nonlinearities create useful depth.
Worked example
A small MLP can learn XOR, which a single linear threshold cannot represent. On tabular data, its value depends on scale, data volume and regularisation.
When to use it
Do not make it the default when
- Do not assume a deeper MLP beats a tree ensemble on ordinary small tabular data.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Poor scaling, bad initialisation, too much capacity and weak validation can produce unstable or overfit networks.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Python practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Use sklearn MLPClassifier on a scaled toy dataset. Compare one hidden layer with no hidden layer and inspect validation behaviour.Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. Why can a single perceptron not solve XOR?
2. What does a hidden layer add?
3. Why do stacked linear layers need nonlinear activations?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Deep Learning. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.