It earns a place when
- Use SVMs for small or medium datasets, high-dimensional features and cases where a margin-based boundary is useful.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 2 · Lesson 17
Support vector machines and kernels
Understand maximum-margin classification and the kernel idea for nonlinear boundaries.
Lesson goal
Understand maximum-margin classification and the kernel idea for nonlinear boundaries.
The core idea
An SVM tries to leave the widest safe street between classes. A kernel lets it measure similarity in a richer feature space without explicitly building every feature.
Mental model
Picture it this way. An SVM tries to leave the widest safe street between classes. A kernel lets it measure similarity in a richer feature space without explicitly building every feature. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
The soft-margin objective trades margin width against hinge-loss violations. The kernel trick replaces dot products with a similarity function such as an RBF kernel.
Worked example
Classify two intertwined shapes. A linear boundary fails, while an RBF kernel can represent curved separation if its scale and regularisation are validated.
When to use it
Do not make it the default when
- Do not use a kernel SVM casually on millions of rows or unscaled features. Avoid interpreting the kernel as a causal explanation.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Training and prediction can scale poorly. Poor C or gamma choices produce underfitting or memorisation.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Scale a two-feature dataset, compare linear and RBF SVC, and draw the boundaries. Record how C changes the margin.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does the SVM margin represent?
2. What does a kernel provide?
3. Why scale features for an RBF SVM?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.