It earns a place when
- Use it for high-dimensional text or small datasets where speed and a strong baseline matter.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 2 · Lesson 13
Naive Bayes
Understand conditional independence as a useful simplifying assumption for fast probabilistic classification.
Lesson goal
Understand conditional independence as a useful simplifying assumption for fast probabilistic classification.
The core idea
Naive Bayes counts how evidence for each feature shifts the odds of a class, pretending the features are conditionally independent once the class is known.
Mental model
Picture it this way. Naive Bayes counts how evidence for each feature shifts the odds of a class, pretending the features are conditionally independent once the class is known. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
Bayes' rule gives P(class | x) proportional to P(class) times the product of P(x_j | class). Log probabilities turn the product into a sum.
Worked example
For email classification, word counts provide evidence for spam or ham. The independence assumption is false, but the classifier can still rank documents effectively.
When to use it
Do not make it the default when
- Do not read the independence assumption as a claim that words are truly unrelated, and do not expect probability calibration without checking it.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. Unseen feature values can make a probability zero without smoothing. Correlated features can cause overconfident evidence.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Train MultinomialNB on a bag-of-words matrix. Compare it with logistic regression and inspect errors involving negation or word order.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What assumption makes Naive Bayes naive?
2. Why use log probabilities?
3. What does smoothing address?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
scikit-learn User Guide. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.