It earns a place when
- Use autoencoders for compression, denoising and anomaly signals, and VAEs when a smooth probabilistic latent space is valuable.
- You can evaluate it against a credible baseline.
- Its output fits the decision and data constraints.
Machine learning models · Phase 4 · Lesson 35
Autoencoders and variational autoencoders
Compare reconstruction-based representation learning with probabilistic latent-variable generation.
Lesson goal
Compare reconstruction-based representation learning with probabilistic latent-variable generation.
The core idea
An autoencoder squeezes an input through a bottleneck and tries to rebuild it. A VAE makes the bottleneck a structured distribution that can be sampled.
Mental model
Picture it this way. An autoencoder squeezes an input through a bottleneck and tries to rebuild it. A VAE makes the bottleneck a structured distribution that can be sampled. The important question is what assumption this picture makes, and whether that assumption fits the data.
Mathematical core
An autoencoder minimises reconstruction loss. A VAE adds a KL divergence term that keeps the approximate latent distribution near a prior, usually a standard normal.
Worked example
Compress product images into a latent vector. An autoencoder may reconstruct them well; a VAE can sample nearby latent points and produce varied images with smoother structure.
When to use it
Do not make it the default when
- Do not treat a low reconstruction loss as proof that the representation is useful for every downstream task.
- A simpler model has not been tested.
- The data or target definition is still unclear.
Failure modes
Watch for this. VAEs can blur outputs. Latent dimensions can be hard to interpret, and a decoder can learn shortcuts that ignore meaningful structure.
When a result looks surprisingly good, inspect the split, target timing, error slices and data-generating process before celebrating.
Practice
Use this as a small experiment rather than a recipe to copy blindly. Change one thing, record the result and explain the change.
Do this. Train a tiny autoencoder on a simple image dataset. Compare reconstruction error with a VAE and inspect interpolations between two latent codes.
Retrieval check
Answer from memory first. The buttons reveal feedback, but the durable step is explaining why.
1. What does an autoencoder optimise directly?
2. What does a VAE add?
3. Why can low reconstruction error mislead?
Transfer prompt. Describe one real problem where this model or idea would be a sensible candidate. Name the target, the main risk and the metric you would inspect.
Primary source
Deep Learning. Use the source for the deeper treatment after you can explain the lesson's core idea without looking.