Variational Autoencoders and GANs

Week 14 · November 24, 2026

Latent variable models, the evidence lower bound, adversarial training, and mode collapse.

Reading Prince, Understanding Deep Learning, Chapters 15 and 17

These notes are still being written. The reading above is authoritative until this banner disappears.

1 Learning Objectives

By the end of this week you should be able to:

1. Explain what a latent variable model is and why the likelihood is intractable.
2. Derive the evidence lower bound and interpret its two terms.
3. Describe the reparameterization trick and why it is necessary.
4. Describe the GAN minimax objective and the roles of generator and discriminator.
5. Explain mode collapse and training instability in GANs.
6. Compare VAEs and GANs on sample quality, diversity, and training stability.

import numpy as np
import matplotlib.pyplot as plt
import torch

torch.manual_seed(774)
np.random.seed(774)

plt.rcParams.update({
    "figure.figsize": (7, 4),
    "figure.dpi": 130,
    "axes.spines.top": False,
    "axes.spines.right": False,
    "axes.grid": True,
    "grid.alpha": 0.25,
    "font.size": 10,
    "legend.frameon": False,
})

NDSU_GREEN = "#005643"
NDSU_GOLD  = "#FFC72C"
ACCENT     = "#C1121F"

---

2 Latent Variable Models

TODO

3 The Evidence Lower Bound

TODO

4 The Reparameterization Trick

TODO

5 Training and Sampling from a VAE

TODO

6 Generative Adversarial Networks

TODO

7 Mode Collapse and Training Instability

TODO

8 Comparing VAEs and GANs

TODO

9 Summary

TODO

---

10 Exercises

Problem numbers refer to Prince, Chapters 15 and 17.

1. TODO

2. TODO

3. Computational. TODO

---

11 Further Reading

• Prince, Understanding Deep Learning, Chapters 15 and 17.
• TODO