IME 775

Mathematical Foundations of Deep Learning

Interactive Visualizations

Week 4: Optimization Fundamentals

📉

Gradient Descent

Animate optimization steps

📐

Taylor Series

Compare approximations

🗺️

Level Contours

Explore contour plots

⚖️

Convexity

Convex vs non-convex

Week 5: Linear Algebraic Tools in ML

📊

Quadratic Forms

Level sets Q(x) = xᵀAx

✅

Positive Definiteness

Classify matrices

📏

Matrix Norms

Spectral vs Frobenius

🎯

PCA

Principal components

🔢

SVD

Low-rank approximation

📄

LSA

Document similarity

Week 6: Probability Distributions in ML

🔔

1D Gaussian

Bell curve & 68-95-99.7 rule

🎯

2D Gaussian

Contour ellipses & eigenvectors

📊

Distributions

Bernoulli, Binomial, Categorical

🎲

Sampling

Law of Large Numbers

Week 7: Bayesian Tools — Entropy, KL Divergence, MLE/MAP

🔀

Entropy Explorer

Shannon entropy for discrete distributions

🎯

Cross-Entropy Loss

Classification loss with softmax

↔️

KL Divergence

Gaussian & discrete, forward vs reverse

Week 8: Function Approximation — Perceptrons, MLPs, Universal Approximation

🧠

NN Architecture Evolution

Perceptron → Logic Gates → MLP → Cybenko → Deep Networks

Week 9: Training Neural Networks — Activation Functions, Gradient Descent, Backpropagation

📈

Activation Functions

Sigmoid, Tanh, Heaviside & derivatives comparison

⛰️

Gradient Descent & LR

Loss surface, learning rate effects, convergence

🔄

Forward & Backprop

Step-by-step through a 1→2→1 MLP