I am working through Mathematical Foundations of Generative AI , Prof. Prathosh AP's public lectures. The playlist is the spine. These notes are my deep dive on each section: a visual when the picture is the point, a detour when a prerequisite is doing real work, and the formula in my own words.

Previously: Section 3a: f-divergences .

Section 3b: The convex conjugate and the variational bound

This is the last part of Week 1, and it fixes the catch from 3a: I cannot compute $D_f$ directly because it needs the densities $P_x(x)$ and $P_\theta(x)$, and I only have samples. By the end of this section, $D_f$ is rewritten using only averages over samples.…