23 DDPM: the estimator becomes a training loss

23.1 A loss you have already minimized

In Machine Learning and Physics you trained a denoising diffusion probabilistic model, and the loss you minimized was a mean-squared error on noise: draw a data point, corrupt it to a random level \(t\), and train a network \(\varepsilon_\theta\) to guess the noise that was added,

\[ L_\text{simple}(\theta) \;=\; \Big\langle \big\| \varepsilon - \varepsilon_\theta\big(\sqrt{\bar{\alpha}_t}\, x_0 + \sqrt{1 - \bar{\alpha}_t}\; \varepsilon,\; t\big) \big\|^2 \Big\rangle_{x_0,\, \varepsilon,\, t} . \]

Nothing in that expression looks like statistical mechanics. There is no partition function and no free energy; it reads as a regression, and it was presented to you as one. \(L_\text{simple}\) is the terminal point of a derivation that starts from Lecture 9 and runs through everything Module 4 has built: it is the variational bound on the probability of an observation, with a whole noising path as the hidden variables; its slack is a Kullback–Leibler divergence between a forward and a reverse path measure, the object Week 11 learned to take apart step by step; and each of its terms is Week 10’s denoising score matching at one noise level. Ho, Jain, and Abbeel (2020) wrote the recipe; Sohl-Dickstein et al. (2015) had built the construction five years earlier, citing the fluctuation-theorem literature. Today we run the derivation forward, from the corruption chain to the regression, and note each step at which an engineering decision, a choice not forced by the mathematics, enters; there are four, collected in Section 23.5.

The required tools are all in hand — the clamped partition function and its bound (Section 9.4), the forward process and its melting barriers (Section 20.2), the reverse-time SDE (Equation 20.2), denoising score matching (Equation 19.4), and the path-measure bookkeeping of Week 11’s two lectures. The new content is how they assemble into a practical model, and the discrete-time bookkeeping this requires.

Figure 23.1: The DDPM as a double chain. Top (navy): the forward corruption, a fixed Markov chain of Gaussian kernels \(q(x_t \mid x_{t-1})\) carrying the two-mode data density through blurred intermediates to a unit Gaussian; it contains no parameters and is never trained. Bottom (orange): the learned reversal, a Gaussian ansatz \(p_\theta(x_{t-1} \mid x_t)\) — one network, conditioned on \(t\) — run from noise back to data. Between the two chains, the variational bound decomposes into one closed-form Kullback–Leibler divergence per step, plus a parameter-free endpoint term \(L_T\) (does the corruption finish?) and a reconstruction term \(L_0\).

23.2 The forward chain: the ladder, discretized

The DDPM forward process is Week 10’s corruption (Equation 20.1) written as a finite Markov chain. Fix a schedule \(\beta_1, \ldots, \beta_T \in (0, 1)\) and corrupt in \(T\) steps,

\[ q(x_t \mid x_{t-1}) \;=\; \mathcal{N}\!\big(x_t;\; \sqrt{1 - \beta_t}\; x_{t-1},\; \beta_t I\big), \qquad t = 1, \ldots, T . \]

A further notation warning: \(\beta_t\) here is a variance schedule — the fraction of variance replaced by fresh noise at step \(t\) — not an inverse temperature, exactly as the \(\beta_k\) of the AIS ladder (Section 22.3) was an annealing schedule. The field’s notation is entrenched and we keep it; no temperature appears anywhere today.

Write \(\alpha_t = 1 - \beta_t\) and \(\bar{\alpha}_t = \prod_{s=1}^t \alpha_s\). Composing the kernels collapses the chain into a single Gaussian: if \(x_{t-1} = \sqrt{\bar{\alpha}_{t-1}}\, x_0 + \sqrt{1 - \bar{\alpha}_{t-1}}\; \varepsilon'\), then one more step gives fresh noise of variance \(\alpha_t (1 - \bar{\alpha}_{t-1}) + \beta_t = 1 - \bar{\alpha}_t\), and by induction

\[ q(x_t \mid x_0) \;=\; \mathcal{N}\!\big(x_t;\; \sqrt{\bar{\alpha}_t}\, x_0,\; (1 - \bar{\alpha}_t)\, I\big), \qquad\text{i.e.}\qquad x_t \;=\; \sqrt{\bar{\alpha}_t}\, x_0 + \sqrt{1 - \bar{\alpha}_t}\; \varepsilon, \quad \varepsilon \sim \mathcal{N}(0, I). \tag{23.1}\]

the marginal: signal decays, total variance is preserved

The construction is variance-preserving — a unit-variance input stays at unit variance, \(\alpha_t + \beta_t = 1\) — and it is precisely the Ornstein–Uhlenbeck corruption of Equation 20.1 sampled at discrete times: setting \(\sqrt{\bar{\alpha}_t} = e^{-t_\text{phys}}\) reproduces Week 10’s transition law exactly, with \(\beta_t \approx 2\,\Delta t\) for small steps. Everything established there carries over unchanged — the endpoint is free (\(\bar{\alpha}_T \to 0\) makes \(x_T\) pure noise, whatever the data were), the intermediates are blurs whose barriers have melted (Figure 20.2), and the intermediates are exactly where denoising score matching trains.

The chain adds a closed form that the continuous picture did not need. Because the chain is Markov and every factor is Gaussian, the corruption step is exactly invertible when anchored at both ends: by Bayes,

\[ q(x_{t-1} \mid x_t, x_0) \;=\; \frac{q(x_t \mid x_{t-1})\; q(x_{t-1} \mid x_0)}{q(x_t \mid x_0)} \;=\; \mathcal{N}\!\big(x_{t-1};\; \tilde{\mu}_t(x_t, x_0),\; \tilde{\beta}_t I\big), \tag{23.2}\]

the two-anchor posterior: the reversal given \(x_0\)

with — completing the square in \(x_{t-1}\), whose precision collects \(\alpha_t/\beta_t\) from the likelihood and \(1/(1 - \bar{\alpha}_{t-1})\) from the prior —

\[ \tilde{\mu}_t(x_t, x_0) \;=\; \frac{\sqrt{\alpha_t}\,(1 - \bar{\alpha}_{t-1})\, x_t + \sqrt{\bar{\alpha}_{t-1}}\, \beta_t\, x_0}{1 - \bar{\alpha}_t}, \qquad \tilde{\beta}_t \;=\; \frac{1 - \bar{\alpha}_{t-1}}{1 - \bar{\alpha}_t}\, \beta_t . \]

Equation 23.2 quantifies the challenge facing generation. Conditioned on where the trajectory started, one corruption step reverses in closed form — a Gaussian, with a mean that interpolates between the two anchors. But the starting point \(x_0\) is exactly what a generator does not have; it is the thing being generated. The model’s entire job will be to supply a substitute for the \(x_0\)-dependence of \(\tilde{\mu}_t\), and the substitute will turn out to be Week 10’s score.

23.3 The bound: an observation clamps a path

The generative model is the reverse chain of Figure 23.1: start at the free end and walk down,

\[ p_\theta(x_{0:T}) \;=\; p(x_T) \prod_{t=1}^{T} p_\theta(x_{t-1} \mid x_t), \qquad p(x_T) = \mathcal{N}(0, I), \]

with kernels to be specified in the next section. This is a normalized probability distribution over whole paths. The question a likelihood asks of it concerns one end of the path only: how probable is an observed data point \(x_0\)? The answer is a sum over every path that ends there,

\[ p_\theta(x_0) \;=\; \int \mathrm{d}x_{1:T}\; p_\theta(x_{0:T}) , \]

and it is the partial measurement of Lecture 9 (Section 9.4), carried over from configurations to paths. As there, read the normalized joint as a Boltzmann distribution at \(T = 1\), now over paths, with path energy \(-\log p_\theta(x_{0:T})\) and global partition function \(Z = 1\) (Equation 9.3). Observing \(x_0\) clamps one end of the path; the unobserved remainder \(x_{1:T}\) plays the part of the hidden units. The probability of the observation is the partition function of the clamped path ensemble, \(p_\theta(x_0) = Z_\theta(x_0)\), and the distribution of the unobserved path given the observation is the clamped Boltzmann distribution, \(p_\theta(x_{1:T} \mid x_0) = p_\theta(x_{0:T})/Z_\theta(x_0)\). The obstruction is the one Lecture 9 met. The reverse kernels depend on \(x_t\) through a network, so the clamped path remains coupled from step to step, and \(Z_\theta(x_0)\) is an integral over \(T\) coupled configurations that no one evaluates in closed form.

The clamped path ensemble is a Boltzmann system in its own right, and Gibbs–Bogoliubov (Equation 9.1) bounds its free energy \(F(x_0) = -\log p_\theta(x_0)\) for any trial distribution over the unobserved path. One trial distribution is available without any construction: the forward corruption run from the observed point, \(q(x_{1:T} \mid x_0) = \prod_{t=1}^{T} q(x_t \mid x_{t-1})\). It is a product of known Gaussians, it can be sampled directly, and it starts where the observation is clamped. With it as \(q\), the variational free energy \(F_q(x_0) = \langle \log q(x_{1:T} \mid x_0) - \log p_\theta(x_{0:T}) \rangle_q\) and the gap of Lecture 9 give

\[ \boxed{\; L(x_0; \theta) \;\equiv\; \Big\langle \log \frac{q(x_{1:T} \mid x_0)}{p_\theta(x_{0:T})} \Big\rangle_{q} \;=\; -\log p_\theta(x_0) \;+\; D_\mathrm{KL}\big( q(x_{1:T} \mid x_0) \,\big\|\, p_\theta(x_{1:T} \mid x_0) \big) \;\geq\; -\log p_\theta(x_0) \;} \tag{23.3}\]

the DDPM objective: Lecture 9’s bound on a clamped path ensemble

— the negative log-likelihood plus the Kullback–Leibler divergence between the trial distribution and the model’s posterior over paths. The objective needs no construction of its own: it is the Gibbs–Bogoliubov bound of the clamped path ensemble, with the corruption chain as trial distribution. In the vocabulary of Equation 9.4, \(L = -\mathrm{ELBO}\), and in Week 7’s language (Equation 14.6) a diffusion model is a variational autoencoder whose latent variable is the entire path \(x_{1:T}\) and whose encoder is fixed. The corruption chain contains nothing to learn, which is why the encoder side contributes no gradients, and neither the reparameterization trick nor an amortization network appears anywhere.

The fixed trial distribution changes what minimizing \(L\) does. In Lecture 9 one optimized \(q\) to approach a given posterior; here \(q\) cannot move, so training over \(\theta\) lowers the bound by moving the model’s posterior over paths toward the forward corruption. The gap vanishes when \(p_\theta(x_{1:T} \mid x_0) = q(x_{1:T} \mid x_0)\) for the data points, that is, when the learned chain reproduces the corruption’s path measure, which happens when its kernels are the exact time reversal of the corruption. Anderson’s theorem (Equation 20.2) makes that target reachable by the Gaussian kernels of the next section in the small-step limit, since the exact reversal of a diffusion is again a diffusion. At the target the bound is tight, \(L = -\log p_\theta(x_0)\).

Two remarks relate the object to Week 11. First, averaged over the data, the bound is a divergence between two path measures, \(\langle L \rangle_{p_\text{data}} = D_\mathrm{KL}\big(q(x_{0:T}) \,\|\, p_\theta(x_{0:T})\big) + H[p_\text{data}]\), with \(q(x_{0:T}) = p_\text{data}(x_0)\, q(x_{1:T} \mid x_0)\) the forward process started from the data. This is the form of Equation 21.5, and Equation 22.5 shows how it resolves step by step and why its minimizer is a Bayes reversal. It is not a dissipated work: Equation 21.5 equated the divergence with \(\beta \langle W_\text{diss} \rangle\) for a protocol that drives a Hamiltonian from equilibrium, while the corruption starts from \(p_\text{data}\), drives nothing, and is compared with a learned model rather than with its physical reverse (Section 21.4). The correspondence with Week 11 holds at the level of path measures. Second, the historical order ran the other way from the pedagogical one: Sohl-Dickstein et al. (2015) wrote the likelihood as an importance-sampling average over forward paths, \(p_\theta(x_0) = \langle p_\theta(x_{0:T})/q(x_{1:T} \mid x_0) \rangle_q\), applied Jensen’s inequality to obtain the same bound, and cited the Jarzynski–Crooks literature for the construction; Ho, Jain, and Abbeel (2020) made it competitive at scale five years later.

The bound as written is a single number per data point; training needs it as a sum of local terms. The move is the telescoping of Section 22.4, executed once more. For \(t \geq 2\), trade the forward kernel for the two-anchor posterior using Markovianity and Bayes (Equation 23.2),

\[ q(x_t \mid x_{t-1}) \;=\; q(x_t \mid x_{t-1}, x_0) \;=\; q(x_{t-1} \mid x_t, x_0)\, \frac{q(x_t \mid x_0)}{q(x_{t-1} \mid x_0)} , \]

and the marginal ratios telescope across the product, \(\prod_{t=2}^{T} q(x_t \mid x_0)/q(x_{t-1} \mid x_0) = q(x_T \mid x_0)/q(x_1 \mid x_0)\) — the same regrouping-by-configuration that proved Equation 21.3 and Equation 22.3. The \(q(x_1 \mid x_0)\) in the denominator cancels against the \(t = 1\) factor of \(q(x_{1:T} \mid x_0) = q(x_1 \mid x_0) \prod_{t \geq 2} q(x_t \mid x_{t-1})\), and the \(t = 1\) factor of the reverse chain stays behind on its own, so

\[ L \;=\; \Big\langle \log \frac{q(x_T \mid x_0)}{p(x_T)} + \sum_{t=2}^{T} \log \frac{q(x_{t-1} \mid x_t, x_0)}{p_\theta(x_{t-1} \mid x_t)} - \log p_\theta(x_0 \mid x_1) \Big\rangle_q . \]

Each logarithm of a ratio averages to a Kullback–Leibler divergence, and the bound decomposes as

\[ L \;=\; \underbrace{D_\mathrm{KL}\big( q(x_T \mid x_0) \,\|\, p(x_T) \big)}_{L_T} \;+\; \sum_{t=2}^{T} \underbrace{\Big\langle D_\mathrm{KL}\big( q(x_{t-1} \mid x_t, x_0) \,\|\, p_\theta(x_{t-1} \mid x_t) \big) \Big\rangle_{q}}_{L_{t-1}} \;-\; \underbrace{\big\langle \log p_\theta(x_0 \mid x_1) \big\rangle_{q}}_{-L_0} . \tag{23.4}\]

the decomposition: one KL per step, two endpoint terms

\(L_T\) contains no parameters: it measures whether the corruption finishes — whether \(q(x_T \mid x_0)\) has actually reached the unit Gaussian the reverse chain starts from — since \(L_T \approx \tfrac{1}{2} \bar{\alpha}_T x_0^2\) for small \(\bar{\alpha}_T\), it is about \(2 \times 10^{-5}\) nat per dimension for the schedule of Ho, Jain, and Abbeel (2020) (\(\bar{\alpha}_T \approx 4 \times 10^{-5}\), unit-scale data) and about \(1.5 \times 10^{-3}\) nat for ours (\(\bar{\alpha}_T = e^{-8}\), data at \(\pm 3\)) — negligible either way, and fixed by the schedule. \(L_0\) is a reconstruction term for the final hop to data; for image models Ho, Jain, and Abbeel (2020) handle it with a discrete decoder over pixel bins, and we set it aside. The sum in the middle contains one Kullback–Leibler divergence per rung, between the two-anchor posterior, the reversal given \(x_0\), and the model’s kernel, which has no access to \(x_0\). Every distribution in sight is Gaussian, so every entry will be closed-form; no expectation in Equation 23.4 ever requires more than drawing \((x_0, \varepsilon, t)\) and evaluating a formula. The global normalizer never appears: the model was built normalized, kernel by kernel, so \(Z = 1\) and only the clamped partition function \(p_\theta(x_0)\) needed a bound.

23.4 From the bound to noise prediction

Now the kernels. The ansatz of Ho, Jain, and Abbeel (2020) is Gaussian with a fixed covariance,

\[ p_\theta(x_{t-1} \mid x_t) \;=\; \mathcal{N}\!\big(x_{t-1};\; \mu_\theta(x_t, t),\; \sigma_t^2 I\big), \]

\(\sigma_t^2\) set by hand — the first engineering decision, examined with the others in the next section. The choice simplifies the bound immediately: the KL between Gaussians of equal covariance (Equation 14.4, with the trace and log-determinant terms now \(\theta\)-independent) reduces each per-step term to a squared distance between means,

\[ L_{t-1} \;=\; \frac{1}{2 \sigma_t^2}\, \Big\langle \big\| \tilde{\mu}_t(x_t, x_0) - \mu_\theta(x_t, t) \big\|^2 \Big\rangle_{q} \;+\; \text{const}. \]

The network’s task is to hit the posterior mean, and the efficient parameterization emerges from writing that target in the right variable. Solve Equation 23.1 for the data point, \(x_0 = (x_t - \sqrt{1 - \bar{\alpha}_t}\, \varepsilon)/\sqrt{\bar{\alpha}_t}\), and substitute into \(\tilde{\mu}_t\); the algebra runs through the same identity \(\alpha_t(1 - \bar{\alpha}_{t-1}) + \beta_t = 1 - \bar{\alpha}_t\) that built the marginal, and leaves

\[ \tilde{\mu}_t \;=\; \frac{1}{\sqrt{\alpha_t}} \Big( x_t - \frac{\beta_t}{\sqrt{1 - \bar{\alpha}_t}}\, \varepsilon \Big) : \]

the posterior mean is the current state, nudged against the noise that was actually added. Give the model the same functional form with a learned noise estimate, \(\mu_\theta = \big( x_t - \tfrac{\beta_t}{\sqrt{1 - \bar{\alpha}_t}}\, \varepsilon_\theta(x_t, t) \big)/\sqrt{\alpha_t}\), and the means difference collapses onto the noise difference:

\[ \boxed{\; L_{t-1} \;=\; \frac{\beta_t^2}{2 \sigma_t^2\, \alpha_t (1 - \bar{\alpha}_t)}\, \Big\langle \big\| \varepsilon - \varepsilon_\theta(x_t, t) \big\|^2 \Big\rangle_{x_0, \varepsilon}, \qquad s_\theta(x_t, t) \;=\; -\, \frac{\varepsilon_\theta(x_t, t)}{\sqrt{1 - \bar{\alpha}_t}} \;} \tag{23.5}\]

each per-step term is denoising score matching; \(\varepsilon\)-prediction is the rescaled score

The left half of the box is the loss you trained, one weight away (\(L_\text{simple}\) drops the prefactor — the second engineering decision). The right half is Week 10’s identity. The conditional score of the marginal Equation 23.1 is \(\nabla_{x_t} \log q(x_t \mid x_0) = -(x_t - \sqrt{\bar{\alpha}_t}\, x_0)/(1 - \bar{\alpha}_t) = -\varepsilon / \sqrt{1 - \bar{\alpha}_t}\), so the \(\varepsilon\)-regression in Equation 23.5 is denoising score matching (Equation 19.4) at noise scale \(\sigma^2 = 1 - \bar{\alpha}_t\), rescaled: by Vincent’s identity its minimizer is \(\varepsilon_\theta^*(x_t, t) = \langle \varepsilon \mid x_t \rangle = -\sqrt{1 - \bar{\alpha}_t}\; \nabla_{x_t} \log q_t(x_t)\), the rescaled score of the blurred marginal. A network trained to predict noise therefore estimates \(\nabla \log p_t\), and the whole objective is denoising score matching at every noise level, tied together by the variational bound. Figure 23.2 verifies the identity with no training in the loop — the conditional mean of the noise, estimated by binned averages from raw corruption samples, lands on the rescaled analytic score of the running two-mode target.

Show code

abar = 0.5
n = 400_000
x0 = sample_p0(n)
eps = rng.normal(size=n)
xt = np.sqrt(abar) * x0 + np.sqrt(1 - abar) * eps

bins = np.linspace(-4.2, 4.2, 43)
idx = np.digitize(xt, bins)
centers, means = [], []
for b in range(1, len(bins)):
    sel = idx == b
    if sel.sum() >= 500:
        centers.append(0.5 * (bins[b - 1] + bins[b]))
        means.append(eps[sel].mean())
centers, means = np.array(centers), np.array(means)
pred = -np.sqrt(1 - abar) * score_t(centers, abar)
rms = np.sqrt(((means - pred)**2).mean())

xs = np.linspace(-4.2, 4.2, 500)
fig, ax = plt.subplots(figsize=(8.4, 4.4))
dens = pt_density(xs, abar)
ax.fill_between(xs, -3.4, -3.4 + 2.2 * dens / dens.max(),
                color=GRAY, alpha=0.25, lw=0)
ax.text(-3.9, -2.6, r"marginal $p_t$", color=GRAY, fontsize=10)
ax.plot(xs, -np.sqrt(1 - abar) * score_t(xs, abar), color=ORANGE, lw=2.2,
        label=r"analytic $-\sqrt{1-\bar{\alpha}_t}\,\nabla \log p_t$")
ax.plot(centers, means, "o", color=NAVY, ms=5,
        label=rf"binned $\langle \varepsilon \mid x_t \rangle$ (rms dev. {rms:.2f})")
ax.axhline(0, color=GRAY, lw=0.8, ls=":")
ax.set_xlabel(r"$x_t$")
ax.set_ylabel(r"noise to predict")
ax.legend(frameon=False, fontsize=10, loc="upper left")
ax.set_title("what the network is asked to learn is the score", fontsize=12)
fig.tight_layout()
plt.show()

Figure 23.2: The \(\varepsilon\)-regression target is the rescaled score, verified without training. At noise level \(\bar{\alpha}_t = 0.5\), corruption samples \(x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\varepsilon\) of the running two-mode target are binned in \(x_t\) and the added noise \(\varepsilon\) is averaged within each bin (navy points, \(4 \times 10^5\) samples, bins with at least 500 hits). The binned conditional mean \(\langle \varepsilon \mid x_t \rangle\) — the minimizer of the DDPM regression Equation 23.5 — lands on the analytic curve \(-\sqrt{1-\bar{\alpha}_t}\,\nabla \log p_t\) (orange) to an rms deviation of \(0.01\). The gray band underneath is the marginal \(p_t\): where the blur has mass the estimate is tight, and the sparse bins near the density minimum scatter visibly — the same coverage effect that Week 10 flagged for learned scores.

Take-home 1

The probability of a data point is the partition function of the path ensemble with one end clamped, and Lecture 9’s Gibbs–Bogoliubov bound, with the forward corruption as trial distribution, is the DDPM objective: the negative log-likelihood plus a divergence between the corruption and the model’s posterior over paths. With a fixed-covariance Gaussian ansatz, each per-step term becomes a noise regression whose minimizer is the rescaled score.

23.5 The design choices

Every remaining feature of the recipe is an engineering decision layered on top of the exact construction. Four entries follow, in the order they appeared.

① Fixed reverse variances. Ho, Jain, and Abbeel (2020) set \(\sigma_t^2\) to \(\beta_t\) or to \(\tilde{\beta}_t\) and report similar results for both. The two are the ends of a bracket — \(\tilde{\beta}_t\) is the posterior variance when \(x_0\) is deterministic, \(\beta_t\) the correct value when \(p_0\) is itself a unit Gaussian — and for the schedules in use they differ by little except at the smallest \(t\). Learning the variances (interpolating inside the bracket) improves the log-likelihood by about \(0.1\) bits per dimension and permits far fewer sampling steps (Nichol and Dhariwal 2021).

② Unit weights. The full bound Equation 23.5 carries the prefactor \(w_t = \beta_t^2 / \big(2 \sigma_t^2 \alpha_t (1 - \bar{\alpha}_t)\big)\); \(L_\text{simple}\) replaces every \(w_t\) by one. Figure 23.3 computes the true weights for a \(T = 1000\) chain: they span more than two orders of magnitude and concentrate at the near-clean steps, where the bound resolves fine detail, exactly the terms that dominate a log-likelihood. Setting them to one down-weights the clean end and spends the network’s capacity at mid and high noise, where the large-scale structure of a sample is decided. The result is a deliberate trade: better perceptual samples, a worse (indeed, no longer guaranteed) likelihood bound. The repair is also known precisely: Song, Durkan, et al. (2021) show that one specific reweighting — the likelihood weighting, \(g^2(t)\) in the continuous-time notation of the next lecture — restores a rigorous upper bound on the negative log-likelihood. The weights interpolate between the two objectives.

③ The schedule, and why \(T \sim 10^3\). The pair (schedule \(\beta_t\), step count \(T\)) plays the part of Week 11’s ladder, and a similar trade-off constrains it. The per-step divergence must be kept small for a reason specific to today’s ansatz: the true one-step reversal \(q(x_{t-1} \mid x_t)\), without the \(x_0\) anchor, is a mixture as structured as the data themselves, and collapses onto a Gaussian only for small \(\beta_t\) (Sohl-Dickstein et al. 2015). The step size therefore controls the ansatz error, which is why a thousand gentle steps outperform ten abrupt ones at fixed total corruption, as many small rungs outperformed a few large ones in AIS (Section 22.5). The worked example below makes this failure visible on demand.

④ What the network predicts. Regressing \(\varepsilon\), regressing \(x_0\), or regressing the mean \(\mu\) are algebraically interchangeable — each is an affine function of the others given \(x_t\) — but they precondition the network differently: the \(\varepsilon\)-target has unit variance at every \(t\), while an \(x_0\)-target collapses at the clean end and a \(\mu\)-target nearly duplicates the input. Karras et al. (2022) study these scalings, together with schedules and samplers, as one decoupled design space; the statistical-mechanics construction fixes none of them.

Show code

T = 1000
tk = 4.0 * np.arange(T + 1) / T
ab = np.exp(-2 * tk)
al = ab[1:] / ab[:-1]
be = 1 - al
ts = np.arange(2, T + 1)
bt_tilde = (1 - ab[ts - 1]) * be[ts - 1] / (1 - ab[ts])
w = be[ts - 1]**2 / (2 * bt_tilde * al[ts - 1] * (1 - ab[ts]))

fig, ax = plt.subplots(figsize=(8.4, 4.2))
ax.plot(ts, w, color=NAVY, lw=2.0, label=r"true weight $w_t$ of the bound")
ax.axhline(1.0, color=ORANGE, lw=2.0, ls="--",
           label=r"$L_\mathrm{simple}$: unit weight")
ax.set_yscale("log")
ax.set_xlabel(r"step $t$  (clean $\to$ noisy)")
ax.set_ylabel(r"weight of $\|\varepsilon - \varepsilon_\theta\|^2$")
ax.annotate("the bound weights\nthe near-clean steps",
            xy=(8, w[6]), xytext=(90, 0.28), color=NAVY, fontsize=10.5,
            arrowprops=dict(arrowstyle="->", color=NAVY, lw=1.1))
ax.annotate("unit weights favor\nmid and high noise",
            xy=(820, 0.93), xytext=(640, 0.09), color=ORANGE, fontsize=10.5,
            arrowprops=dict(arrowstyle="->", color=ORANGE, lw=1.1))
ax.legend(frameon=False, fontsize=10, loc="lower left")
ax.set_title("the per-step weights of the bound", fontsize=12)
fig.tight_layout()
plt.show()

Figure 23.3: What \(L_\text{simple}\) actually reweights. True per-step weights \(w_t = \beta_t^2/\big(2\sigma_t^2 \alpha_t (1-\bar{\alpha}_t)\big)\) of the bound Equation 23.5 (navy), for a \(T = 1000\) discretization of the Week-10 corruption (\(\sqrt{\bar{\alpha}_t} = e^{-4t/T}\), \(\sigma_t^2 = \tilde{\beta}_t\)), against the unit weight of \(L_\text{simple}\) (orange dashed). The bound weights the near-clean steps most heavily — \(w_t \approx 0.50\) at \(t = 2\) against \(0.004\) at \(t = T\), a factor of \(125\), diverging as \(t \to 1\) — because small corruptions carry the fine, likelihood-dominating detail. Replacing \(w_t\) by \(1\) shifts the training effort toward mid and high noise, where perceptual structure is decided: a deliberate engineering trade of likelihood for sample quality.

Trap

The standard misreadings of the DDPM recipe. ① \(L_\text{simple}\) is not a likelihood bound. Dropping the weights \(w_t\) forfeits the inequality of Equation 23.3; reporting \(L_\text{simple}\) (or a naively reweighted variant) as a negative log-likelihood is a category error; a valid bound requires the weights, or the likelihood weighting of Song, Durkan, et al. (2021). ② The Gaussian reverse kernel is an approximation, and the step size is its control parameter. The exact reversal of one corruption step is as multimodal as the data; only small \(\beta_t\) makes it Gaussian. Large steps do not merely coarsen a sound method — they break its central ansatz (Figure 23.4). ③ \(\varepsilon_\theta\) is a rescaled score. Any sampler bolted onto a trained DDPM — Langevin refinement, the reverse SDE, the next lecture’s probability-flow ODE — needs \(s_\theta = -\varepsilon_\theta/\sqrt{1-\bar{\alpha}_t}\); dropping the rescaling is a common error that produces no warning, wrong by a factor that varies with \(t\).

23.6 Example: the ansatz needs small steps

Entry ③ concerns the structural approximation of the construction. We use the running two-mode target with its exact scores, so that learning error is zero by construction and whatever fails is the ansatz. The sampler is exactly Ho’s ancestral chain: start at \(x_T \sim \mathcal{N}(0,1)\), step down with \(x_{t-1} = \mu_\theta(x_t, t) + \sigma_t z\) — the mean built from the exact score via Equation 23.5, \(\sigma_t^2 = \tilde{\beta}_t\), no noise on the final step — and vary only the number of steps \(T\) at fixed total corruption (\(\sqrt{\bar{\alpha}_t} = e^{-4t/T}\), as throughout).

Show code

def ancestral(T, n=200_000):
    tk = 4.0 * np.arange(T + 1) / T
    ab = np.exp(-2 * tk)
    al = ab[1:] / ab[:-1]
    be = 1 - al
    x = rng.normal(size=n)
    for t in range(T, 0, -1):
        mu = (x + be[t - 1] * score_t(x, ab[t])) / np.sqrt(al[t - 1])
        if t > 1:
            bt = (1 - ab[t - 1]) * be[t - 1] / (1 - ab[t])
            x = mu + np.sqrt(bt) * rng.normal(size=n)
        else:
            x = mu
    return x

fig, (a1, a2) = plt.subplots(1, 2, figsize=(10.6, 4.2))

# left: the true one-step reversal at T=2 versus its best Gaussian
tk = 4.0 * np.arange(3) / 2
ab = np.exp(-2 * tk)
al0, be0 = ab[1] / ab[0], 1 - ab[1] / ab[0]
xs = np.linspace(-6, 6, 1200)
p0 = (wts * np.exp(-(xs[:, None] - mus)**2 / (2 * sigs**2))
      / (np.sqrt(2 * np.pi) * sigs)).sum(1)
post = p0 * np.exp(-(0.0 - np.sqrt(al0) * xs)**2 / (2 * be0))
post /= np.trapz(post, xs)
m_post = np.trapz(xs * post, xs)
v_post = np.trapz((xs - m_post)**2 * post, xs)
gauss = np.exp(-(xs - m_post)**2 / (2 * v_post)) / np.sqrt(2 * np.pi * v_post)
a1.fill_between(xs, post, color=NAVY, alpha=0.30)
a1.plot(xs, post, color=NAVY, lw=2.0, label=r"true $q(x_0 \mid x_1 = 0)$")
a1.plot(xs, gauss, color=ORANGE, lw=2.0, ls="--",
        label=rf"best Gaussian ($\mu={m_post:.1f}$, $\sigma^2={v_post:.1f}$)")
a1.set_xlabel(r"$x_0$"); a1.set_ylabel("density")
a1.set_title(r"a single large step: the reversal is bimodal", fontsize=12)
a1.legend(frameon=False, fontsize=9.5, loc="upper right")

# right: ancestral samples vs step count
offsets = {2: 0.0, 8: 0.55, 64: 1.10}
for T, off in offsets.items():
    x = ancestral(T)
    hist, edges = np.histogram(x, bins=140, range=(-6, 6), density=True)
    cen = 0.5 * (edges[1:] + edges[:-1])
    col = RED if T == 2 else (NAVY if T == 8 else ORANGE)
    a2.fill_between(xs, off, off + p0, color=GRAY, alpha=0.18, lw=0)
    a2.plot(cen, hist + off, color=col, lw=1.6)
    spur = ((x > -1.5) & (x < 1.5)).mean()
    lab = f"{spur:.2f}" if spur >= 0.01 else f"{spur:.0e}"
    a2.text(-6.1, off + 0.34, rf"$T={T}$: spurious mass {lab}",
            color=col, fontsize=10)
a2.set_xlabel(r"$x$")
a2.set_yticks([])
a2.spines["left"].set_visible(False)
a2.set_title("ancestral samples, exact score, varying $T$", fontsize=12)
fig.tight_layout()
plt.show()

Figure 23.4: The Gaussian ansatz needs small steps — exact scores, so every failure is the ansatz. Left: the true one-step reversal \(q(x_0 \mid x_1)\) for a two-step chain (\(T = 2\), \(\beta_t = 0.98\)), evaluated at \(x_1 = 0\) (navy): it is bimodal with modes at the two data modes and variance \(7.7\), and no Gaussian — the moment-matched one is shown in orange — can represent it. Right: ancestral samples against the true density (gray) for \(T = 2\), \(8\), \(64\) at fixed total corruption. At \(T = 2\) the sampler places \(55\%\) of its mass in the region \(|x| < 1.5\), which truly holds \(0.001\); at \(T = 8\) the spurious mass is down to \(7\%\); at \(T = 64\) it is \(3 \times 10^{-4}\), with the mode weight at \(0.295\) against the true \(0.300\). The steps must be small enough that the true reversal kernel is Gaussian; a thousand gentle steps is a requirement of the ansatz.

The left panel of Figure 23.4 is the anatomy of the failure. After a two-step corruption each step replaces \(98\%\) of the variance, and the true reversal \(q(x_0 \mid x_1)\) at \(x_1 = 0\) is the data distribution itself, barely tilted: two humps, six units apart. The Gaussian ansatz can place its mean correctly — with the exact score it does, at \(-1.19\), the posterior mean — but a single Gaussian centered between two modes puts its mass where neither mode lives. The right panel shows the consequence and its cure: at \(T = 2\) over half the generated mass lands in the inter-mode region that truly carries \(10^{-3}\) of the probability, and the defect decreases as the steps shrink, by three orders of magnitude at \(T = 64\). Nothing was learned or mislearned anywhere in this figure; the error is purely the distance between the true reversal kernel and the Gaussian family, and step size is the only parameter that controls it. This is Week 11’s near-equilibrium lesson relocated into a generative model: gentle protocols are the ones whose reversals are simple.

Take-home 2

Four engineering decisions sit on top of the exact construction: fixed reverse variances, unit weights (\(L_\text{simple}\) trades the likelihood bound for sample quality; a specific likelihood weighting restores it (Song, Durkan, et al. 2021)), the schedule with \(T \sim 10^3\) (small steps keep the true reversal Gaussian), and the choice of regression target (\(\varepsilon\) for its unit variance at every \(t\)).

The probability of a data point under a diffusion model is a partition function of paths with one end clamped: Lecture 9’s bound on it, with the forward corruption as trial distribution, is the DDPM objective, and each per-step term is Week 10’s denoising score matching.

23.7 Outlook: one construction, many parametrizations

The workhorse of modern image generation, trained in the companion course as a recipe, decomposes into objects this course built one at a time: a forward corruption chain (Week 10’s Ornstein–Uhlenbeck process, discretized), a variational bound that is Lecture 9’s clamped construction on paths (taken apart step by step with Week 11’s telescoping, the reverse path measure handed to a neural network), per-step terms that are denoising score matching in closed form (Week 10’s regression), and four engineering decisions whose consequences and repairs are each known precisely.

What the discrete bookkeeping cannot deliver is a likelihood without a bound; that requires the continuous limit, the subject of the next lecture. There, with the step index renamed \(k\) so that \(t\) can become continuous time, the chain converges to the score-SDE of Song, Sohl-Dickstein, et al. (2021), Anderson’s reverse-time SDE returns as the sampler, and a deterministic counterpart, the probability-flow ODE, carries the same marginals while permitting exact likelihood evaluation; flow matching reaches the same object without any stochastic construction.