Bayesian inference by gradient descent
Variational inference is basically solving Bayesian inference with gradient descent.
In the 1D-case, quantization works better than variational inference. Q: Why?
In a multivariate case, discretization of the latent space is intractable.
For the exam, not asked for you to memorize the closed form solution for Gaussian inference, but you are expected to know how to derive it. See lecture 0.
Trick: rephrase integration as an optimization problem.
Define "variational distribution" q, which looks like a posterior, but then parameterized, with mean and variance.
Now we need a loss function. We'll use the KL-divergence. Problem: this expression still uses the true posterior p(x|D), but the whole point was that we are looking for this!
But we only care about the gradient of the loss!
See eq. 7: rewrite where P(D) disappears, since constant contributions do not matter for the gradient.
This derivation is also discussed in PDL2-Stochastic_variational_inference.md. The crux is that the gradient of the KL-divergence between sampling and posterior is equal to the gradient over the negative ELBO.
Problem: now we have another integral to solve. However, this one is a bit more tractable.
Making the integral tractable
Imagine the integral expression for evidence p(D).
Imagine we don't know the mean and variance. Why is finding the mean so difficult? Well, the domain of the integral is infinite!
You need a fine discretization in order to pick up the very small region where the Gaussian is non-zero. But then you pick an insane amount of zero-points.
Now, the situation with the variational sampler is different! We do know the relevant region, namely the region of the variational approximation $q(x;\mu, v)$. This defines the region where we do gradient descent, since q nullifies the exponential e.g. of a Gaussian in regions where q is vanishing.
Relation variational inference and Bayesian inference (and MAP)
Insight after equation 15:
To summarize, at the limit of zero variational variance the vocational gradient of the mean parameter is identical to the regular gradient of the joint distribution $f(x)=p(D|x)p(x)$. This establishes a close connection between variational inference and the maximum-a-posteriori (MAP) algorithm.
Stochastic gradients and the reparameterization trick
This section discusses computing the gradient of the variational loss. This can be used in the gradient descent rule, but in this section we compute them by hand in closed form by taking the limit $N \rightarrow \infty$ and then setting the gradient to 0 to get an exact solution.
$dxn/d\mu$: consider this dependency from derivative of sample on mu ! We sampled from the variational distribution, which depended on its own mean. Your samples would have been different for a different mean.
There is no general approach for writing the derivative of the sample. You have to work out the reparameterization trick! We've seen how to do this for a Gaussian (eq. 20)
Eq. 21: z-scoring. Subtract mean and divide by sigma so $\epsilon$ follows a standard Normal with mean 0 and 1 variance.
Now the derivative of $\epsilon$ w.r.t. $\mu$ is 0, and aforementioned derivative of the data point w.r.t. $\mu$ is 1.
REVIEW: if you don't do this, the derivative of datapoint w.r.t. mu will instead be 0, which causes the gradient to nullify. Verify why this is the case.
See eq. 25: for N -> infty, our empirical average converges on the true mean. This was part of the difficult integral; so this helps making it tractable.
Same for variance, but now we cannot ignore the differential entropy. Eq. 27-30.
+1 in eq. 30 reflects the prior.
Eq. 34: derivative of sample w.r.t. mu left out because it's 1, but it's there in principle.
Variational gradient descent
Strong connect mean of gradient of variational approximation and gradient of ...
Show one converges to the other.
Eq. 16; q will converge to deterministic distribution for limit.
Stochastic variational gradient descent for Gaussian models with non-linear dependency
In some cases the normalization integral does not have a closed form, but the variational gradient does, e.g. in some cases where we don't have conjugacy.
I.e. the mean of the data is some non-linear function g(z) of the latent variable.
Lecture notes discuss an example with a "quadratic link", i.e. the mean parameter of the normal specifying the data mean, is $z^2$. I.e. the mean of the observations depend on the latent quadratically, not linearly.
Example: shows you get a bimodal posterior symmetric around 0 OR 0 becomes a true solution. I.e. there's a pathological case of mode dropping. Generally speaking, this is a risk that comes with gradient descent.
If the variational variance is very big, your less likely to converge on 0 by "sitting in the middle" of the two modes.
KL-divergence is very undemocratic: it forces a fit either to the left mode or right mode.
When doing gradient descent, we should be sign-blind; whether we fit the left or right mode is only dependent on initialization and shouldn't matter in this case.