Reparametrization trick
How can we do backpropagation over a sampling operation?
The problem is that we want to use gradient descent to find the best variational parameters, i.e. those parameters that lead to a variational distribution that approximate the true posterior.
The problem is that you compute the gradient with respect to the samples $z$ of the variational distribution, not the variational parameters. The main question thus is: How do you backpropagate from the samples to the parameters of the underlying distribution? You can't back propagate through a sampling operation because it is not a deterministic function.
Idea of reparameterization: rewrite the parameterized sampling operation w.r.t. a fixed distribution $q_0(z)$ and a parameterized differentiable function $f_{\theta}$ such that:
$$f_{\theta}(\epsilon) ~ q_{\theta}(z)$$ $$\epsilon ~ q_0(z)$$
So notice that you need to choose $f_{\theta}$ such that, while sampling from a fixed distribution $q_0$, you "reconstruct" the original variational distribution. The sampling distribution for $\epsilon$ has to be fixed. If $\epsilon$ is again dependent e.g. on $\mu$ in case of a Gaussian, then we have simply moved the problem and still have an unknown derivative.
In the case of the Gaussian, we can use a simple reparameterization formula that translates and scales. Example with Gaussian:
$$f_{\theta}(\epsilon) = \mu +_\sigma \epsilon ~ \N(\mu,\sigma^2)$$
with
$$\epsilon ~ N(0,1)$$
Given an existing sample $x_n$, you can transform into $\epsilon$ as a sample from a fixed standard Gaussian by z-scoring:
$$\epsilon_n = \frac{ x_n - \mu }{ \sigma }$$
Now, the sampling operation does not depend on $\theta$. Take a sample $x_n = \mu + \sigma \epsilon_n$ Then we have:
$$\frac{ \partial x_n}{ \partial \mu} = 1 + \sigma \frac{ \partial \epsilon_n}{ \partial \mu}$$
But now, we know that $\epsilon_n$ does not depend on $\mu$, so $\frac{ \partial \epsilon_n}{ \partial \mu}=0$ Similar reasoning for $\sigma$.
Example with computing the ELBO
Why is this important?
When computing the gradient over $L_{ELBO}(\theta)$ we can now push the gradient inside the expectation. Cf. ELBO.
Before we had the following gradient over the ELBO:
$$\nabla_{\theta} \E_{ z ~ q_{\theta}(z)} log \frac{ p(z,D) }{ q_{\theta} z }$$
In this case, when you push the gradient inside the expectation you get incorrect results, because the joint term (or if you decompose, the likelihood term) does not depend on $\theta$, meaning you'll get a gradient of 0. Also, if you move the gradient inside the expectation, you ignore the sampling operation that does depend on $\theta$.
What we want is that the gradient pushes the variational distribution to the region of $z$ for which we have high likelihood $p(D|z)$ (of course, the ELBO also takes into account prior, entropy etc.).
But now that we have our differentiable parameterized (i.e. on $\theta$) function $f_{\theta}$, also the likelihood term depends on $\theta$ and we can push the gradient inside the expectation term:
$$\nabla_{\theta} \E_{ z ~ q_{\theta}(z)} log \frac{ p(z,D) }{ q_{\theta} z } = $$
$$\nabla_{\theta} \E_{ z ~ q_{0}(\epsilon)} log \frac{ p( f_{\theta}(\epsilon),D) }{ q_{\theta} f_{\theta}(\epsilon) } = $$
$$\E_{ z ~ q_{0}(\epsilon)} \nabla_{\theta} log \frac{ p( f_{\theta}(\epsilon),D) }{ q_{\theta} f_{\theta}(\epsilon) }$$
We still cannot exactly compute the expectation, but we can now use an unbiased Monte Carlo sampler to approximate the expected ELBO gradient. But notice that instead of sampling from the variational distribution, we now sample from the fixed base distribution $q_0$!
The variance of the estimator depends on the number N of samples (as usual) but also on the choice of reparameterization.
Often reparameterization is already implemented in practice. In PyTorch we have .rsample() where r does not send for random but for "reparameterized".