PDL Lecture 3: Variational Supervised Learning
Variational inference is so general and can be applied both to unsupervised and supervised learning. This week we will apply the variational inference framework to machine learning.
Two approaches to classification:
- Discriminative: directly output class probabilities
- Generative: model the data distribution per class and then invert this with Bayes rule to get a prediction over classes
Normally we predict a class label. Here we are instead interested in predicting a predictive distribution over class labels for each data point.
This is important since the information encoded in the feature is often incomplete so that it cannot fully specify a single label.
Error calibration
We care about failing in a predictable way, i.e. when the network makes mistakes we also want our posteriors to express this uncertainty.
So we do not only care about accuracy, but about the error calibration measure:
- Look at all misclassified points
- Look at log probability of misclassified points
- Notice that this is still the attributed probability to the correct target $y_n$
- The higher this measure, the better the error calibration
- When p(y_n|x_n) is small (closer to 0) -> the log is a large negative number
- When p(y_n|x_n) is big (closer to 1) -> the log is smaller negative number
$$\frac{1}{N_{mis}} \sum_{mis} log p(y_n | x_n)$$
Rationale:
- So if the network predicts the wrong label, but was very certain about the wrong label, this is bad!
- Generally this means a lower probability was attributed to the correct class label
- If instead we predicted the wrong label, but our predictive posterior distribution expressed much uncertainty about this decision, then we have a better error calibration.
- "Expresses more uncertainty" means that generally speaking the correct label will also have a higher probability because the posterior is flatter.
What's actually measured in the error calibration formula is the probability assigned to the correct class, for misclassified data points. The reasoning for the two cases is almost identical, again:
- When the probability for the correct class is higher, then the difference between the likelihood of the correct and incorrect prediction is smaller, i.e. more uncertainty in the prediction.
- So a higher probability assigned to the true label (for a misclassified data point) leads to a higher error calibration
- So a higher error calibration is better
- When the probability for the correct class is lower, then the difference between the likelihood of the correct and incorrect class label is larger, i.e. the classifier was apparently relatively certain in the misclassified data point (and wrongly so!)
- So a lower probability for the correct label give a lower contribution to the error calibration
- So a lower error calibration is bad.
Now, because we are working with log probabilities, what does "higher" mean? The logarithm is monotonic, so the interpretation does not change. But because probabilities are in [0 .. 1], we have negative log probabilities with the exception of 0.
log(1)=0. log(0.1)=-1 log(0.01)=-10
So a higher (i.e. better error calibration) in terms of log probabilities means: a less negative number, approaching 0.
The error calibration also does not depend on the accuracy, since accuracy only considers correctly classified data points, whereas error calibration only considers misclassified data points.
Consolidated here probabilistic error calibration.
Binary logistic regression
Also see logistic regression.
How to construct a variational Bayesian version of the simple technique of binary logistic regression?
Main assumption:
$$log(\frac{p_j}{1-p_j}) = w x_j$$
i.e. the logarithm odd ratio of the label is a linear function of feature vector $x_j$, with $p_j => p(y_j = 1)$ The sigmoid loss comes directly from this basic assumption! We can also express the probability of $y=1$ as a function of weights by rewriting:
$$p_j(w)=Sigmoid(w x_j)$$
Derivation:
In the logit function, we are actually interested in $p_j$ i.e. the function argument. To get to it, we must compute the inverse of the logit (which is the sigmoid), by rewriting the formula as an expression for $p_j$. For convenience, I write $w \dot x_j = y$. Then we have:
$$y = log \frac{ p_j }{ 1 - p_j}$$ $$e^{y} = \frac{ p_j }{ 1 - p_j}$$ $$1+e^{y} = \frac{ 1 }{ 1 - p_j}$$ $$\frac{1}{1+e^{y} } = 1 - p_j$$ $$p_j = 1 - \frac{1}{1+e^{y} }$$
Also see Bishop p. 197 for a way to write posterior class probabilities in binary classification as the logistic sigmoid. He instead directly starts with the sigmoid and derives the logit later as the inverse.
Use $Sigmoid(-x)=1-Sigmoid(x)$ to write out likelihood $p(D|w)$ under the assumption of i.i.d. from general form $p(D|w) = \prod_{j}^J p(y_j=1)^{y_j} (1-p(y_j=1)^{1-y_j})$ :
$$p(D|w) = \prod_{j}^J p_{j}(w)^{y_j} p_j(-w)^{1-y_j}$$
As usual, we use the log likelihood in practice. If we take the negative log likelihood as a loss function, we get the cross entropy loss.
Also handy to know the derivative of the sigmoid
$$\frac{ \partial \sigma }{ \partial a} = \sigma ( 1 - \sigma)$$
Bonus exercise: Show that the norm of the maximum likelihood estimate of the weights tends to infinity when the dataset is linearly separable.
I found that this is exercise 4.14 in Bishop, under 4.3.2. Logistic Regression.
If our data is linearly separable, then for each data point we either have $w^T \phi(x_n) > 0$ let's say when it belongs to $C_1$, or $w^T \phi(x_n) < 0$ when it belongs to $C_2$. We can assume that we have already found the weight vector linearly separating the dataset. From here on, the log likelihood loss can only be lowered by assigning $p(C_1|x_n)=1$ to all $x_n \in C_1$ and $p(C_1|x_m)=0$ for all $x_m \in C_2$. Due to the squashing behavior of the sigmoid, this is achieved by taking the limit $w^T \phi(x_n) -> \infty$ and $w^T \phi(x_m) -> -\infty$. This in turn can only be achieved by $|w| -> \infty$ because $\phi$ is a fixed linear basis function.
This is clearly a scenario of overfitting. For linearly separable data, you will observe that the loss keeps decreasing, but this does not actually mean you learn anything new, because you are achieving just only by inflating the norm of the weight vector.
This will lead to very bad error calibration unless you have infinite data points in the training set. If the training set is linearly separable then for each prediction the assigned probability will be very high (approaching 1). This means that any errors made on the testing set will also be assigned a high probability; hence the bad error calibration.
From Bishop:
It is worth noting that maximum likelihood can exhibit severe over-fitting for data sets that are linearly separable. This arises because the maximum likelihood solution occurs when the hyperplane corresponding to $\sigma=0.5$, equivalent to $w^T \phi = 0$, separates the two classes and the magnitude of $w$ goes to infinity. In this case, the logistic sigmoid function becomes infinitely steep in feature space, corresponding to a Heaviside step function, so that every training point from each class $k$ is assigned a posterior probabibility $p(C_k|x)=1$. Furthermore, there is typically a continuum of such solutions because any separating hyperplane will give rise to the same posterior probabilities at the training data points, as will be seen later in Figure 10.13. Maximum likelihood provides no way to favour one such solution over another, and which solution is found in practice will depend on the choice of optimization algorithm and on the parameter initialization. Note that the problem will arise even if the number of data points is large compared with the number of parameters in the model, so long as the training data set is linearly separable. The singularity can be avoided by inclusion of a prior and finding a MAP solution for $w$, or equivalently by adding a regularization term to the error function. (Bishop, p.206)
Mean-field variational approach to logistic regression for better error calibration
Step 1: mediate these overfitting problems by adding a prior. In this case, bias is good! We can use an (unmotivated) spherical Gaussian:
$$p(w) = N( w; 0, \sigma^2 I)$$
Step 2: Define a parameterized family of variational distributions that is meant to approximate the posterior. In this case, we choose the mean-field family to do mean-field approximation: each weight is assumed to be statistically independent from each other weight.
$$q(w;\mu,\sigma) = \prod_k^K N(w_k;\mu_k,s_k^2)$$
Note: the goal is finding the posterior weights $w$.
Two assumptions:
- We assume the weights to be independent so we can factorize the posterior
- In practice the weights are correlated (e.g. when regressors are correlated), so mean-field approximation leads to an underestimation of variance.
- So mean-field is a good starting point, but we should really question its assumption as it often does not hold.
- We assume the posterior can be captured sufficiently as a Gaussian
Step 3: Write out ELBO
- Write out joint P(D,w), i.e. add the log prior to the likelihood
- ELBO: Write out expectation (under sampling distribution) of log of joint P(D,w) divided by q(w)
- Notice the element wise division because each weight can have its own variance
- Notice the square norm term in eq. 10
TODO decompose this; can I replicate?
Step 4: Apply reparameterization trick
Goal is to express the sampling operation now w.r.t. to a fixed non-parametrized distribution. The reparameterization of a Gaussian is luckily straightforward.
$$w=\mu + \sigma \epsilon$$ $$\epsilon \sim N(0,1)$$
Notice that when substituting w, we'll get $||\epsilon||^2/2$ which does not depend on the variational terms so we can drop it out. This is quite insightful: the entropy of a Gaussian does not depend on its mean.
Step 5: Replace expectation with sampler
Step 6: Do SGD both for the mean and the standard deviation; i.e. we are working in distribution space.
Step 7: Prediction
We end up with a posterior over the true classification boundary.
We can increase our probabilistic calibration by making predictions as follows: We can predict label y by sampling weights from the posterior, evaluate the prediction, and take the average prediction. For infinitely many data points this will converge on the MLE solution.
The variational multivariate normal approach to logistic regression
In mean-field approximation we ignore that weights are typically correlated. To account for this we use a covariance matrix instead of $\sigma^2 I$.
Problem: these covariance matrices need to be 1) symmetric and 2) positive definite.
- Symmetry: $C^T = C$
- Positive definite: $v^T C v \leq 0$ for all possible vectors $v$
So SGD needs to respect this!
Cholesky decomposition
You can express any positive definite covariance matrix as a product of two square matrices. S is usually called a "matrix square root".
$$C = S^T S$$
If we can find S, then $C$ is guaranteed to be symmetric. Using $(A B)^T = B^T A^T$ we can show:
$$C^T = (S^T)^T S^T(S^T)^T = S^TS = C$$
Similarly it can be shown that C satisfies positive definiteness.
Once we have symmetry and positive definiteness, it is proven that we can always find a matrix square root that is an upper diagonal matrix, where all non-zero entries are in the top right corner above the diagonal. This is called the Cholesky decomposition.
Reparameterization
Now we can satisfy the constraints on covariance matrix C by doing a smarter reparameterization:
$$w = \mu + U^T \epsilon$$ $$\epsilon \sim N(0,I)$$
The normal moments are preserved, i.e. expected value is still the mean, and expected covariance is still the covariance matrix.
Using the Cholesky decomposition, our approach is only parameterized on the "upper" part $U^+$. So we have less degrees of freedom and less parameters to optimize.
Write out $ELBO(\mu, U^+)$.
N.B. Determinant of an upper triangular matrix is just the product of the diagonal!
$$log | det(U) = \sum_{k}^K log |U_{kk}$$
Bayesian deep learning Generalize the variational approach to any deep neural network.
Before we defined priors on the weights of logistic regression. We can apply the same principle to the weights of a (deep) neural network.
We now take a deep neural network binary classification network with D layers. For each layer, we have a separate weight vector.
$$p_j = Sigmoid( f(x_{j}; w^{(1)}, ..., w^{(D)}))$$
The log likelihood is similar to regular logistic regression.
In convolution, weights are a 3D tensor, but in our variational approach we just want all weights in a single large vector.
$w^{(1:D)}$ is a concatenation of all weights.
Standard approach is to define a spherical Gaussian prior to each weight in each layer (see eq 30).
$$p(w^{(1:D)}) = \prod_{k=1}^T N( w_{k}^{(1:D)}; 0, \sigma^2)$$
We can then compute the ELBO same as always.
Now you do not train the weights like in regular deep learning, but you instead train the mean and standard deviation of the approximate posterior of each weight. This allows us to quantify uncertainty over the learned weights.
This means that if we want to make a prediction based on the learned variational weights, then we need to sample all weight vectors from the variational distribution. You can do this multiple times and average the predictions. Hopefully, this leads to a better error calibration.
Problems with Bayesian deep learning
- Error calibration heavily depends on the choice of prior
- The assumption of uncoupled Gaussian priors (mean field approx.) is not independently justified for just every classification problem
- Generally we should e.g. consider that classifying natural images leads to another choice of prior
- But now we just assign priors on weights generally and this is very tricky
- The challenge really is to construct meaningful priors.
- Mean-field approximation in itself is quite problematic due to the assumption of uncorrelated weights
So take-away: blindly applying the Bayesian approach will not in itself give you a better probabilistic calibration! You need to think about a meaningful prior!
Interesting challenge: learning prior before seeing the data with meta-learning techniques.
E.g. in self-supervised learning you can take image data and scramble them; then let the network learn to distinguish the original and scrambled images. So you solve some arbitrarily defined problems on the type of data you have and then use the learning weights as prior for the real problem. Quite interesting, but intuitively hard to say why this works. If you do this on a set of arbitrary problem, he hope is that this somewhat generalizes to the real problem.
Multi-class classification
Problem with multi-class categorical likelihood: the sum of exponential functions within a log tend to either underflow or overflow quite easily.
Trick for stability:
- Introduce $e^\alpha e^{-\alpha} =1$ within the log.
- Take $\alpha$ in front of the log, write $-\alpha$ in the exponent
- Make $\alpha$ equal the biggest exponent term, to reduce overflow.
TODO eq. 41; one of these alphas need to be positive, right? Now both are negative.
Regression and beyond
Robust likelihoods for regression
"From a probabilistic point of view, a regression model differs from a classification model only by the use of a different likelihood.
A standard squared regression loss (SSE, I guess) can be reproduced by using a Gaussian likelihood with unit variance.
For classification there are limited options for choosing a probabilistically justified loss. In case of regression, there are more options.
Squared loss amplifies outliers and suppresses small differences. The Laplace distribution leads to the absolute deviation loss and is less sensitive to outliers.
Counting and frequencies
The Poisson distribution is a common choice for the likelihood here.
The Poisson distribution has the interesting property that the variance is equal to the mean. You should evaluate whether this is an appropriate restriction for a given problem!