PDL Discussion of assignment 4
- We observe that the error calibration of the variational model is actually way worse than that of the regular ML model. How is that possible?
- A: Well, looks like the prior is not well-calibrated!
- You can try cross-validation your prior choices for various choices of parameters. Very expensive!
- What are better choices for priors then? We only used independent Gaussians now (i.e. mean field approximation)
- But how would we e.g. find a good correlation for priors? Where do we get this info from?
- Transfer learning / meta learning approach
- It is also still possible that the variational model becomes overconfident
- The measure of error calibration is also quite sensitive to outliers
- REVIEW In the case of logistic regressions we can sometimes get a log likelihood of -Inf (when data is linearly separable, I think?)
- This problem should be remedied when you have more data, as this is then less likely to be linearly separable.