Discussion a4

Nothing radically new here.

Ex. 2 more complex model with multivariate model. Correlate params in posterior (?).

Mean-field posterior should underestimate variance and thus have worse error calibration.

One source of bad error calibration is

  1. a bad prior
    • But in our caes our prior is perfect because we handpicked Brownian motion for Brownion motion
  2. But now the mean-field really breaks down, because we are dealing with time-series, so independence does not hold.

Very important that you should be able to do exercise 1.

  • Finish template code
  • Write ELBO
  • Write training rule
  • Not asked, but nice to do error calibration!
    • But how does this work with a regression problem?
    • Luca didn't really answer.

Implicit assumtion of Brownian motion:

  • Timeseries is continuous, no huge jumps
  • But als not smooth

So it's a pretty good model for stock prices. Then you'd use exponential Brownian motions. These do not have a derivative.

Exercise 2 and 3 are a bit more advanced.

You should be able to get the full posterior and thus good performance.

Questions

  • Where to define prior? VariationalModel? Or BrownianTransition? (I did latter, Gianluigi former)
  • Why use observation noise in log_prob of VariationalModel? Is this something we have access to?
    • E.g. isn't that inaccuracy in your measurements that we cannot avoid?

Diff between ex. 1 and ex. 2 not so visible.

To see the difference, take out observations all together. This is the "Brownian bridge" problem. In this case, we should just not evaluate the middle part.

Info on exam

Memorize:

  • Definition ELBO
  • Elements of ELBO
  • Jensen equality

ELBO is core, so make sure you nail it.

three loose parts:

  1. Closed form solution;
    • E.g. derive formula for normal-normal model closed form
    • Conceptual: Why Bayesian inference is intractable.
      • But very concise please, just a few sentences.
  2. Question on core theory, connected to ELBO
    • E.g. show that ELBO is lower bound on model evidence (CORE of the course!)
    • Use Jensen equality
  3. Applying ELBO on specific case
    • E.g. write down ELBO of time series model; e.g. with Brownian model
    • Reparametrization trick: be able to write down and explain.