Probabilistic Modeling and Parameter Estimation
A probabilistic model captures statistical (in)dependencies and relative probabilities of the variables of some problem you are modelling.
Mathematically such a probabilistic model is just a (complex) probability distribution. A probability model uses such a probability distribution of a real-world phenomenon.
- The same distribution can be used to model many things
- (The same distribution can often result in many DAGs)
- My own idea here; a DAG is a particular type of modelling; but doesn't this show that model != probability distribution.
Why do we need probabilities?
- Sparsity in observations
- Indirect observations
- Randomness
We want to factor in uncertainty. But at the same time, we want to factor in the things we do know. We can use priors and such.
Main goal of this course:
But how do we factor in known mathematical equations coming from various fields into the deep learning components? How to include domain knowledge into deep learning? E.g. Hoiler equations about atmosphere.
Modelling
Notice that the physical processes behind a coin toss are very complex: interaction with air molecules etc. There is a particular art in using your intuition to find the right abstraction level. In the case of coin tosses, we have a very good model with a single parameter $\theta$ and a Bernouilli or Multinomial distribution.
Example: Random walk down Wall Street
We know stock prices are difficult to predict, since selling prices are about what people are willing to sell and buy at. So this would require us to model many people!
We can capture some of the trend however using the model of a random walk!
$$x_{t+1} = x_t + \eta_t$$
$$\eta_t ~ N(\eta_t; 0, \sigma^2)$$
I.e. the have normally distributed residuals. The mean is zero because we do not assume on average the stock is always increasing or always decreasing.
This is not very predictive at all! But it does allow us to model the uncertainty when buying a stock: how much fluctuation is there in this stock! If there is little, we have low risk.
We want to now model transition probabilities.
$$P(x_{t+1}|x_t) = N(x_t, \sigma^2)$$
We can express the probabilities over a whole sequence. Our model is now a joint probability distribution.
Often we do not care about the whole distribution. How do we then "ask" the model specific questions? For example, how do we compute the estimate of the estimate at a particular time point?
I.e. marginalization: integrating over all variables I do not care about. This is usually infeasible so we rely on approximations.
In our case (with Gaussian residuals) we can be smart and have a recursive definition. And sums of Gaussians are again a Gaussian (yay, Gaussians).
Mean of random variables is sum of the means. In this case: 0. The variance of the resulting Gaussian
The variance scales as T! This is a deep relation: the standard deviation grows proportional to the square root of T!
Why does the standard deviation scale as a square root, i.e. why does it flatten out? At some point random movements up and down start cancelling each other out when the variables are independent. This is a very important law of randomness!
Example 2: Physical Motion
Newton's law: $F = m * a$ and $m*dot(V) = F(x,V)$.
For simplicity we assume a particle moves either up or right.
dot(V) -> the time derivative of velocity. So tells you how much velocity changes through time.
What are relevant forces? Resistance, g-force. But the trajectory of a molecule is quite erratic; there are collisions with other particles. This "collision force" is not constant and can be modelled as a random force. We can again use a Gaussian model to model this randomness. This is obviously simplifying, but nevertheless this is clearly better because we won't gain anything from modelling each individual molecule.
These formulas use continuous time. To write this as a probabilistic model, we will discretize the differential equation. I.e. we have a $dt$ and approximate the differential equation for that time frame. You approximate the differential equation on dt (think high-school), then you rewrite this in a recursive form. This is Euler-Mayorama.
We apply this trick to estimate the new mean after dt; now the variance is expressed in terms of dt. Again, the same fundamental probabilistic dynamics!
If you do this discretization with a random force only, we have a wrong result! Then: $var(a, \eta) = a^2 var(\eta)$.
$x_{t+1} = x_t + dt\eta_t$ but then we end up with an error scaling with $dt^2$
TODO: didn't get this completely! What's the difference exactly?
Trick: usually you write second order derivatives in terms of two single derivative equations. I.e. express $v_{t+1}$ (which is a first-order derivative) and then $x_{t+1} = x_t + v(t)dt$.
N.B. I spotted several mistakes in the lecture notes.
- In the discretization in eq. 12, also the error should scale down with m
- in eq. 11, it should be -g*m instead of just -g
These results do not really invalidate the assignment I handed it, but in terms of getting the quantitatively correct answer they should be fixed.
General time series models
We ended up with a model where each successive time point can be seen as a regression on the previous time point. This is an autoregressive model
We write time series using the chain rule of probability.
This is similar to Markov processes, but Markov processes have no memory whatsoever. Note, it is assumes all fundamental natural processes are memoryless.
A time series is a bit more general, because it may take into account variables from further from the past. He said that we can turn non-Markov processes usually into Markov-processes by adding more variables (this is a bit counter-intuitive to me). Perhaps that's to remove any dependency on memory by adding all variables we need to the "previous timestep".
REVIEW: I may have gotten that the other way around.
Example: Natural Language Generation
Why does a language model need to be probabilistic? There are multiple ways of expressing the same and that are consistent with a sentence start and we want the model to use this freedom and be "creative".
Conditional independence
In the real world we can hardly say anything is really independent: otherwise we'd live in two universes. When I'm talking to you and examining spikes in your brain, we are connected through speech and your spikes have a dependence on the stimulus I'm providing. If you look long enough, all things are connected in a extremely complex way.
But statistically speaking we can with only a minor abstraction meaningfully speak of conditional independence. In real life we will probably not be able to identify the complete set of confounding variables.
Explicit Statistical models and machine learning
How can we model deterministic factors/knowledge as well as sources of uncertainty?
In Luca's terminoloy, a statistical model is one that is relatively free of external knowledge, i.e. it mainly learns from data.
How do we apply explicit modeling to machine learning?
Another advantage of explicit modeling is that the parameters have a bit more meaning.
Check lecture notes. Didn't take notes anymore.
$\eta_t = x_{t+1}-x_t$ used for loss.
Stochastic Gradient Descent
Often we cannot maximize the log-likelihood directly. When a probabilistic model is differentiable with respect to its parameters $\theta$, we can use gradient descent.
$$\theta_{n+1} = \theta_n - \eta \nabla \L(\theta_n)$$
Often we cannot compute the gradients over the whole dataset at once. In that case we compute the gradients over mini-batches of b datapoints, which is stochastic gradient descent:
$$\theta_{n+1} = \theta_n - \eta \sum_b \nabla log p(x^{(b)};\theta_n)$$
The Bayesian approach
Combine prior knowledge with data
Idea: explicitly model the parameters as prior knowledge. Then our model is now the join probability $p(\mathbf{x},\theta)=p(\mathbf{x}|\theta)p(\theta)$.
Using Bayes' theorem, we can now optimize $p(\theta|\mathbf{x})$ to find a posterior distribution over the parameters $\theta$.
MAP loss
Computing that posterior is expensive and sometimes we just want the best parameter estimate anyways.
The MAP loss defines a loss function that is minimized by the mode of the posterior, i.e. optimizing the MAP loss allows us to estimate the mode of the posterior directly using SGD.
$$\L(\theta)_{MAP} = \sum_j^J logp(\mathbf{x}^{(j)};\theta) + log p(\theta)$$
Q: In the above formulation (eq. 30 in the lecture notes), shouldn't we maximize this instead? If I'm not mistaken, we can derive above formula by taking the MAP over the log likelihood.
A: I asked in the lecture and indeed, you should maximize the above loss, or minimize the negative map loss.
Then we get $argmax log p(\theta|x) = argmax log( p(x|\theta)p(\theta))$, which then gives $argmax log(p(x|\theta))+log(p(\theta))$.
So when minimizing using SGD, we should instead take the negative of equation 30, right?
Even uninformative priors help counter overfitting.
In practice, MAP is often not satisfactory. E.g. in case of a medical diagnosis, the posterior quantifies the amount of uncertainty, which is relevant in the decision-making process.
Remarks from tutorial
We switched to PyTorch after all because Luca is making assignments on the fly and he prefers PyTorch. Also, in PyTorch we'll be focusing more on lower-level implementation.
Abstract class for probability model does some bookkeeping. Probability models essentially require implementation of two methods. One to get (differentiable) samples, the other for evaluating the log likelihood of the sample.
- We'll implement a Markov model.
self.paramsis just a dictionary of whatever params we need.- Note that probability distributions are also probability models.
- Implementation of random walk and "Brownian" motion model.
- Gradient descent on Brownian model with learnable parameters
- Something with unbounded space
- For the Markov models: we need to basically adjust the initial distributions and the transition probabilities; the Markov framework itself is already there.
- $x(t)=v(t)$ is deterministic so in this case doesn't add anything to the loss!
- Ex.4: with "recover" weights is meant that we try to recover the weights that were used for generating the data
- Length of time series is not defined; feel free to play around