Stochastic Gradient Descent
When a probabilistic model is differentiable with respect to its parameters $\theta$, we can use gradient descent.
$$\theta_{n+1} = \theta_n - \eta \nabla \mathbb{L}(\theta_n)$$
Often we cannot compute the gradients over the whole dataset at once. In that case we compute the gradients over mini-batches of b datapoints, which is stochastic gradient descent:
$$\theta_{n+1} = \theta_n - \eta \sum_b \nabla log p(x^{(b)};\theta_n)$$
It's called "stochastic" because we randomly sample mini-batches. (Usually, you first shuffle all the data and then take sub-samples)
Note that strictly speaking, what is called SGD nowadays is actually mini-batch gradient descent. The original stochastic gradient descent computed the gradient over one random sample at a time.