Exponential Smoothing in time series
@time_series @smoothing
Forecasting techniques preceding the idea of exponential smoothing are:
1) Moving averages over a "rolling" window 2) Weighted (moving) average - This is a simple modification to the moving average by assigning relatively more weight to recent observations.
The weighted moving average defines k decreasing weights for the past k observations.
What if we extend this idea to all available past observations and assign increasingly lower weights over them?
Single exponential smoothing
This is the main idea of exponential smoothing
$$ \hat{y}{t} = \alpha \cdot y{t} + (1 - \alpha) \cdot - \hat{y}_{t-1}$$
Where $\alpha$ is a "smoothing factor" that determines how fast the latest observation will be "forgotten."
In other words, a bigger smoothing factor will prioritize recent observations, and a smaller smoothing factor will assign more importance to past observations.
Alpha determines a super simple distribution between past (1-alpha) and current (alpha), summing up to 1.
So the easiest way to interpret alpha is to ask: how important are recent observations? (This is more intuitive, because "more" alpha is not more smoothing, but less).
Why is this exponential?
It is important to realize that you multiple each past observation/prediction with 1-alpha. This value was in turn also weighted with the smoothing scheme, so you effectively stack multiplications of (1-alpha).
All values for the smoothing parameters are constrained between [0,1], also for the methods below.
Double exponential smoothing
The same ideas can be applied to not predict one, but two future points.
For this it is required to do series decomposition of the series into two different components:
- its @intercept (level; expected series value)
- its slope (rend)
To first get the right level at a time point, you apply single point exponential smoothing like before, only now using the decomposed notation.
The reasoning is simple: the level at timepoint t is the level of timepoint t-1 plus the slope (N.B. slope is by definition the increase per timepoint, that's why simply adding it is sufficient).
This gives for the level at timepoint t:
$$l_{x} = \alpha y_{x} + (1 - \alpha )( l_{x-1} + b_{x-1})$$
So you apply the same weighting scheme as before, the main difference is that you write $y_{t-1}$ in its decomposed form.
Then what is left is calculating the slope b. Similar to $\alpha$, you now have a smoothing parameter $\beta$:
$$b_{x} = \beta( l_{x} - l{x_1}) + (1 - \beta)b_{x-1}$$
The current slope is defined, trivially, as the difference between the current and previous value. This in itself applies no smoothing whatsoever. To add smoothing, you also consider pasts slopes. This makes the assumption that the future trend of the time series is not only influenced by the directly preceding moment, but by the weighted changes of past time points as well.
So, crucially, you now have two smoothing parameters, alpha for the expected level, and beta for the expected trend.
The prediction of the next timepoint t+1 is made by adding the current level and slope:
$$\hat{y}{x+1} = l{x} + b_{x}$$
Triple exponential smoothing (Holt-Winters)
More info on triple exponential smoothing
If you want to produce confidence intervals with triple smoothing for application in anomaly detecten, for example, have a look at Brutlag's algorithm (pdf).