STDP learning rule
- Long Term Potentiation (LTP): if a presynaptic neuron fires briefly (~10ms) before the postsynaptic neuron, their weighted connection is strengthened.
- Long Term Depression (LTD): if a presynaptic neuron fires briefly (~10ms) after the postsynaptic neuron, their weighted connection is weakened.
"Long term" is relative to events that only last milliseconds.
To learn we want to know $$\delta w$$.
There are two cases.
- $t_{pre}-t_{post} \leq 0$
So this is the scenario where post fires after pre (or pre before post) In this case we need a positive learning rate A > 0
- $t_{pre}- t_{post} > 0$
Now pre fires after post and we need a strictly negative learning rate B.
The rule then looks like
$$\delta w = [A or B] e^{\frac{-(|t_{pre}-t_{post}|)}{\tau}}$$
$\tau$ is here a normalization constant that expresses the time window. Also notice that we take the absolute value of the difference and then the negative of that.
Q: What if the time difference is bigger than the time window we can measure?
Building some intuition:
- Having a negative exponent ensures that the computed value will fall into [0,1]
- If the difference is smaller (e.g. 0.1), then the weight change will be bigger: $e^{-0.1} \approx 0.9$
- So the weight update will either be strongly positive or strongly negative, depending on whether we pick A or B.
- If the difference is bigger (e.g. 0.9), then the weight change will be smaller: $e^{-0.9} \approx 0.41$
So to conclude, this rule makes weight updates sensitive to situations where two spikes are very close to each other, strengthening their connection in one case and weakening them in the other.
TODO; review: Why is it capped between -0.5 and 0.5?
So basically the SDTP learning rule is Hebbian rule in a temporal context.
A and B are learning rates. Tau is a normalization that takes into account what your measuring window is.
Expressions are on long term dynamics to be in accordance with the "speed of the simulation".
STDP shapes neural selectivity
STDP affects a neuron's behavior in response to repeated spike patterns
-
The latency between post and presynaptic potential is reduced if STDP continues
- Loop: you strengthened the connection, but then in the future the same amount of input will sooner cause a potential; this in turn strengthens the connection again.
- On network level, this increases the speed of neuronal processing of repeatedly presented stimulus patterns, because these will be STDP-modulated spike trains.
-
Conversely, if pre consistently spikes after post consistently, they are apparently uncorrelated. Because now we reduce the strength of this connection, this connection will basically vanish.
- This promotes sparseness in the network, because unimportant connections are reduced.
To summarize, in STDP LTP increases the processing speed of relevant input patterns, and LTD promotes sparseness in the network by reducing irrelevant connections.
STDP backpropagation and supervised learning
In the context of SNNs: minimize error with desired spike train.
Backpropagation:
$$d^{\mu}{j} = g' a^{\mu}{j} \sum_{k} w_{kj}\delta^{\mu}_{k}$$
Three main issues with backpropagation in SNNs:
- Activation function g' (non-linear) must be differentiable
- But in SNN we have a discrete binary decision, 1 or 0; to spike or not to spike. So no general derivative!
- There need to be symmetric feedback weights ('weight transport problem')
- In the brain, such a feedback mapping is highly implausible; how are weight updates transported backwards?
- The feedback synapses would have to be the same as the feedforward connections
- credit assignment problem: How do you determine the weight adjustment w.r.t. a global signal based on local information alone?
Alternatives:
- Use substitute solutions e.g. assume a surrogate gradient
- Assume that you have a different non-linearity in the feedback connections than in the forward connections, such that this activation function is differentiable; e.g. sigmoid.
- Loses biological plausibility
- Use random feedback weights (only works in specific conditions)
- See Lillicrap et al. 2016; system may still be able to do some backpropagation