Lecture 3: Machine Learning with Spikes

This is a "highlight" lecture, because we can't cover much.

All discussed material is in Tavanaei 2019 et al.: Deep learning in spiking neural networks.

Learning objectives

  • Be able to explain how STDP works and how STDP affects single neuron responses and network level behavior in general.
  • Why it is hard to implement backpropagation in spiking neural networks
  • Why it's hard to train large-scale neural networks.
  • How we can scale up spiking neural networks

ANNs vs. SNNs

Regular Artificial Neural Networks have clear computing units and non-linear differentiable activation functions.

Without non-linearity stacking multiple layers is useless. Because the used functions are differentiable, we can use gradient-based optimization methods (SGD, ADAM, etc.).

Years ago people saw potential in ANN, but it was just too hard too train them. Larger datasets, GPU tech and improved regularization allows the "deep" learning revolution.

First DNN: AlexNet.

Even though ANNs are "brain-inspired", they really differ in their structure, computations, and learning dynamics.

Most striking difference: ANNs use continuous activations whereas the brain uses discrete spikes.

SNN principles

Information:

  • More biologically plausible
  • SNNs convey information through spike trains
  • Spikes can be characterized as sparse events and have a high information content.
    • In an ANN there is continuous communication, versus event-based communication
    • So in SNN a spike is a rare event and contains a lot of information
  • Information: spike-timing, latencies (inter spike-time interval), spike rates.

Energy:

  • Because they are sparse, they are energy efficient.
  • This is because computation is now event-driven, instead of constantly feeding activations back and forth.

Temporality and local learning:

  • Sensitivity to temporal aspects of information transmission that is known to be important in neural coding
    • E.g. Binaural coding (different latency between sounds arriving at ears )
    • This happens so fast that there is no time to integrate this information
    • So this is probably encoded directly in the spike timing
  • Combined with a local learning rule that recognizes "spatio-temporal patterns"
    • The brain only has access to local information (that's on the synaptic path) for learning
    • Learning is done on things that are close in time and close in space (so spatio-temporal)
    • So SNN is a good paradigm for studying neural information processing.

SNNs: architecture STDP learning rule

  • Analog input has to be encoded into spikes
  • Extremely simplified w.r.t. real neurons (that are thought to have around +-50 relevant state variables)

Most popular model: leaky integrate-and-fire (LIF) neurons,

  • input accumulation
  • rudimentary memory
    • potential that slowly decades; i.e. keeps some "memory" of past inputs.
  • threshold dynamics

  • Offline learning, conversion

  • Backpropagation
  • STDP local learning

Unsupervised learning via STDP

The learning rules of spiking nets cannot be replicated in non-spiking nets.

Neuroscientists have found many different variants that fall under the umbrella term "spike-time-dependent plasticity".

STDP learning rule

Consolidated here: 202010231520-STDP_learning_rule.md

  • Long Term Potentiation (LTP): if a presynaptic neuron fires briefly (~10ms) before the postsynaptic neuron, their weighted connection is strengthened.
  • Long Term Depression (LTD): if a presynaptic neuron fires briefly (~10ms) after the postsynaptic neuron, their weighted connection is weakened.

"Long term" is relative to events that only last milliseconds.

To learn we want to know $$\delta w$$.

There are two cases.

  1. $t_{pre}-t_{post} \leq 0$

So this is the scenario where post fires after pre (or pre before post) In this case we need a positive learning rate A > 0

  1. $t_{pre}- t_{post} > 0$

Now pre fires after post and we need a strictly negative learning rate B.

The rule then looks like

$$\delta w = [A or B] e^{\frac{-(|t_{pre}-t_{post}|)}{\tau}}$$

$\tau$ is here a normalization constant that expresses the time window. Also notice that we take the absolute value of the difference and then the negative of that.

Q: What if the time difference is bigger than the time window we can measure?

Building some intuition:

  • Having a negative exponent ensures that the computed value will fall into [0,1]
  • If the difference is smaller (e.g. 0.1), then the weight change will be bigger: $e^{-0.1} \approx 0.9$
    • So the weight update will either be strongly positive or strongly negative, depending on whether we pick A or B.
  • If the difference is bigger (e.g. 0.9), then the weight change will be smaller: $e^{-0.9} \approx 0.41$

So to conclude, this rule makes weight updates sensitive to situations where two spikes are very close to each other, strengthening their connection in one case and weakening them in the other.

 STDP_Curve.JPG

TODO; review: Why is it capped between -0.5 and 0.5?

So basically the SDTP learning rule is Hebbian rule in a temporal context.

A and B are learning rates. Tau is a normalization that takes into account what your measuring window is.

Expressions are on long term dynamics to be in accordance with the "speed of the simulation".

STDP shapes neural selectivity

STDP affects a neuron's behavior in response to repeated spike patterns

  • The latency between post and presynaptic potential is reduced if STDP continues

    • Loop: you strengthened the connection, but then in the future the same amount of input will sooner cause a potential; this in turn strengthens the connection again.
    • On network level, this increases the speed of neuronal processing of repeatedly presented stimulus patterns, because these will be STDP-modulated spike trains.
  • Conversely, if pre consistently spikes after post consistently, they are apparently uncorrelated. Because now we reduce the strength of this connection, this connection will basically vanish.

    • This promotes sparseness in the network, because unimportant connections are reduced.

To summarize, in STDP LTP increases the processing speed of relevant input patterns, and LTD promotes sparseness in the network by reducing irrelevant connections.

STDP backpropagation and supervised learning

In the context of SNNs: minimize error with desired spike train.

Backpropagation:

$$d^{\mu}{j} = g' a^{\mu}{j} \sum_{k} w_{kj}\delta^{\mu}_{k}$$

Three main issues with backpropagation in SNNs (or two, really?):

  1. Activation function g' (non-linear) must be differentiable
    • But in SNN we have a discrete binary decision, 1 or 0; to spike or not to spike. So no general derivative!
  2. There need to be symmetric feedback weights ('weight transport problem')
    • Backpropagation uses feedforward weights; so the feedback synapses would have to be the same as the feedforward connections
    • In the brain, such a feedback mapping is highly implausible; how are weight updates transported backwards?
  3. credit assignment problem: How do you determine the weight adjustment w.r.t. a global signal based on local information alone?

Alternatives:

  • Use substitute solutions e.g. assume a surrogate gradient
    • Assume that you have a different non-linearity in the feedback connections than in the forward connections, such that this activation function is differentiable; e.g. sigmoid.
    • Loses biological plausibility
  • Use random feedback weights (only works in specific conditions)
    • See Lillicrap et al. 2016; system may still be able to do some backpropagation

See slide for paper list; project ideas:

  • Spiking backprop (Mostafa, 2017)
  • ResuMe (Ponulak & Kasinski, 2010)
  • BP-STDP (Tavanaei & Maida 2018)
  • Maximum Likelihood methods (Pfister et al. 2006)
    • PROJECT IDEA?
  • Competitive learning with STDP (Wang et al. 2014)

Approaches to scaling up SNNs

To tackle more interesting problems, SNNs need to be scaled up in width (amount of neurons) and depth (layers).

However, traditional SNNs with STDP-like local learning suffer from credit assignment problem: How do you determine the weight adjustment w.r.t. a global signal based on local information alone?

So how can we scale up SNNs that rely on local learning?

  • How to scale?
    • ANN-to-SNN conversion
    • Backpropagation on SNNs
    • Scale up STDP-based approaches

Q: What's the credit assignment problem?

ANN-to-SNN conversion

SNN are hard to train but are extremely efficient. ANN are easier to train but extremely inefficient.

ANN is not handy when fast and efficient inference is needed.

Get the best of both worlds: we can train an ANN, and then convert to an SNN for inference.

  1. Transfer learned parameters to an equivalent SNN
  2. Apply transformations
    • Normalization to preserve functional performance
    • Quantization may be necessary to run on hardware; need to be stored to be used in 32bit or 64 bit CPU.

This approach is currently one of the top available choices.

Slide 26: study results for ANN-to-SNN conversion.

Pfeiffer & Liu 2016 used 500 different SNNs. Some were optimized to be accurate, others were optimized to be fast, others to be energy efficient.

Vertical dotted black line represents traditional ANN. On x-axis: number of compute operations. Additional labels on x-axis indicate the amount of computations needed to reach 98% performance on MNIST.

The results show that most SNNs could reach that performance threshold in fewer steps than a traditional ANN, in the best case in only 25% of the amount of computation steps (one optimized for speed/efficiency).

Backpropagation in SNNs

Why is backpropagation important for scaling? If we would have backpropagation for SNNs we would be able to propagate errors throughout a large network; otherwise we are limited to local learning only. Backpropagation is essential for modern day ANN applications.

This is dense and leaves out intuition building; ok if we don't fully get the details.

Lee et al. 2016: treat the neuron's membrane potential as a differentiable signal, analogous to non-linear activation functions in regular deep neural networks.

LIF neuron model:

$$v_{mp}(t_p) = V_{mp}(t_{p-1}e^{\frac{t_{p-1}-t_p}{\tau_{mp}}}) + w_i^{(p)w_{dyn}}$$

Terms:

  • $V_{mp}$: membrane potential
  • $t_p$: spike time
  • exponential decay of the membrane; the "leaky part"
  • $w_i^{(p)}$ synaptic input i to the neuron
    • p represents whether there is an input spike at that time
  • $w_{dyn}$: 0 during refractory period where the neuron does not respond to input

In words: the voltage of a membrane at a timestep is the voltage from the previous timestep scaled down with a decay plus the current weighted incoming spikes. If the refractory period is active, all incoming inputs are ignored.

The issue is: spikes are not differentiable.

Lee et al. define two terms.

  1. Accumulated effect of k-th active input synapse for all previous timepoints $t_p$ with spikes

$$x_k(t)=\sum_{p} e^{\frac{t_{p}-t}$$

This can be seen as low-pass filtering the input (?).

  1. The same but now for the output spike times.

$$a_i(t)=\sum_{q} e^{\frac{t_{q}-t}$$

These are differentiable and the LIF neuron model can be rewritten in these terms. Lee et al. make some assumptions however.

  • Each layer is WTA: a "winner takes all" circuit.
    • All neurons in a layer have inhibitory connections with each other. So the first one that spikes inhibits all others; so winner takes all.
    • Also guarantees sparse outputs for each layer.

Q: What are the benefits of a "winner takes all" circuit?

Reformulation of LIF model:

  1. Weighted sum over inputs from previous layer $w_{ik}x_{k}(t)$
  2. Minus membrane threshold $-V_{th}a_i(t)$: reset voltage after output spike $a_i(t)$.
  3. Right hand side is lateral inhibition; so input from all other neurons in the layer except itself, which will be a negative number
  4. $\sigma$ modulates the strength of the inhibition.

Based on this reformulation, they derive an activation function $a_i$ that is in fact differentiable.

What can we do now?

  1. Optimize the feedback connection
  2. Optimize the membrane threshold.
  3. Optimize the inhibitory relations

Note that between layers there is only one connection with spikes due to the sparseness induced by the WTA design.

Scale up STDP

Try to scale op STDP for deeper neural networks.

Most successful is a form of layer-wise STDP.

Tavanaei & Maida, 2017: Spiking CNN with layer-wise STDP.

  1. Convolutional pooling layer
  2. Fully connected layers for feature extraction
  3. SVM for final classifier

But... the input and output will be spikes.

Input: input image by spike trains. Spike trains based on normalized grayscale pixel intensities in range (0,1).

Idea: Have a convolution filter and integrate over a time window (e.g. 20 timesteps), and record the number of spikes, then fire if a threshold is reached.

Units in feature maps are LIF neurons. Each neuron in a feature map accumulates convolution results of image patch over 20 timesteps, and fires when a threshold is reached. This indicates that a certain feature is present at a certain location (the one to which the filter is applied). Really funny concept! The input now also has a temporal dimension.

Slide 37; k is feature map; filter is p x p.

  • $s_m(i,j,t)$ : at timestep t, was there a spike at pixel (i,j) "in the picture"?
  • Multiply that by weights of the feature map
  • Do that for 20 timesteps

Feature discovery layer

Generates samples in which each neuron corresponds to a feature.

Threshold depends on membrane potential and firing probability (usually expressed with a softmax). Again LIF neurons.

This is a way to introduce competition, Abdullahi said, because softmax is a distribution; so the probability mass of is divided over all output neurons.

So we basically have this softmax:

$P(z_h(t)=1|W,y_t) = \frac{ e^{W^T_h * y_t}}{ \sum_{j=1}^H e^{W^T_h * y_t} }$

Q: A few slides back you mentioned that firing probability is included in the threshold because then you get "competition for free". That sounded interesting but went a bit too fast for me. Why was that again?

Convolution layer and feature discovery layers where trained layer-wise using STDP.

SVM is used for taking the output spikes and translating that back into predictions.

Performance comparisons

A limitation is that current benchmarking is done on very easy datasets such as MNIST. It's hard to extrapolate the performance of SNNs to more complex problems currently.

W.r.t. benchmarking, the strengths of SNNs do not come forward in conventional static ML data sets.

Perhaps more dynamic data with spatio-temporal aspects; and with noise. SNNs can be way more noise robust.

Nengo Assignment

Learn how to work with Nengo. And learn about learning in Nengo.

Have a look at the review paper.