Neuromorphic Computing: Guest Lecture Johan Mentink
- Limitations of existing tech
- Physical limits for bit operations
- Potentials
- Neuromorphic Spintronics
- Outlook
Power densities
See slides for graph.
Orange arrow: power density of a cooking plate, enough to cook an egg.
Red arrow: power density of nuclear reactor.
About 75 iPhones needed to meet the power of a fuel rod in a nuclear reactor.
In practice, industry lowers clock speed to avoid that.
Transistors require only 1 attoJoule to switch a bit.
The sheer power density comes from the insane amount of transistors.
Physical limits
But what's the physical limit required for a single bit switch?
The lower bound energy consumption is determined by switching between 0 and 1 (i.e. writing)
Landauer limit:
Logical irreversibility <-> physical irreversibility.
Thermodynamics
First law: conservation of energy (E)
$\Delta E = Q + W$
Where Q is the heat supplied and W is the work done.
Insight here in 19th century is that
Second law: Entropy(S)
Physics S = see formula slides.
Boltzmann constant put in front for consistenty physics and thermodynamics.
Law deals with irreversible processes. If you do nothing, the preservation of energy does not apply in practice; energy is usually not supplied back. I.e. in nature entropy stays the same or increases.
If entropy lowers, we know we had to put some work in.
Weird: if you switch from 0 to 1, there's no loss in entropy; so no heat dissipation; so no work to be done. That's not very intuitive to me.
Faster and with less energy
10x faster, 1000x more efficient. (slide says 10000x but is outdated).
When going down to 0.01 microWatt, we could increase clocking speed to 10 Tera Hertz. Now we are stuck at 3/4 GHz.
Potential neuromorphic computing
See graph.
The landau limit is determined for bit flips only. But transfer of bits between parts of the system also takes energy.
So we are not currently reaching the limit because the connectivity is limiting.
Supercomputer:
2-3
Data is stored where it is processed; processing and storage integrated -> parallel.
10 Hz, 10 Watt; review slide.
How much more inefficient is the brain than the theoretical limit?
Theoretical power for the computations of brains do: 3 microWatt. So theoretically we could improve on the brain as well.
Neuromorphic computing with CMOS
You need more transistors than you have neurons;
Numbers 1000x efficiency programming; but this is in operating mode.
In programming mode, we have 120 pJ/synapse (Loihi).
Neuromorphic Computing with Spintronics
Current chips still operate in CMOS.
See published paper.
- Spintronics
- MRAM
Outlook on how we can move beyond that. Discussion of optical control of magnets.
- "nonvolatile, nonlinear, stable at the nanoscale"
- "dynamical connectivity"
Main benefit of reversible computing is that you get rid of heat dissipation.
Thanks for the guest lecture! I have a question about the theoretical limit: this limit assumed that the prior state of the bit is unknown. However, we normally are able to retrieve the state of a bit, right? Does this assumption then still hold?
Johan Mentink 11:20 AM: yes if you know the initial state you can in principle compute without energy dissipation. This is called reversible computing (since it avoids any irreversible proces)