Spiking Neural Netorks (SNN)
Artificial Neural Networks have been around for many years (since 1943 through the work of McCouloch and Pitts) and is one of the main lines of research of the Artificial Intelligence community. Basically artificial neural networks are processing units capable of pairing a certain output to a given set of inputs which in essence means they can be trained to activate on a specific set of cases and generalize to unknown inputs. This flexibility and adaptability is what makes artificial neurons so powerful.
In recent years there has been a trend among researches, product of the twilight of artificial intelligence (where it was shown that classic models of artificial neurons couldn't correctly replicate the human brain), to create artificial neurons that emulate more closely the behavior of biological neurons. One such attempt comes from the development of Spiking Neural Networks or SNN for short.
In recent years there has been a trend among researches, product of the twilight of artificial intelligence (where it was shown that classic models of artificial neurons couldn't correctly replicate the human brain), to create artificial neurons that emulate more closely the behavior of biological neurons. One such attempt comes from the development of Spiking Neural Networks or SNN for short.
How do they work?
Spiking Neurons came to be from the idea that information can be coded through the time of arrival of incoming impulses. In this case the neuron fires only when the membrane potential reaches a certain value and then resets, after which it sets a recovery period in which it doesn't respond to inputs before reactivating.
Famous Spiking Neuron Models
0) Reflex neuron model
The simplest neuron model found in nature. It can be found in animals such as the Aplysia which is a mollusk that possesses an instinctive withdrawal reflex evolved as a defense mechanism against predators.
Matlab code.
v = 4; pls = [0 0 1 0 0]; x = [pls pls pls pls pls pls]; [nRows nCols] = size(x); nTs = nCols; y = zeros(1, nTs); for t=1:nTs y(t)=v*x(t); end plot(x) plot(y) |
Python code.
|
Habituation
The simplest form of learning is habituation. It involves a decrease in reflex sensitivity due to repeated stimulation. That means that a neuron that is continually excited will find it harder and harder to spike.
The Aplysia for example may get habituated to the effect of water currents over its body and thus stop treating them as a danger. In this case we added a new variable called red which reduces the input during every spiking cycle of the neuron. Value was set as 0.7 or 70% of the spiking value.
The Aplysia for example may get habituated to the effect of water currents over its body and thus stop treating them as a danger. In this case we added a new variable called red which reduces the input during every spiking cycle of the neuron. Value was set as 0.7 or 70% of the spiking value.
|
Positive feedback neuron response
snn03_positivefeedback.py | |
File Size: | 1 kb |
File Type: | py |
By setting feedback weight w = 1 the neuron converts an input response into an output step due to the lack of decay in the signal. It can be said that with w = 1 the neuron has essentially integrated the input.
|
By setting a bound and saturation level we can account for the non-linearity of neuron model (thresholding).
|
1) Integrate and Fire
The most basic of the spiking neuron models.
|
This example takes three input trains and computes the output to each of them.
|
|
Yet another leaky integrate and fire example based on the code found [3].
|
1.1) Leaky Integrate and Fire (LIF)
Series Leaky Neuron.
|
1.2) Extended Integrate and Fire (extIF)
1.3) Spike Response Model (SRM)
2) Hodgkin-Huxley model (HH)
One of the most biologically plausible models available, based on studies on the squid axon. Its computationally expensive.
3) Izhikevich neuron model
A compromise between LIF computational cost and Hodgin-Huxley's biological plausibility, the Izhikevich neuron model provides the possibility to imitate common behaviors of biological neurons using only 2 equations and 4 parameters [4].
|
References
[1] https://en.wikipedia.org/wiki/Spiking_neural_network
[2] https://msdn.microsoft.com/en-us/magazine/mt422587.aspx
[3] http://neurdon.wpengine.com/2011/03/06/spiking-neural-networks-in-python-part-1/
[4] http://neurdon.wpengine.com/2011/02/02/neural-modeling-with-python-part-3/
[5] http://cs.ioc.ee/yik/schools/win2007/paun/snppalmse.pdf
[6] http://www.neurotheory.columbia.edu/pdfs/Ostojic2011.pdf
[7] https://gist.github.com/aaron-santos/a133899e3f2accbca4a1
[8] http://compneuro.uwaterloo.ca/files/publications/eliasmith.2012.pdf
[9] http://www.izhikevich.org/publications/dsn.pdf
[10] http://stackoverflow.com/questions/36050119/simulating-a-neuron-spike-train-in-python
[11] https://praneethnamburi.wordpress.com/2015/02/05/simulating-neural-spike-trains/
[12] http://www.dreamincode.net/forums/topic/72868-a-simple-neuron-model-the-integrate-and-fire-neuron/
[13] http://www.brytewave.com/webapp/wcs/stores/servlet/ProductDisplay?catalogId=10001&categoryId=9602&storeId=216405&productId=4000000000003077708&langId=-1
[14] https://en.wikibooks.org/wiki/Sensory_Systems/Computer_Models/NeuralSimulation
[2] https://msdn.microsoft.com/en-us/magazine/mt422587.aspx
[3] http://neurdon.wpengine.com/2011/03/06/spiking-neural-networks-in-python-part-1/
[4] http://neurdon.wpengine.com/2011/02/02/neural-modeling-with-python-part-3/
[5] http://cs.ioc.ee/yik/schools/win2007/paun/snppalmse.pdf
[6] http://www.neurotheory.columbia.edu/pdfs/Ostojic2011.pdf
[7] https://gist.github.com/aaron-santos/a133899e3f2accbca4a1
[8] http://compneuro.uwaterloo.ca/files/publications/eliasmith.2012.pdf
[9] http://www.izhikevich.org/publications/dsn.pdf
[10] http://stackoverflow.com/questions/36050119/simulating-a-neuron-spike-train-in-python
[11] https://praneethnamburi.wordpress.com/2015/02/05/simulating-neural-spike-trains/
[12] http://www.dreamincode.net/forums/topic/72868-a-simple-neuron-model-the-integrate-and-fire-neuron/
[13] http://www.brytewave.com/webapp/wcs/stores/servlet/ProductDisplay?catalogId=10001&categoryId=9602&storeId=216405&productId=4000000000003077708&langId=-1
[14] https://en.wikibooks.org/wiki/Sensory_Systems/Computer_Models/NeuralSimulation