<jupyter_start><jupyter_text>Spiking Neural Networks - 1:

The difference between an ordinary artificial neural network (ANN) and SNN is that, a SNN's neuron only activates/spikes upon crossing a threshold value. The most common model for simulating or exhibiting such spikes are integerate&fire or leaky integerate&fire. More powerful than traditional machine learning techniques, it is rare to see such powerful implementation not widespread in the booming ML/AI world. However, an important reason for such an effect, is the inability to differentiate such spiking or binary functions. In essence, it is our incapacity to train in a supervised fashion such a model with efficient learning progress. Therefore, it has seen rare use cases, and can only be trained using unsupervised learning methods.

 Installation:

`pip install brian2`<jupyter_code>!pip install brian2<jupyter_output>Requirement already satisfied: brian2 in /usr/local/lib/python3.7/dist-packages (2.4.2)
Requirement already satisfied: cython>=0.29 in /usr/local/lib/python3.7/dist-packages (from brian2) (0.29.22)
Requirement already satisfied: setuptools>=24.2 in /usr/local/lib/python3.7/dist-packages (from brian2) (54.0.0)
Requirement already satisfied: pyparsing in /usr/local/lib/python3.7/dist-packages (from brian2) (2.4.7)
Requirement already satisfied: numpy>=1.15 in /usr/local/lib/python3.7/dist-packages (from brian2) (1.19.5)
Requirement already satisfied: sympy>=1.2 in /usr/local/lib/python3.7/dist-packages (from brian2) (1.7.1)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python3.7/dist-packages (from brian2) (2.11.3)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.2->brian2) (1.2.1)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2>=2.7->brian2) (1.1.1)<jupyter_text>Imports:<jupyter_code>from brian2 import *
import matplotlib.pyplot as plt
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Units:<jupyter_code>20*volt
20*amp
1000*namp
10*nA
(10*nA)*(5*mohm)
5*ohm<jupyter_output><empty_output><jupyter_text>Defining a Neuron:<jupyter_code>tau = 10*ms
eqs = '''
dv/dt = (1-v)/tau : 1
'''
G = NeuronGroup(1, eqs)
G<jupyter_output><empty_output><jupyter_text>Running a simulation:<jupyter_code>start_scope()

tau = 10*ms
eqs = '''
dv/dt = (1-v)/tau : 1
'''

G = NeuronGroup(1, eqs)
run(100*ms)
start_scope()

G = NeuronGroup(1, eqs, method='exact')
print('Before v = %s' % G.v[0])
run(100*ms)
print('After v = %s' % G.v[0])<jupyter_output>Before v = 0.0
After v = 0.9999546000702376<jupyter_text>As, tau = 10ms and run limit = 100ms, therefore, t = \[0ms, 100ms\].

Now, looking at our differential equation, 
```
dv/dt = (1-v)/tau
```

thus, v = 1 - e^(-t/tau).

Going through with the limits at the two ends, we get v=0 @ t=0 and v=~1 @ t=100ms<jupyter_code>print('Expected value of v = %s' % (1-exp(-100*ms/tau)))<jupyter_output>Expected value of v = 0.9999546000702375<jupyter_text>Plotting our simulation:<jupyter_code>start_scope()

G = NeuronGroup(1, eqs, method='exact')
M = StateMonitor(G, 'v', record=True)

run(30*ms)

plot(M.t/ms, M.v[0])
xlabel('Time (ms)')
ylabel('v');<jupyter_output><empty_output><jupyter_text>Theoretical v/s Simulations:<jupyter_code>start_scope()

G = NeuronGroup(1, eqs, method='exact')
M = StateMonitor(G, 'v', record=0)

run(30*ms)

plot(M.t/ms, M.v[0], 'C0', label='Brian')
plot(M.t/ms, 1-exp(-M.t/tau), 'C1--',label='Analytic')
xlabel('Time (ms)')
ylabel('v')
legend();<jupyter_output><empty_output><jupyter_text>Simulation 2:<jupyter_code>start_scope()

tau = 10*ms
eqs = '''
dv/dt = (sin(2*pi*100*Hz*t)-v)/tau : 1
'''

G = NeuronGroup(1, eqs, method='euler')
M = StateMonitor(G, 'v', record=0)

G.v = 5 # initial value

run(60*ms)

plot(M.t/ms, M.v[0])
xlabel('Time (ms)')
ylabel('v');<jupyter_output><empty_output><jupyter_text>Simulation 3: Spikes<jupyter_code>start_scope()

tau = 10*ms
eqs = '''
dv/dt = (1-v)/tau : 1
'''

G = NeuronGroup(1, eqs, threshold='v>0.8', reset='v = 0', method='exact')

M = StateMonitor(G, 'v', record=0)
run(50*ms)
plot(M.t/ms, M.v[0])
xlabel('Time (ms)')
ylabel('v');
start_scope()

G = NeuronGroup(1, eqs, threshold='v>0.8', reset='v = 0', method='exact')

spikemon = SpikeMonitor(G)

run(50*ms)

print('Spike times: %s' % spikemon.t[:])
start_scope()

G = NeuronGroup(1, eqs, threshold='v>0.8', reset='v = 0', method='exact')

statemon = StateMonitor(G, 'v', record=0)
spikemon = SpikeMonitor(G)

run(50*ms)

plot(statemon.t/ms, statemon.v[0])
for t in spikemon.t:
    axvline(t/ms, ls='--', c='C1', lw=3)
xlabel('Time (ms)')
ylabel('v');<jupyter_output><empty_output><jupyter_text>Refractory:

This means that after the neuron fires a spike it becomes refractory for a certain duration and cannot fire another spike until this period is over.<jupyter_code>start_scope()

tau = 10*ms
eqs = '''
dv/dt = (1-v)/tau : 1 (unless refractory)
'''

G = NeuronGroup(1, eqs, threshold='v>0.8', reset='v = 0', refractory=5*ms, method='exact')

statemon = StateMonitor(G, 'v', record=0)
spikemon = SpikeMonitor(G)

run(50*ms)

plot(statemon.t/ms, statemon.v[0])
for t in spikemon.t:
    axvline(t/ms, ls='--', c='C1', lw=3)
xlabel('Time (ms)')
ylabel('v');
start_scope()

tau = 5*ms
eqs = '''
dv/dt = (1-v)/tau : 1
'''

G = NeuronGroup(1, eqs, threshold='v>0.8', reset='v = 0', refractory=15*ms, method='exact')

statemon = StateMonitor(G, 'v', record=0)
spikemon = SpikeMonitor(G)

run(50*ms)

plot(statemon.t/ms, statemon.v[0])
for t in spikemon.t:
    axvline(t/ms, ls='--', c='C1', lw=3)
axhline(0.8, ls=':', c='C2', lw=3)
xlabel('Time (ms)')
ylabel('v')
print("Spike times: %s" % spikemon.t[:])
start_scope()

tau = 5*ms
eqs = '''
dv/dt = (1-v)/tau : 1
'''

G = NeuronGroup(1, eqs, threshold='v>0.8', reset='v = 0', refractory=5*ms, method='exact')

statemon = StateMonitor(G, 'v', record=0)
spikemon = SpikeMonitor(G)

run(50*ms)

plot(spikemon.t/ms, spikemon.i, '.k')
xlim([0, 50])
xlabel('Time (ms)')
ylabel('Neuron Spikes');<jupyter_output><empty_output><jupyter_text>Multiple Neurons<jupyter_code>start_scope()

N = 100
tau = 10*ms
eqs = '''
dv/dt = (2-v)/tau : 1
'''

G = NeuronGroup(N, eqs, threshold='v>1', reset='v=0', method='exact')
G.v = 'rand()'

spikemon = SpikeMonitor(G)

run(50*ms)

plot(spikemon.t/ms, spikemon.i, '.k')
xlabel('Time (ms)')
ylabel('Neuron index');
start_scope()

N = 100
tau = 10*ms
v0_max = 3.
duration = 1000*ms

eqs = '''
dv/dt = (v0-v)/tau : 1 (unless refractory)
v0 : 1
'''

G = NeuronGroup(N, eqs, threshold='v>1', reset='v=0', refractory=5*ms, method='exact')
M = SpikeMonitor(G)

G.v0 = 'i*v0_max/(N-1)'

run(duration)

figure(figsize=(12,4))
subplot(121)
plot(M.t/ms, M.i, '.k')
xlabel('Time (ms)')
ylabel('Neuron index')
subplot(122)
plot(G.v0, M.count/duration)
xlabel('v0')
ylabel('Firing rate (sp/s)');<jupyter_output><empty_output><jupyter_text>Modelling Noise:

by introducing xi<jupyter_code>start_scope()

N = 100
tau = 10*ms
v0_max = 3.
duration = 1000*ms
sigma = 0.2

eqs = '''
dv/dt = (v0-v)/tau+sigma*xi*tau**-0.5 : 1 (unless refractory)
v0 : 1
'''

G = NeuronGroup(N, eqs, threshold='v>1', reset='v=0', refractory=5*ms, method='euler')
M = SpikeMonitor(G)

G.v0 = 'i*v0_max/(N-1)'

run(duration)

figure(figsize=(12,4))
subplot(121)
plot(M.t/ms, M.i, '.k')
xlabel('Time (ms)')
ylabel('Neuron index')
subplot(122)
plot(G.v0, M.count/duration)
xlabel('v0')
ylabel('Firing rate (sp/s)');<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1799375487900078, 'fraction_numerical': 0.0468384074941452, 'mean_word_length': 4.648052902277737, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 70, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '92395', 'license': 'MIT', 'n_tokens_mistral': 3450, 'n_tokens_neox': 3211, 'n_words': 693, 'path': 'spike_basics.ipynb', 'repo_name': 'AmanPriyanshu/Spiking-Neural-Networks', 'size': '7686'}