Smooth nonlinear fitting scheme for analog multiplierless implementation of Hindmarsh - Rose neuron model

The Hindmarsh-Rose (HR) neuron model is built to describe the neuron electrical activities. Due to the polynomial nonlinearities, multipliers are required to implement the HR neuron model in analog. In order to avoid the multipliers, this brief presents a novel smooth nonlinear fitting scheme. We first construct two nonlinear fitting functions using the composite hyperbolic tangent functions and then implement an analog multiplierless circuit for the two-dimensional (2D) or three- dimensional (3D) HR neuron model. To exhibit the nonlinear fitting effects, numerical simulations and hardware experiments for the fitted HR neuron model are provided successively. The results show that the fitted HR neuron model with analog multiplierless circuit can display different operation patterns of resting, periodic spiking, and periodic/chaotic bursting, entirely behaving like the original HR neuron model. The analog multiplierless circuit has the advantage of low implementation cost and thereby it might be suitable for the hardware implementation of large-scale neural networks. ________________________________________

Circuit implementations of mathematical neuron models and large-scale neural networks on a hardware level have received much attention recently [21]. Based on the off-the-shelf analog components and on-chip digital devices, numerous analog and digital neuromorphic circuits were developed to simulate standalone neurons and coupled neural networks [22,23]. Due to the power-efficiency and compactness of analog circuits, mathematical neuron models were widely implemented in analog [24][25][26].
Based on numerous off-the-shelf discrete components, Bao et al optimized an analog circuit design to implement an adapting synapse-based neuron model [24], Behdad et al presented an experimental electronic neuron to achieve a complete Morris-Lecar model [25], and Arthur and Boahen developed a silicon-based integrate-and-fire neuron using a dynamical system approach [26].
However, when complex nonlinearities are involved, hardware circuit implementations for mathematical neuron models become extremely difficult. To solve this issue, a large number of digitally FPGA-based piecewise liner approximation approaches were proposed to implement neuromorphic circuits without multipliers [27][28][29][30][31][32][33]. Using the digital multiplierless implementation method, Jokar and Soleimani aimed at a calcium-based plasticity model [27], Gomar and Ahmadi focused on a biological adaptive-exponential neuron model [28], and Hayati et al sighted a two-coupled biological HR neuron model [29]. Meanwhile, Rahimian et al implemented a two-compartmental Pinsky-Rinzel pyramidal neuron model in digital [30], Imani et al investigated the multiplierless realization of a coupled Wilson neuron model in digital [32], and Haghiri et al employed a low-cost digital design to implement the noisy Izhikevich neuron model without multipliers [33].
These electronic neurons with analog and digital circuit implementations are capable to reproduce lots of different neuron dynamics and spiking/bursting behaviors that might appear in biological neurons [25,31]. Encouraged by the above piecewise liner approximation approaches, this paper presents a novel smooth nonlinear fitting scheme to implement an analog multiplierless circuit for the 2D/3D HR neuron model. Using this smooth nonlinear fitting scheme, the polynomial nonlinearities in the 2D/3D HR neuron model can be perfectly fitted by the composite hyperbolic tangent functions. Therefore, the presented smooth nonlinear fitting scheme is suitable for implementing electronic neurons in analog and the designed nonlinear fitting functions are smooth and continuously differentiable. By contrast, the piecewise liner approximation approaches reported in [27][28][29][30][31][32][33] are applied by the digitally FPGA-based electronic neurons, and the utilized piecewise linear functions are non-smooth and continuously non-differentiable.
The contributions of this paper are summarized as follows. (1) A novel smooth nonlinear fitting scheme is presented to multiplierlessly implement the 2D/3D HR neuron model and different neuron dynamics and spiking/bursting behaviors are reproduced. (2) An analog multiplierless circuit for the 2D/3D HR neuron model is designed and fabricated on a printed circuit board (PCB). Hardware experiments show that the analog multiplierless circuit can perfectly generate different operation patterns, such as the resting pattern, periodic/chaotic bursting patterns, and periodic spiking patterns with different oscillating frequencies.
The rest of this paper is organized as follows. Section 2 presents a smooth nonlinear fitting scheme and constructs two nonlinear fitting functions for the HR neuron model. Section 3 designs the circuit modules of the smooth nonlinear fitting functions and develops a hardware device for the analog multiplierless circuit. Finally, this paper is briefly concluded in Section 4.

Nonlinear fitting scheme for the HR neuron model
This section presents a smooth nonlinear fitting scheme to implement two polynomial nonlinearities in the HR neuron model.

Brief descriptions for the HR neuron model
Simplified from the Hodgkin-Huxley neuron model [4], the 2D HR neuron model was firstly introduced in 1982 [7], and it is described as 3 2 2 , , x y ax bx I y c dx y where x, y, and I represent the membrane potential, spiking variable, and steady current, respectively.
To exhibit the rich electrical activities of neuronal membrane potential, the 3D HR neuron model was then introduced [8] and it is expressed as , x y ax bx I z y c dx y z r s x x z where z stands for the bursting variable and x 1 represents the resting potential. The extra variable z is coupled into the 2D HR neuron model to regulate the current I. Generally, the model parameters in (1) and (2) are selected as a = 1, b = 3, c = 1, d = 5, x 1 = 1.6, s = 4, and r = 0.01 along with an adjustable current I [9]. Note that the parameter r is a small value and it is amplified to facilitate the selection of circuit parameters.
For the sake of simplicity, denote 3 2 2 ( ) 3 , After substituting the aforementioned parameters and the above equations into (1) and (2), the original 2D HR neuron model in (1) can be rewritten as ( ) , ( ) , and the original 3D HR neuron model in (2) can be rewritten as ( ) , ( ) , 0.04 0.064 0.01 , Then, the 2D/3D HR neuron model involves two polynomial nonlinearities: the cubic polynomial F(x) and the quadratic polynomial G(x).

Novel smooth nonlinear fitting scheme
To physically implement the 2D/3D HR neuron model in analog, the multiplier is a basic component for constructing the quadratic and cubic polynomials. However, the analog circuit with multiplier has higher implementation cost than that without multiplier. To reduce the implementation cost of the 2D/3D HR neuron model, we present a smooth nonlinear fitting scheme without any multiplier to implement the polynomial nonlinearities of the 2D/3D HR neuron model.
The hyperbolic tangent function tanh(·) in mathematics is smooth differentiable and has the upper and lower bounds. It can be easily implemented using some off-the-shelf analog electronic components. In the presented smooth nonlinear fitting scheme, two composite hyperbolic tangent functions are utilized to fit the two polynomial nonlinearities of the 2D/3D HR neuron model.
Corresponding to the cubic and quadratic polynomials F(x) and G(x) shown in (3), two smooth nonlinear fitting functions H 1 (x) and H 2 (x) are realized by the composite hyperbolic tangent functions and described as 1 1 where m j , κ j , and δ j with j = 1, 2, 3, 4, 5, as well as δ o1 and δ o2 are the parameters of the fitting functions H 1 (x) and H 2 (x), as listed in Table 1. Thus, H 1 (x) consists of three hyperbolic tangent functions while H 2 (x) contains two.   It should be stressed that the nonlinear fitting functions H 1 (x) and H 2 (x) using the presented nonlinear fitting scheme are smooth and continuously differentiable, whereas the piecewise linear approximation functions appeared in [27][28][29][30][31][32][33] are non-smooth and continuously non-differentiable.

Spiking and bursting in the fitted HR neuron model
The classical HR neuron model can show two operation patterns, namely the spiking and bursting. In the spiking pattern, the slow variable z is set as zero and the corresponding system is reduced to a 2D HR neuron model. In this way, the oscillating frequency is closely related to the steady current I. By contrast, in the bursting pattern, the slow variable z is what generates the bursting patterns. Correspondingly, the system is described by a 3D HR neuron model and its operation patterns have the mode transitions from resting to periodic bursting, to chaotic bursting, and finally to spiking patterns as the steady current I increases successively.
To demonstrate the feasibility of the presented smooth nonlinear fitting scheme, we introduce the two fitting functions H 1 (x) and H 2 (x) into the 2D/3D HR neuron model to replace the original nonlinear functions F(x) and G(x). Thus, the fitted 2D/3D HR neuron model is coined. To exhibit the nonlinear fitting effects, several representative values of the steady current I are selected for the original and fitted 2D/3D HR neuron models.
When the steady current is set as I = 0.5 and 2, respectively, the original and fitted 2D HR neuron models operate in the spiking patterns with different oscillating frequencies, and their membrane potentials are shown in Figs. 2a, b. As can be observed, the oscillating frequency increases as the steady current increases. Meanwhile, the two spiking patterns generated by the original and fitted 2D HR neuron models are basically consistent.  For different values of the steady current I, the 3D HR neuron model shows different operation patterns. The four representative values of the steady current are considered as I = 0.1, 2, 3.3, and 5, respectively. When imposing these steady currents on the original and fitted 3D HR neuron models, the membrane potentials can be induced and illustrated in Fig. 3. As can be seen, with the increasement of the steady current, the operation patterns undergo the resting, periodic bursting, chaotic bursting, and periodic spiking patterns sequentially. In the bursting patterns, the busters of the original and fitted 3D HR neuron models have slight difference but their dynamical behaviors are the same. As a result, the fitted 3D HR neuron model can be used to simulate the rich electrical activities of the original 3D HR neuron model, indicating the feasibility of the presented smooth nonlinear fitting scheme.
Since the original and fitted 2D HR neuron models only have the simple periodic spiking dynamics, we take the original and

Analog multiplierless circuit design and hardware experiment
This section designs an analog multiplierless circuit of the fitted 2D/3D HR neuron model using the circuit modules of composite hyperbolic tangent functions. Thereafter, a hardware device is developed for the analog multiplierless circuit.

Analog Multiplierless circuit design
The circuit modules for positive and negative hyperbolic tangent functions were designed in [24,35], as shown in Figs. 5a, b. They are designed using some off-the-shelf analog components, including operational amplifiers, voltage sources, transistors, and resistors. Note that the current source I 0 is implemented using two transistors, three resistors, and one voltage source [35], and its value is achieved by adjusting a resistance.
respectively, where E is a fixed offset voltage, μE = δ j , κ = R F /(2RV T ), and μ = R F /(2R E V T ) = κR/R E . In our design, the offset voltage is connected to the inverting input terminal of the operational amplifier via an adjustable resistor. The intent is to easily achieve different offset values of the hyperbolic tangent function by adjusting the resistance R E . However, when realizing the offset values of the hyperbolic tangent function, the offset voltage connection method reported in the literature [24,35] should adjust the offset voltage. This requires the power supply to provide multiple and high precision DC voltage outputs, and thus greatly increases the difficulty of hardware circuit experiment. In short, the circuit modules of positive and negative hyperbolic tangent functions in Figs. 5a, b are designed optimally in this paper.  With these circuit modules of hyperbolic tangent functions shown in Figs. 5a, b, two nonlinear fitting functions H 1 (x) and H 2 (x) given in (6) and (7) can be directly designed and their circuit modules are shown in Figs. 5c, d, respectively. When applying an input v i , the outputs of the two circuit modules in Figs. 5c, d can be represented as where m j = R/R mj with j = 1, 2, 3, 4, 5, V 1 = δ o1 = RE/R o1 , and V 2 = δ o2 = RE/R o2 .
The circuit parameters R = 10 kΩ, R C = 1 kΩ, E = 15 V, I 0 = 1.3 mA, and V T = 26 mV are determined. Using the parameter settings of the nonlinear fitting functions H 1 (x) and H 2 (x) shown in Table 1, the desired resistances of all circuit modules in Fig. 5 can be calculated and listed in Table 2. Thus, the cubic and quadratic polynomials F(x) and G(x) given in (3) can be approximately achieved using relatively cheap commercial electronic components.
Using the two circuit modules shown in Figs. 5c, d, the analog multiplierless implementation circuit of the fitted 2D/3D HR neuron model can be designed and drawn in Fig. 6. The upper part is two integral channels with the circuit modules of H 1 (v x ) and H 2 (v x ), and the lower part is an extra integral channel for implementing the slow variable v z . Based on three capacitor voltages v x , v y , and v z in Fig. 6, the circuit state equations under the physical time t = RCτ are established as where R I is a resistance to adjust the steady current.
2D HR neuron model 3D HR neuron model Compared (11) with (5), the resistance R I for adjusting the steady current I can be expressed as Thus, different spiking and bursting patterns can be displayed from the designed analog multiplierless circuit by adjusting the resistance R I achieved by a precise potentiometer.

PCB-based hardware experiments
Based on the circuit schematics shown in Figs. 5 and 6, a hardware device for the analog multiplierless circuit of the fitted 2D/3D HR neuron model is welded on a PCB level. Fig. 7 shows a picture of the hardware device. Here, the resistors, capacitors, precise potentiometers, MPS2222 transistors, and TL082CP operational amplifiers are employed. The off-the-peg ±15 V DC power module is used to provide the supply voltages of the operational amplifiers and offset voltages of the circuit modules. The experimental outputs are captured by the WaveSurfer 510 oscilloscope. Due to the limitation of saturation output level, it is necessary to ensure that the operational amplifier supplied by ±15 V DC power module operates within a linear operating range about ±13 V. In addition, I 0 can be adjusted by a specific potentiometer and one can refer to [35] for more details. Fig. 8 shows the screenshots of the hardware experimental prototype with the captured transfer characteristic curve for the circuit module H 1 (v i ) and periodic bursting pattern for the fitted 3D HR neuron model.  with the numerical simulations given in Fig. 1, manifesting the feasibility of the presented smooth nonlinear fitting scheme in the analog circuit implementation. Following the relation in (12), the resistance R I is employed for adjusting the values of the steady current I. First, disconnect the jumper and select the implementation circuit of the fitted 2D HR neuron model. Corresponding to Fig. 2, the spiking patterns for two representative values of R I are measured and shown in Fig. 10. Next, connect the jumper and select the analog implementation circuit of the fitted 3D HR neuron model. Corresponding to Fig. 3, the resting, periodic/chaotic bursting, and periodic spiking patterns for the four representative values of I are measured, as shown in Fig. 11. Consequently, various electrical activities can also be acquired from the analog multiplierless circuit of the fitted 2D/3D HR neuron model as well. The hardware experiments show that the presented smooth nonlinear fitting scheme can implement the 2D/3D HR neuron model without any multiplier.  In the hardware experiments, only the resistances in the circuit modules H 1 (v i ) and H 2 (v i ) need to be fine-tuned. Corresponding to the desired resistances listed in Table 2, the measured resistances are determined by Precision LCR Meter and listed in Table 3. By comparing the values in Tables 2 and 3, one can see that the measured and desired resistances have some slight deviations. These slight deviations are caused by the model idealizations, parasitic parameters, measurement errors, and so on.
The Mean Absolute Percentage Error (MAPE) introduced in [36] can be used to evaluate the deviations between the measured and desired resistances. Denote R desiredk as the desired value in Table 2 and R measuredk as the measured value in  (13) where N represents the number of resistance samples. According to the values listed in Tables 2 and 3, we can obtain that N = 17 and MAPE = 2.4% and this evaluation result is an acceptable precision for the PCB-based hardware experiments.

Conclusions
In this paper, we presented a smooth nonlinear fitting scheme for the 2D/3D HR neuron model. When implementing an analog circuit, the presented smooth nonlinear fitting scheme can avoid multipliers by employing two nonlinear fitting functions.
Therefore, without any multiplier, the analog circuit has low implementation cost, and thus is suitable for the hardware implementation of large-scale neural networks. Numerical simulations of the fitted 2D/3D HR neuron model and hardware experiments of the analog multiplierless circuit can reproduce various neuronal electrical activities with an acceptable error.
Certainly, the presented smooth nonlinear fitting scheme can also be applied in other neuron models or neural networks, such as the Morris-Lecar neuron model [25] and coupled Hindmarsh-Rose neuron model [29], which deserve our future study.