Deriving optimal silicon neuron circuit specifications using Data Assimilation

Mixed signal neuromorphic circuits represent a promising technology for implementing compact and ultra-low power prosthetic devices that can be directly interfaced to living tissue. However, to accurately emulate the dynamical behavior of the biological tissue, it is necessary to determine the optimal set of specifications and bias parameters for these circuits. In this paper we show how this can be done for a silicon neuron design, by applying a statistical Data Assimilation method (DA). We present a conductance-based silicon neuron based on the Mahowald-Douglas (MD) design and use the DA method to estimate its state variables and the ion channels parameters, so that it can accurately emulate the properties of biological neurons involved in the Central Pattern Generators (CPGs) responsible for producing the respiratory and heart-rate rhythms. While previous work has shown how DA well-estimates and predicts parameters from membrane voltage measurements using a semi-empirical Hodgkin-Huxley neural model, here we show how the same method is suitable for simplified Very Large Scale Integration (VLSI) circuit designs and demonstrate how it allows us to reliably predict the response of the MD neuron to different input current profiles.


I. INTRODUCTION
Mutually inhibitory networks generate the rhythms of life.These rhythms are supported by Central Pattern Generator (CPG) such as those located in the medulla which set the pace of respiration and modulate heart rate [1].The sequential pattern of neuron discharge depends in a complex manner on physiological feedback.This feedback allows heart rate and respiration to vary over time depending on arterial gas concentration, blood pressure and lung inflation.Although the architecture of medullary CPG and generic conductance models have been established experimentally [1] the internal parameters of neurons and synapses remain unknown.Knowledge of these parameters is essential to condition VLSI models in such a way that they behave identically to biological networks.This task is complicated by the fact that the parameters which control the kinetics of ionic gates and the conductance of ion channels are inaccessible to experiment.These parameters are also highly nonlinear which means it is extremely difficult to fit model parameters by trial and error.
In this paper we use nonlinear optimization to construct quantitative models of VLSI neurons.The method relies on Taken's embedding theorem [2] which states that under certain conditions the time series observations of the membrane voltage induced by a known current protocol will contain all the information about the model.This approach has successfully constructed predictive neurons models [3], [4].Here we use interior point optimization [5] to construct VLSI models of the Mahowald-Douglas neuron [6], [7].
We first generated time series voltage sequences by integrating current protocols with the NaKL Hodgkin-Huxley model [8].The membrane voltages were then assimilated over appropriate time windows to obtain the model parameters.A set of parameters was obtained by assimilating the MD models.We then constructed the completed models by inserting parameters in the model equations.The completed MD model was then integrated forward to predict the membrane voltage under arbitrary stimulation.
We also applied the data assimilation method to simulated data.In these experiments we used the MD model to obtain the membrane voltage and then to estimate the original model's parameters back from it.In Bayesian inference, this is known as "Posterior Predictive Check (PPC)" [9].Toth et al [10] have referred to this as "the twin experiment".This is a well-defined inverse problem.A more challenging problem arises when assimilating data from a real voltage trace or from a voltage trace synthesized by another model.The lack of knowledge on the accurate model's equations and parameters makes the problem ill-defined.The presented research study in this paper is focused on an ill-defined problem.We demonstrated the results from the ill-defined problem.Results from our "twin experiments" are not shown in this paper.
In Section II we explain why a hardware implementation of a CPG can be extremely useful for biomedical application, overcoming the limitation of the software ones.In Section III we describe the equation of MD model, used for the DA.In Section IV the method used to extract and predict parameters of MD models described.In Section V hardware requirements derived from DA are discussed and finally, in Section VI we summarize our results with concluding remarks.

II. HARDWARE IMPLEMENTATION OF CPG
Neuromorphic circuits are a class of hybrid analog/digital circuits that implement hardware models of biological systems [11].It has been argued that these types of circuits can be used to develop a new generation of computing technologies based on the organizing principles of the biological nervous system, which are optimally suited for building ultra-low power devices that have can interact with the environment in real-time [12].The styles of computation used in neuromorphic circuits are fundamentally different from those used by conventional computers: as the biological systems they model, neuromorphic circuits process information using energy-efficient asynchronous, event-driven, methods and, most importantly, they are adaptive and fault-tolerant.
With power budgets of hundreds of micro Watts, these systems are not meant to implement the types of complex deep networks or machine learning algorithms typically executed on Field Programmable Gate Arrays (FPGAs), Graphical Processing Units (GPUs), or Central Processing Units (CPUs) (that can dissipate from 10 3 to 10 7 times more power).On the other hand, they represent a very promising solution for implementing compact embedded systems that can process bio-signals and adopt on-chip and on-line learning strategies to adapt to the changes in the statistics of both input signals and internal state variables.Within this context, we explore the design of a silicon neuron model with circuit specifications and parameters optimally suited for a hardware implementation of CPG that can emulate the motor pattern activity of respiratory CPGs, while reacting in real time to incoming input signals.Previous works, that use hardware CPG to model the gait locomotion in robotics applications have already showed how they can interact with the environment in real-time [13], [14], [15].

III. THE SILICON NEURON
Figure 1 shows the circuit schematics of the spiking silicon neuron we propose, derived from the MD design proposed in [6], [7].
The building blocks of the MD neuron are the trascoductance amplifier (TCA), follower integrator (FI) and current mirror (CM).All current and voltage bias variables that end with an exclamation mark in the figure and represent global parameters shared among all neurons on the chip which can be precisely tuned by an on-chip 12bit resolution temperature compensated bias generator [16].The basic spiking mechanism comprises three main blocks: the leakage circuit which models the neuron's leak conductance; the sodium circuit which models the effect of the activation and inactivation channels for producing the spike; and the potassium block which models the resetting of the neuron.An additional AHP block provides a second negative feedback mechanism which introduces additional variables that increase the variety of the dynamical behavior [17].The AHP block is in fact responsible for the spike-frequency adaptation and bursting behaviors of the neuron.This mechanism is described by: where I m is the current across the membrane capacitance, I leak is the leakage current, I N a and I Kd the sodium and potassium current respectively, I inj the stimulation current and I AHP is the after-hyperpolarization current.The input current, injected into the capacitor C m , increases the membrane voltage V m .The voltage V m depends also on the leakage current Fig. 1: The silicon neuron schematic.The basic spike mechanism comprises the sodium channel (pink block), potassium channel (blue block) and the leakage conductance (torquise block).The yellow block represents the calcium and AHP conductance.
I leak and the current-voltage relationship is given by the one of TCAs: where I gleak is the maximum bias current of the TCA.The sodium current is composed by an activation I m and inactivation I h current.The activation current is associated to the positive feedback and it turns on when the membrane potential reaches the voltage threshold T hres, producing the spike.The inactivation current is turned on by the slower negative feedback.The inactivation current is slower than the activation because it is generated only after the voltage at C n , V f na rises above the threshold T hres.After the I m turns on, the V m steeply rises and the voltage V f na rises above T hres.
The sodium current can be expressed by the equation: The potassium conductance is only an inactivation current and it constitutes the second negative feedback.This current depends on the voltage at C k , V f kd and it is on as long as V f kd is above the T hres.V f kd increases and decreases linearly with V m and the current equation can be described by: The calcium block simulates the intracellular calcium concentration.Every spike is converted into a digital pulse of fixed width trough a spike discriminator and it is used as input to the adaptation mechanism.For each of them, a calcium current I Cain is injected into the soma, increasing V c .When V c reaches the threshold, the AHP circuit is turned on and it inhibits V m leading to spike frequency adaptation or bursting behavior.V c decays to a resting concentration through I Cabuf .The MD neuron is able to produce three different biologically plausible neuron's behaviors, shown in Fig. 2. The top plot shows the membrane potential produced by the basic spiking mechanism.The middle plot shows the spiking behavior with the spike frequency adaptation (AHP current) mechanism enabled, and the with the corresponding Calcium level voltage.The bottom plot shows bursting activity, with the Calcium level trace superimposed.The bursting behavior was obtained by properly tuning the parameter controlling the Potassium conductance levels the AHP amplitudes.IV.DATA ASSIMILATION Data assimilation is a methodology for estimating hidden variables.The estimation of these parameters involves the combination of observational data with the underlying dynamical principles governing the system under observation.Data assimilation estimates a best possible representation of the actual state of the system in order to make a prediction of a future state of the actual system.Due to their non-linear nature, inferring the internal parameters of neuron and neural networks require sophisticates algorithms [18].Takens' Embedding Theorem [2] explains how, under certain condition, a non-linear system can be reconstructed using measurements of a single variable.In this article we are identifying multichannel conductance models for assimilating the times series data of CPG respiratory neurons.Our approach started from the semi-empirical Hodgkin-Huxley model [8], a typical neuron model for a total of up to 71 parameters [3], [4].The HH neuron model was simulated in Matlab and the output voltage trace was re-scaled to the range 0 -10 V.This voltage trace was used to extract the gating parameters of sodium (Na + ) and potassium (K + ) ionic channels based on interior point optimization.Then, we generated time series voltage sequences by integrating current protocols with the re-scaled Hodgkin-Huxley model.This voltage trace was used for data assimilation to extract the parameters of the MD model.The results in Table I show that plausible values were extracted for the voltage threshold, Na + parameters and the membranes capacitance.As mentioned earlier, we employed different models for data generation and assimilation.Therefore, our problem is ill-defined and this explains why some parameter values hit the boundaries of the search interval.
In Fig. 3, we show the results of model validation, obtained by using the equations described in Section III to carry out behavioral simulations.The prediction accuracy of the assimilated model was assessed by forward integration beyond the assimilation window.Initially, the full MD model was constructed by plugging in the extracted parameters to the equations.The initial states of the gating variables were obtained from the end of the assimilation window.Then, the model was used to produce the prediction voltage trace (red line) by forward integrating the input current of another window.The width of the data assimilation window was 549.68 ms.The full quantitative model fits well the reference data (black line).

V. CIRCUIT DESIGN SPECIFICATIONS
Developing VLSI neuromorphic circuits for biomedical applications raises significant technological challenges, especially when the device is required to implement biologically exact and ultra-low power models that should interact in real-time with the environment.The circuits and methods proposed here address the challenge of finding the proper circuit specifications and network parameters that enable an electronic system to emulate the biological respiratory CPG.To obtain these specifications and network level parameters, it is first necessary to derive the optimal parameters of individual neurons.The parameters we derived are listed in Table I.These represent the starting point, for deriving the optimal silicon neuron specifications at the network level.Re-scaled DA results suggest that C m , C n , C k should be of the order of pico Farads, because of the requirement of implementing the slow respiratory neurons dynamic (0.5-4 Hz).Implementing such values with modern electronic hardware is not trivial, as it requires a substantial amount of area.A possible alternative solution is to decrease the capacitances dimensions and its input current in order to increase the neuron integration time.In addition, from Table I is possible to notice that C n and C k are bigger than C m , by a factor of three and four respectively.To decrease layout area we can design all the capacitances with the same dimensions of few pico Farads (1-2) and, by adjusting the relation between injected and leakage currents we can obtain the same channels dynamic.The two involved parameters I natau and I kdtau and it is important their working range, which should not be below 1 pA otherwise, it is in same noise range and the system behavior is uncontrollable.Another important requirement is the dynamic range of the current.In order to emulate biological neural computing systems efficiently, such as the respiratory CPG neuron, the transistor should be biased in the sub-threshold regime.This is obtained working in a range of current from pico Amperes to tens of nano Amperes with tunable values to set the desired behavior.Due to such small currents, during the VLSI design, it is important to measure the transistor leakage current.This value should be below 0.1 pF.To minimize the leakage, it is possible to act on the transistor's length.

VI. CONCLUSIONS
In this paper we showed how it is possible to estimate the dynamic state of a conductance-based silicon neuron circuit using a large scale variational method.Our study has used interior point optimization to construct two conductance-based neuron models: a re-scaled version of the HH neuron model and the silicon MD neuron model.Synthetic biological voltage traces were obtained from the re-scaled HH neuron model.The voltage traces were then used to extract the parameters of the MD model.The differences between the original data and the resconstructed data arise from differences between the model used to generate the data and the model used to perform assimilation.In the VLSI model, the activation and inactivation variables of Na + are subtracted rather than multiplied.Second, the time constants are fixed rather than being voltage-dependent.Third, the model does not incorporate the state variable for Na + activation.It is substituted by a constant parameter.Despite these differences the results show that our attempt to fit the biological synthetic data was successful.DA results provide an important starting point to estimate the silicon neuron requirements and to move towards implementing more sophisticated VLSI models to correct discrepancies between data and prediction in Figure 3.These results provide and important basis for building a quantitative model for a biologically plausible respiratory CPG network.

Fig. 2 :
Fig. 2: Different biologically plausible neuron's behaviors.Top figure represents the fast spiking behavior.Middle figure shows the regular spiking in which the calcium and AHP channels are active with the corresponding calcium level.Bottom figure represents the bursting activity with corresponding calcium level.

Fig. 3 :
Fig. 3: A voltage trace from HH neuron model was used to estimate the parameters of a silicon neuron model.The bottom figure shows the current stimulation (blue line) used to generate the voltage traces.The top figure shows the voltage trace from the HH neuron model (black line), the membrane voltage solution of the constrained optimization problem (green line) and the predicted membrane voltage (red line).

TABLE I :
Parameters of the silicon neuron model estimated from the assimilation window