Reliability-Aware Design of Spike-Event Neuromorphic Circuits

— Very Large Scale Integration (VLSI) based neuromorphic circuits also known as Silicon Neurons (SiNs) emulate the electrophysiological behavior of biological neurons. With the advancement in technology, neuromorphic systems also lead to various reliability issues and hence making their study important. Bias Temperature Instability (BTI) and Hot Carrier Injection (HCI) are the two major reliability issues present in VLSI circuits. In this work, we have investigated the combined effect of BTI and HCI on the two types of integrate-and-fire based SiNs namely (a) Axon-Hillock and (b) Simplified Leaky integrate-and-fire circuits using their key performance parameters. Novel reliability-aware AH and SLIF circuits are proposed to mitigate the reliability issues. Proposed reliability-aware designs show negligible deviation in performance parameters after aging. The time-zero process variability analysis is also carried out for proposed reliability-aware SiNs. The power consumption of existing and proposed reliability-aware neuron circuits is analyzed and compared.


I. INTRODUCTION
euromorphic computing primarily aims at emulating electro-physiological behaviours and computations of real biological neurons and neuronal systems using, mixedmode analog/digital VLSI system and was coined by Carver Mead [1]. The concept of neuromorphic computing adopts various computational models based on biological neural systems to develop massively parallel, hybrid processing electronic systems on silicon chips [1]. Progressive microelectronics has been engineered to be highly competent towards processing signals in low-power circuit designs; making them particularly relevant in realizing cognitive functions in the nervous system such as object recognition, data, and pattern classification, etc. [2], [3]. Neuromorphic processors like IBM's TrueNorth [4] , and Intel's Loihi [5] are examples of breakthroughs in the field of neuromorphic engineering that can carry out complex cognitive tasks in the realm of visual and sensory applications. In comparison to the compartmental processing of conventional supercomputers which require MWs of power, neuromorphic-based systems have proved to be successful in achieving the key ingredients of power efficiency (orders of less magnitude), accuracy and superior performance in mimicking the human brain [6]. Therefore, research interests in neuromorphic engineering has Jani Babu Shaik, Sonal Singhal is with the Department of Electrical Engineering, Shiv Nadar University, UP,201314, India (e-mail: * skjanibabu786@gmail.com and sonal.singhal@snu.edu.in). risen in recent decades towards becoming the current prime contender for progress in futuristic technology.
Extensive research in the field of neuromorphic engineering in recent years has led to the development of various bioinspired circuits on silicon chips using advanced CMOS technology nodes [6]- [13]. Therefore, it is notable to consider reliability issues such as Bias Temperature Instability (BTI) and Hot Carrier Injection (HCI) [14]- [17] at these advanced nodes. BTI leads to the shift of device parameters like the threshold voltage (ΔVTH), drain current (ID), subthreshold slope (ΔS), transconductance (gm) because of gradual accumulation of charges at or near gate insulator [18]- [24]. HCI affects similar device parameters as BTI with charge carriers in the channel attaining high energy due to a rise in the electric field [25]- [29]. In addition to reliability issues, process (time-zero) variabilities such as random dopant fluctuation (RDF), Metal Gate Granularity (MGG), and line edge roughness (LER) deviate device parameters from its mean value [30]- [39] and eventually affect the circuit performance [41,42].
Reliability analysis aids to estimate the impact of aging issues on system performance over its lifetime providing a quantitative assessment at the design stage empowering designers to set guard band or design margins before the manufacturing process [42].
Reliability analysis has been predominantly carried out on logic circuits such as flip-flops, SRAMs, logic gates, etc. due to its feature size scaling and thus set design guard bands [43]- [51]. Additionally, limited reliability research has been focused on analog/mixed VLSI circuits such as transconductance amplifiers, digital to analog (DAC) circuits, etc. [20], [52]- [61] [61]. SiN circuits falls in the category of analog/mixed VLSI circuits and their performance is susceptible to degradation due to both BTI and HCI degradation mechanisms. Degradation in neuron circuit may result in the neural network failure. This motivates us to explore neuromorphic circuits from a reliability point of view and propose reliability aware SiN designs that can provide a solution to sustain device degradation. To the best of our knowledge, this is the first study addressing reliability concerns of spiking-events in neuromorphic circuits along with a novel approach employed to enhance the designs from its degradation perspective.
This work is focused on two neuromorphic circuits based on the I&F model; the popular axon-hillock (AH) and simplified leaky integrate-and-fire (SLIF) circuits that has been used in different applications [8], [9], [62]- [68]. These circuits are implemented in the lower frequency ranges from 65Hz to 1KHz, enabling them to fit into the operational range of biological neurons. Combined impact of BTI and HCI along with the time-zero variability on these neuromorphic circuits are studied in detail. Sections 2, 3, 4 and 5 of the paper are organized as follows: In section 2, the simulation framework of this work is discussed. Section 3 describes the working of AH and SLIF neuron circuits with their detailed analysis from reliability point of view. In section 4, we present the proposed reliability-aware designs followed by simulation results of the two neuron circuits. Finally, we conclude in section 5. Fig. 1 shows simulation framework to evaluate the impact of BTI, HCI, and time-zero variability. All simulations are performed in CADENCE environment. Industry-standard 45nm High-K metal gate MOS technology library from foundry is used to create neuromorphic circuits considered for this study. The AgeMOS reliability model [55], [69] provided by the foundry based on BERT (Berkeley Reliability Tool) [70] model is used. Cadence integrates the AgeMOS model in its reliability simulation tool 'Cadence RelXpert'. The reliability tool considers the circuit netlist and voltage applied to the devices through the transient simulation. It then estimates degradation in device parameters after ten years of a lifetime based on the device operating conditions. Circuit netlists and reliability model (AgeMOS) files are fed to the RelXpert tool for generating aged netlists. Time-zero and aged netlists are provided to the SPICE simulator for simulations. Time-zero and aged simulations are referred as 'Time0' and 'Aged' respectively. Aged simulations are performed for combined effects of BTI and HCI; by considering lifetime spans of ten years. Frequency of spiking-events (fSPK), spike-ON duration (TON), spike-OFF duration (TOFF), average power (Pavg) of SiN are considered as the key parameters to analyze the performance of neuron circuits. Performance of aged neuron circuits is characterized by calculating the percentage change in Aged parameter with respect to Time0 value and is calculated by using equation (1):

II. SIMULATION FRAMEWORK
where Time0 and Aged are performance parameter values of pre-and post-degradations respectively. For time-zero process variability analysis, 1000 Monte-Carlo (MC) simulations are performed. MC simulations are carried out for both AH and SLIF circuits at a constant spiking frequency of 200Hz.

III. NEUROMORPHIC CIRCUITS
This section discusses the operation of two neuromorphic circuits, axon-hillock (AH) and simplified leaky integrate-andfire (SLIF). Impact of BTI and HCI on these neuromorphic circuits are also presented.

A. Axon-Hillock circuit
The spiking neuron circuit namely the axon-hillock [8], [71] is shown in Fig. 2(a). An external current source (ISYNP) models the synaptic current which charges the membrane capacitor (CMEM) and thus develops a membrane voltage (VMEM). Membrane voltage increases until it reaches the membranethreshold (VTM). VTM is governed by switching point of the first inverter (PM0-NM0) which raises the VSPK to VDD. It provides positive feedback to VMEM through PM1 and CFB, and causes a steep charging of CMEM. Simultaneously negative feedback path is activated and reset current (IRST) flows through NM2 resulting in slow discharge of VMEM.
The membrane potential (VMEM) and generated spike event (VSPK) are shown in Fig. 2  performance is characterized by parameters such as spike-ON duration (TON), spike-OFF duration (TOFF), minimum and maximum value of membrane voltage VMEM (MEMLOW, and MEMHIGH). Parameters are marked on VMEM waveform as shown in Fig. 2(b). Inset shows zoomed portion of VMEM and VSPK with TON marked. TON and TOFF parameters are crucial to analyze the aging behaviour of neuron circuits. Any change in these parameters can cause variation in spike frequency Variation in spike frequency due to BTI and HCI degradation is shown in Fig. 3. It is plotted as a percentage change in spike frequency (fSPK) with spike frequency fSPK, at the end-of-life performance (after 10 years of circuit operation) [29], [55], [72]- [74]. Percentage change in fSPK has a larger deviation towards lower frequencies with a maximum value of 13% as shown in Fig. 3. Degradation analysis shows that transistors PM0 and NM0 are mainly responsible for performance degradation in AH circuit. Transistor PM0 suffers from BTI as it remains in inversion mode during VMEM integration time. i.e., for TOFF time duration. Transistor NM0 degrades due to HCI as drain current flows during the TOFF time. Degradation in transistor PM0 and NM0 affect switching point of the first inverter. It consequently causes a shift in membrane-threshold (VTM) and results in deviation in fSPK values after degradation. Moreover, shift in VTM is similar for all fSPK values. However, time to charge CMEM and eventually to reach to shifted VTM (∆TOFF) decays exponentially with increase in synaptic current. Thus, percentage deviation in fSPK shows an increased degradation at low fSPK values and saturate for high fSPK values as shown in Fig. 3. Fig. 4(a) shows simplified leaky integrate-and-fire (SLIF) spiking neuron circuit [65]. In SLIF circuit, spiking parameters such as membrane-threshold (VTM), refractory period (TRFR) is governed by external voltage sources. Current source (ISYNP) models the synaptic current that charges CMEM, thus membrane potential (VMEM) develops. Source follower (NM2 & NM3) with voltage source (VTHR) sets the membrane-threshold (VTM). Output of source follower (VSF) is provided as input to the first inverter (PM2-NM4). Inverter output (VFB) goes low when VSF reaches its switching point, leading to a positive feedback current (IFB) through transistor PM1. Consequently, output voltage of the second inverter (PM3-NM5) goes high, allowing a reset current (IRST) to flow through NM1, discharging the VMEM to a resting potential. The refractory period (TRFR) is controlled by transistor NM6 (kept in the sub-threshold region) with voltage source (VRFR). The membrane potential (VMEM) and a generated spike event are shown in Fig. 4(b). Spike train performance is characterized by parameters such as spike-ON duration (TON) and spike-OFF duration (TOFF). Parameters are marked on VMEM waveform as shown in Fig. 4(b). Inset shows zoomed portion of VMEM and VSPK.

IV. PROPOSED RELIABILITY-AWARE NEUROMORPHIC CIRCUITS
This section discusses the proposed reliability-aware design of axon-hillock (AH) and simplified leaky integrate-and-fire (SLIF) neuron circuits.
To minimize the impact of aging induced degradation, techniques like VDD lowering can be used; which is commonly proposed in the literature [43], [44], [46], [75]- [77]. It is discussed in section III, that transistors PM0 and NM0 are mainly responsible for performance degradation in AH neuron circuit. Overdrive voltages of these two transistors can be reduced to mitigate the reliability issues in axon-hillock circuit. Table-I lists the cases which are considered in this work to reduce the impact of aging and process variation induced degradation in AH circuit. Case-1 is the reference case, discussed in section-2, now referred as Conv-AH. In case-2, a global VDD scaling approach is resorted to as a means to reduce VDD in all transistors. In case-3, a selective VDD scaling approach is used by employing two distinguished power levels namely VDD and VDD_LOW, as shown in Fig. 6(a). VDD_LOW is connected to transistors namely NM0 and PM0, while rest of the circuit operates on VDD. Case-4 is our proposed reliabilityaware AH (Rel-AH) circuit consists of diode connected transistors to reduce overdrive voltages of NM0 and PM0 transistors as shown in Fig. 6(b). Adding these additional transistors increases the area overhead of AH neuron circuit by ~ 15%. Table-I also lists the supply voltage values used in simulations. Fig. 6(c) shows VMEM for each case. It is observed from Fig. 6(c) that membrane potential waveform remains intact for all cases and only a shift in membrane-threshold (VTM) is observed with respect to Conv-AH. Fig. 7 shows the percentage change in VTH of degraded transistors for all considered cases of AH circuit. The shift in VTH of NM0 and PM0 for case-2, case-3 and case-4 are found to have lower values with respect to case-1 (Conv-AH). For case-2 and case-3, VDD lowering effects in reducing stress on the transistors which causes a lesser drain current in the first inverter. This results in significant drop of VTH shift with reference case. For case-4, the additional transistors reduce drain to source voltages of both NM0 and PM0 and eventually lower the drain current in that path. This leads to very negligible shift in VTH (~ 0.5%) of NM0 and is shown in Fig. 7.   fSPK values and gets saturated for higher fSPK values. This behavior of percentage change in fSPK with spike frequency is discussed earlier in section-II. All other cases show less than 3% deviation for full range of fSPK values. This is attributed to the reduced overdrive voltages of transistors NM0 and PM0. Monte-Carlo (MC) simulations are performed, at fSPK of 200Hz, to analyze time-zero process variability. Fig. 9 shows probit plots of fSPK for all cases. Probit plots shows that fSPK in each case follows the Gaussian distribution. Corresponding standard deviation (SD) is shown in figure. Case-4 shows the least standard deviation amongst the considered cases. Case-4 (Rel-AH) is therefore a superior circuit with equivalent mitigation of reliability degradation projecting least impact of process variability. Hence from here we will compare the performance of Rel-AH with Conv-AH neuron circuits. Fig. 10 shows the normalized average power dissipation of the AH neuron circuit for all cases at time-zero. The average power is normalized to the power dissipation value obtained at the lowest frequency point in the reference case (case-1). The power dissipation in the AH circuit is mainly due to the first inverter, which is ON for the entire spike-OFF duration (TOFF). Case-2 and Case-3 both employ the VDD scaling, thereby reducing power dissipation w.r.t case-1, also evident from Fig.  10. Proposed reliability-aware design (case-4) shows even lesser power dissipation. Case-4 employs the diode-connected transistors (PM2 and NM4), which reduces the gate-source voltage of transistors NM0 and PM0. The decreased gatesource voltages of NM0 and PM0 result in the lesser current drive of the first inverter, reducing overall power dissipation in case-4. Impact of BTI & HCI degradation on membrane voltage is characterized by measuring the difference in Time0 membrane voltage and Aged membrane voltage. It is expressed as the difference in MEMLOW or MEMHIGH for membrane high and low potential respectively at time-zero and aged circuits. Fig. 11(a) and (b) shows difference in MEMLOW and MEMHIGH for Conv-AH and Rel-AH circuits. Conv-AH circuit shows difference of ~60mV in MEMLOW and MEMHIGH due to degradation. It is attributed to the degradation in NM0 and PM0 transistors. Rel-AH circuit shows significant reduction in the difference in MEMLOW and MEMHIGH. The performance degradation in SLIF neuron circuit is discussed in section-III. It is largely due to the BTI and HCI degradation in transistor PM2. Table-II list cases considered to mitigate reliability issues in SLIF neuron circuit.
Case-1 is the conventional SLIF neuron and is referred as Conv-SLIF. In case-2, global VDD scaling approach is sought. In case-3, a diode-connected transistor PM5 is used to reduce the overdrive voltage of PM2 to mitigate the reliability issues. Adding transistor PM5 increases the area overhead of SLIF neuron circuit by ~ 11%. Table also lists supply voltage values used in simulations. SLIF circuit with diode connected transistor PM5 is shown in Fig. 13(a). Fig. 13(b) shows simulated waveforms of membrane voltage for each case. It is observed that membrane voltage shape remains intact while a shift in membrane-threshold (VTM) is observed with respect to Conv-SLIF.  It is observed that SD of case-1 and case-3 are in similar range and are higher than case-2. Case-2 shows less deviations with process variations but it is more sensitive to aging as the overdrive voltage of the second inverter has been compensated with VDD value. This results in the slow charging and discharging of VRST after degradation. It is noteworthy that case-3 shows an improvement against aging induced degradation. Case-3 is therefore considered as the proposed reliability-aware SLIF (Rel-SLIF). Hence, from now on we will compare only the performance of Rel-SLIF with Conv-SLIF neuron circuits. Fig. 17 shows the normalized average power dissipation of the SLIF neuron circuit for all three cases at time-zero. Power dissipation in the SLIF circuit largely depends on the transistors NM2 and NM3. Both NM2 and NM3 are ON in spike-OFF duration (TOFF). The power dissipation in case-2 is found lesser than case-1 and is attributed to VDD scaling. In case-3, the added transistor PM5 impacts power dissipation in two ways: (i) it reduces the gate-source voltage of transistor NM2 and thus reduces its current (ii) it also reduces membrane-threshold voltage (VTM), as shown in Fig. 13(b). Reduction in VTM decreases the time frame for which transistors NM2 and NM3 are ON. Both these effects are coupled together and hence effectively reduce overall power dissipation. Fig. 18(a) shows the percentage change in spike-ON duration (TON) with variation in fSPK for Conv-SLIF and Rel-SLIF neuron circuits. For Conv-SLIF neuron circuit, percentage change in TON vary from 1% to 8% with fSPK values. Whereas, proposed Rel-SILF neuron circuit shows a smaller range from 10% to 15%. Fig. 18(b) shows percentage change in parameter TOFF with fSPK for Conv-SLIF and Rel-SLIF neuron circuits.
Rel-SLIF neuron circuit shows marginal improvement in TOFF w.r.t. Conv-SLIF neuron circuit across the frequency range. Although the TON undergoes deviations, the more critical parameter TOFF (integration duration of CMEM and refractory period) tends to impact the frequency of SLIF neuromorphic circuits, which is least affected. These findings make the proposed reliable-aware circuit effective to mitigate the reliability issues in SLIF circuit. Fig. 19 shows the percentage change in Pavg with fSPK for reliability-aware neuron circuits. Percentage change in average power (Pavg) is calculated by considering power dissipation at time-zero and aged reliability-aware neuron circuits. The reliability-aware AH neuron circuit shows ~ 15% improvement, whereas the SLIF circuit does not show any change.

V. CONCLUSION
In this paper, a detailed analysis of reliability (BTI and HCI) impact and time-zero variability were carried out on the key performance parameters of two spike-event neuron circuits namely AH and SLIF. Modified reliability-aware AH and SLIF circuits are proposed. Conv-AH shows a larger deviation in spike frequency for lower range of fSPK values while Conv-SLIF circuit projects a marginal fSPK deviation with increasing fSPK. Reliability analysis aided in identifying the transistors responsible for performance degradation in Conv-AH and Conv-SLIF circuits. In order to mitigate reliability issues, global and local VDD scaling approaches were considered. The proposed reliability-aware design consisting of diodeconnected transistors was found to be most effective in lowering degradation of transistors in both AH and SLIF circuits. Consequently, percentage change in fSPK of the proposed reliability-aware AH and SLIF circuits show negligible deviation in all fSPK values across considered range. Results of the time-zero variability analysis indicates an improvement in standard deviation (SD) of Rel-AH over Conv-AH whereas SD of Rel-SLIF and Conv-SLIF were approximately same. Reliability-aware neuron circuits show significantly lesser power dissipation compared to conventional neuron circuits. This work demonstrates the significance of considering the aspect of reliability in neuromorphic circuit designs, enabling development of robust neuromorphic systems.