Addressing Resiliency of In-Memory Floating Point Computation

In-memory computing (IMC) can eliminate the data movement between processor and memory which is a barrier to the energy-efficiency and performance in Von-Neumann computing. Resistive RAM (RRAM) is one of the promising devices for IMC applications (e.g. integer and Floating Point (FP) operations and random logic implementation) due to low power consumption, fast operation, and small footprint in crossbar architecture. In this paper, we propose FAME, a pipelined FP arithmetic (adder/subtractor) using RRAM crossbar based IMC. A novel shift circuitry is proposed to lower the shift overhead during FP operations. Since 96% of the RRAMs used in our architecture are in High Resistance State (HRS), we propose two approaches namely Shift-At-The-Output (SATO) and Force To VDD (FTV) (ground (FTG)) to mitigate Stuck-at-1 (SA1) failures. In both techniques, the fault-free RRAMs are exploited to perform the computation by using an extra clock cycle. Although performance degrades by 50%, SATO can handle 50% of the faults whereas FTV can handle 99% of the faults in the RRAM-based compute array at low power and area overhead. Simulation results show that the proposed single precision FP adder consumes 335 pJ and 322 pJ for NAND-NAND and NOR-NOR based implementations, respectively. The area overheads of SATO and FTV are 28.5% and 9.5%, respectively.


I. INTRODUCTION
In the big data era, conventional CMOS-based Von-Neumann architecture platforms are unable to face real-time data processing requirements [1]. Memory and computing elements are decoupled from each other in Von-Neumann architecture [2] which apply frequent communication between memory and computing cores [3]. The compute energy has been scaled asymmetrically compared to data transport energy with transistor scaling. Data movement in modern computing systems dominates energy-efficiency and performance [4].
In Memory Computing (IMC) is one of the promising compute models to fully or partially eliminate the need to transport data between processors and memory. The main concept of IMC is to infuse compute capability into the memory cells [5]. IMC is achievable by using emerging Non-Volatile Memories (NVM) e.g., RRAM, Spin Transfer Torque (STT) RAM and Phase Change Memory (PCM) [5], [6], [7], [8]. Near memory processing [9] and logic-in-memory, which employ NVMs in the logic space [10], [11] to preserve states between powering sequence have been proposed in the literature. However, they cannot solve the problem of separation between logic and memory.
Due to immature fabrication technology limitations, manufacturing yield is still a serious concern for NVMs such as, RRAM crossbar. Faults in RRAM crossbar arrays are categorized into hard and soft faults [1]. Previous studies have been predominantly focused on soft faults [19] whereas few attempts are made to recover crossbar arrays from hard faults. The soft faults (e.g., read disturb) can be recovered by calibrating the resistance [19] [20]. However, hard faults are recovered through mapping algorithms (i.e., by assigning inputs of faulty RRAMs to the redundant rows or columns) [1], [21], [22].
Stuck-at fault is defined as a situation when the RRAM is permanently stuck at High Resistance State (HRS) or Low Resistance State (LRS). It has been reported [23] that only 63% of Hf O 2 − based RRAM devices for 4Mb crossbar array are fault-free and about 10% of RRAM devices contain stuckat faults. Retention failure which is similar to the resistive switching due to the generation or recovery of oxygen vacancy is another type of hard faults in RRAMs. In the proposed IMC architecture, only 4% of the RRAMs are in LRS and the other 96% are in HRS. Therefore in this paper, we focused on the HRS retention failure and stuck-at-1 (i.e., stuck-at HRS) faults. If the yield of a single RRAM device is 99%, there is only 10 −9 probability for a column of 64*32 array to be fault free. The stuck-at failures and HRS to LRS switching [24] can be fixed by employing few redundant rows/columns when RRAM array is considered working as a memory. However, the whole array is needed for IMC application. Consequently, computations will fail due to errors in the absence of fault tolerance schemes.
We have considered Floating Point (FP) operations to evaluate the proposed resilience techniques. This is motivated by the fact that emerging applications e.g., mission-critical systems like autonomous cars require huge amount of data processing in real-time at low-power (to make timely decisions). The autonomous cars make complex decisions in a tight deadline using algorithms e.g., Kalman filters for data fusion, ray tracing for path planning and, edge detection and deep neural networks for classification. Most of these algorithms require FP vector operations involving transpose, inverse, addition/multiplication. Therefore, the capability to perform these tasks, quickly and accurately can be of utmost importance to enable the safe and energy-efficient autonomous systems. Conventionally, FP architectures are implemented as full custom VLSI or in FPGA. Although fast and power efficient, these custom designs impose cost and complexity. In this paper, we propose FAME (Single Precision Floating Point Arithmetic using In-Memory Computing) implemented on crossbar RRAM. We employ a modified version (Section ??) of Dynamic Computing In Memory (DCIM) [7] based architecture as our baseline compute substrate for FAME. Additionally, two approaches namely, Shift-At-The-Output (SATO) and Force To V DD (GN D) (FTV(G)) are proposed to enable in-memory computing in presence of HRS to LRS retention failures. We focus on this failure mechanism due to two reasons: (i) HRS to LRS switching is more common in RRAM [25]; (ii) majority of the RRAMs (96%) are in HRS for both N AN D-N AN D and N OR-N OR arrays. Carry Select Adder (CSA) based on DCIM implementation is used for the demonstration. We add extra peripheral circuits on each array to implement the proposed techniques.
In particular, we make the following contributions in this paper: 1) Alternative low-overhead realization of DCIM for FP computation; 2) In-memory shift circuit embedded in the peripherals e.g., sense amplifier (SA); 3) Enabling pipeline architecture using the latch embedded in the SA; 4) Propose fault mitigation approaches such as, SATO and FTV/FTG for DCIM architecture; 5) Conduct PV analysis of the RRAM array to check the integrity of SATO and FTV/FTG. Rest of the paper is organized as follows. Section II introduces related work on IMC. Section ?? explains the proposed FAME circuit and architecture. Section III presents the simulation results of FAME and comparison with other IMC logic implementation. Section IV explains proposed approaches to overcome SA1 faults in IMC architecture. Section V presents the proposed fault tolerance approaches and simulation results. Section VI draws the conclusion.

A. Memristor Aided Logic (MAGIC)
MAGIC [26] (shown in Fig. 1) is an IMC architecture in which logic state of the gates are represented by the memristor (RRAM in this paper) resistance where high (low) resistance is considered as logic '1' ('0'). The inputs to a MAGIC gate are the logic states stored in the input memristors and the output is the final state of the output memristor. MAGIC executes operations in two steps: 1) setting the output memristor to a known logic state (e.g., for N OR operation the output

Out Gateway
In-2

In-1
Step # Opertaion Application of Voltage In DCIM, wordlines (WL) serve as the inputs and the bitlines (BL) serve as the outputs of the arrays. Separate preprogrammed AN D and OR arrays are dedicated to implement the desired function. For instance, in order to implement in 0 .in 1 , the bitcells connected to in 0 and in 1 are programmed to LRS while the bitcells connected to in 0 and in 1 are programmed to HRS ( Fig. 2a. All bitcells which are not part of AN D gate inputs are programmed to HRS (e.g., the bitcells connected to input in n and in n ). Fig. 2 shows the implementation of XOR function using DCIM. Initially, Pre signal is activated to pre-charge BLs of the AN D array. Next, inputs (in 0 and in 1 ) are applied by asserting EN AN D . As shown in Fig. 2b, both BL 0 and BL 1 drop below the reference voltage (V Ref −AN D ) when in 0 = in 1 = 1. As a result, SA output which determines the results of in 0 .in 1 and in 0 .in 1 functions are pulled down to '0' at the edge of SE AN D . Next, AN D array SA outputs are provided as inputs to the OR array. Since inputs of the OR array are '0', the BL (BL0 OR ) remains discharged which results in in 0 ⊕ in 1 = 0. If in 0 = 0, in 1 = 1 (in 0 .in 1 = 0 and in 0 .in 1 = 1), BL 0 discharges while BL 1 remains pre-charged. Therefore, BL0 OR starts charging at the edge of EN OR . Finally, the voltage of BL0 OR is compared against V Ref −OR at the edge of SE OR which produces '1' at the output of SA.

C. FP Addition/Subtraction
In IEEE 754 standard, a single precision FP number is represented by 1 Sign bit, 8 Exponent bits, and 23 Fraction bits. A negative (positive) number is represented with a sign bit  [27] equal to '1'('0'). In order to demonstrate negative exponents, IEEE 754 uses a bias of 127 for single precision (e.g., -1 is represented by -1+127=126). The general representation of a FP number is given by: The flowchart for FP addition/subtraction as per IEEE 754 standard is shown in Fig. 3.

III. FAME SIMULATION RESULTS
The simulations are carried out in 65nm PTM [28] technology by employing ASU RRAM model [29] and bi-directional selector diode model [30]. Worst-case Sense Margin (SM), BL-delay, average delay, average power, and energy consumption (Table III) are calculated to evaluate FAME architecture. Key parameters of devices for simulations are listed in Table   TABLE I Table II to mimic process variations. The worst-case SM is obtained under process variation @25 o C for worst case compute array (i.e., fraction addition array). The BL-delay is the time when 100 mV SM is achieved. The proposed FP adder/subtractor implementation with both N AN D-N AN D and N OR-N OR architecture are compared against MAGIC and ASIC design.
The write latency is obtained by performing 1000 points MC simulation. The worst-case write latency for low-to-high and high-to-low switching under process variation is 20ns. FAME achieves 828X, 3.2X and 3.7X improvement in latency, power and energy, respectively compared to MAGIC. The higher energy associated with MAGIC is attributed to the need to write into the RRAMs when an operation is done. Furthermore, compared to the power, energy consumption, and   (Table III). V N AN D0 (NAND array BL voltage when input is '0'), V N AN D1 , V N OR0 , and V N OR1 distributions at worst-case temperature are shown in Fig. 4. In order to achieve the read access pass yield (RAP Y ) [31] [7] we have performed SA offset voltage analysis. The SA offset voltage can be modeled by a Gaussian distribution with σ = 16mV and µ = 8mV . To obtain RAP Y we assume that V Ref is produced by a voltage regulator with negligible variation (5mV ). We assigned V Ref in such a way to maximize RAP Y . Based on the Monte-Carlo simulation, the RAPY of N AN D and N OR operations are found to be 4.6σ and 4.5σ respectively.

IV. RESILIENCE TO STUCK-AT FAULT
In this section, we describe SATO and FTV, two fault mitigation techniques proposed for DCIM architecture. In the following we use, (i) faulty BL to denote each BL with an  undesired stuck-at-1 (SA1) RRAM; (ii) faulty WL (BL) to denote each WL (BL) with an undesired SA1 RRAM.
Computations are performed in two cycles when the proposed fault mitigation techniques are applied (Fig. 5). The computations of fault-free BLs (BL 1 , BL 2 and BL 3 in Fig.  5 (a)) are performed in the first cycle and the computations of the faulty BLs (BL 0 in Fig. 5 (a)) are performed in the second cycle. In FTV, the WLs corresponding to faulty RRAMs (In 2 in Fig. 5 (a)) for N AN D (AN D) array are forced to V DD to mask faulty bits. In a dual Force-to-Ground (FTG) technique, the faulty BLs are forced to 0V for NOR (OR) arrays. FTV/FTG tolerates 99% of stuck-at faults (SAF) while reducing power consumption of the array. In SATO approach, operations of fault-free BLs are executed in the first cycle and then the outputs are shifted in the SAs. Then, the operations of faulty BLs are computed using fault-free BLs (operation of BL 0 is done in BL 1 ). SATO covers 50% of SAFs without affecting power consumption. The high level timings of FTV and SATO are illustrated in Fig. 5

(b) and (c), respectively.
A. Shifting-At-The-Output (SATO) As described before, in this technique the normal operation for fault-free BLs are performed in the first cycle and computation of faulty BLs are performed in the second cycle. SATO does not use faulty BLs for performing an operation and executes all the operations on the fault-free BLs. SATO shifts the data stored in SAs' latch of fault-free BLs to prevent overwriting. When computation of first cycle is completed, the data are shifted in SAs (three shifts are needed if an adder/subtractor is implemented). As shown in Fig. 6, inputs of the WLs should get shifted too, so computation is performed using fault-free BLs. Peripherals of SATO incurs 29.5% area overhead.
1) Non-fixable Faults: SATO cannot handle faults that appear on two consecutive sets of BLs (each three consecutive BLs are a set if an adder/subtractor is implemented). More multiplexers are needed for each WL to handle faults on consecutive sets of BLs. The number of multiplexers per WL increases linearly with the number of consecutive faulty sets of BLs to be handled by SATO. For example, if faults occur on two consecutive sets of BLs (e.g., if BL 3 in Fig. 6 also contains a fault) SATO cannot handle it unless two or more multiplexers are dedicated to each WL. The probability of two faults occurring on two consecutive BL is less than 3% for a 64*32 crossbar array. However, SATO is able to handle less than 50% of the faults if a yield of 99.5% is considered on a crossbar array.
2) Handling multiple faults: SATO's efficiency degrades for increasing number of faults. In this paper, we considered a yield of 99.5% in a 64*32 crossbar array for SATO simulations. This corresponds to 11 randomly distributed faults throughout the array. SATO is able to mitigate around 50% of the faults in the array. Faults have been distributed on the memory cells using rand function provided by C + + programming language. FTV uses a multiplexer for the enable signal of SAs to ensure that the array is capable of working in two cycles. F · CS and F · CS are inputs of the multiplexer, where F is '0' if the BL is fault-free and is '1' if the BL is faulty. CS is the clock sequence initialized to '0' in the first cycle and '1' in the second cycle. Enable signal of SAs connected to fault-free BLs are asserted in the first cycle while the enable signal of SAs connected to faulty BLs is asserted in the second cycle to save power and maintain the correct logic. Furthermore, FTV uses 4 additional transistors compared to DCIM at the WL input to enable the test procedure (explained in Section IV-C) to find faulty RRAMs.
As shown in Fig. 7a, FTV uses two transmission gates to connect input and input to WLs. Also, one PMOS transistor is added to each WL to force the WL to V DD when is needed in the second cycle. In the second cycle, SC signal of faulty WLs gets activated to force faulty WLs to V DD . FTV employs a fault signal (F W ) for each WL to track the faulty WLs and set them to V DD in the second cycle. FTV also defines a fault signal (F B ) for each BL to keep track of faulty BLs. Additional circuitry and peripherals needed to apply FTV to the DCIM increase the area by 9.5%.
1) Forcing-to-Ground (FTG): The basic concept of FTG is similar to FTV but it applies to N OR (OR) arrays. FTG follows simple logic that number of '0's is not important in N OR (OR) operation. Therefore, FTG forces inputs of faulty   RRAMs to the ground. Peripherals and the rest of the FTG's operation are the same as FTV.
2) Non-fixable Faults : Although FTV can fix most of the faults in a crossbar array, it is unable to handle some rare situations. For example, if there are two faulty BLs and the faulty RRAM on one of the BLs is the operand of the other BL. As shown in Fig. 8, BL 0 and BL m are faulty and their operation must be done in the second cycle. In this case, the logic of BL m gets lost if FTV forces input of the faulty RRAM (RRAM1) on BL 0 to V DD since one of its inputs is set to V DD . The N AN D operation for BL m is incorrectly performed between in n and '1' instead of between in n and in m . The probability of occurrence of such a fault for fabrication yields of more than 99% is less than 1%. We randomly distributed the faults for 100 times using C + + language rand function in order to achieve the percentage of faults occurring in an array.
3) Handling multiple faults: As long as faulty RRAMs in the crossbar array are independent of each other, FTV can handle as many as possible faults. For our simulations we inserted 30 faults in a 64*32 crossbar array and FTV was able to solve more than 99% of fault distribution over the array.

C. Finding Faults using FTV Peripherals
It is required to find the faulty RRAMs to set fault signals of the BLs and WLs. Faults can be found by the peripherals that are included in the FTV. However, the BLs must be tested one at a time. To find the faults in a N AN D array, input of each RRAM, which is set to LRS in a BL is forced to V DD and the rest are forced to '0'. The output of the SA indicates whether a BL is faulty ('1') or fault-free ('0'). If the BL is faulty, we need to find out which RRAM is faulty on that BL. A divide-and-conquer approach cannot be used in this architecture since there might be more than one faulty RRAM per BL, so we use brute force algorithm to find faulty RRAM. The input of the RRAM-under-test is set to '0' while inputs of all other RRAMs are set to V DD . If SA output is '0', the RRAM-under-test is deemed faulty and its flag is set to '1'. All faulty RRAMs can be found by repeating this operation for each RRAM in each BL sequentially.

D. Usage and Limitations of SATO/FTV/FTG
SATO/FTV/FTG should be enabled only when a fault has been detected in the test process. Therefore, the fault-free array will only incur area overhead but no performance loss. The faulty array will be salvaged at the cost of performance overhead. Note that SATO/FTV/FTG are only applicable to DCIM-based IMC. They cannot be applied to MAGIC or RRAM-based static IMC in the current form.

V. SIMULATION RESULTS
To evaluate SATO and FTV, we compute performance metrics that include worst-case SM, BL-delay, average delay, average power, and energy consumption of a 64*32 DCIM RRAM crossbar array (VI). Based on the simulation results, FTV is more efficient than SATO.

A. SATO Simulation Results
Applying SATO to DCIM increases power and energy consumption by 12% and 127%, respectively (the worst case) and also performance is reduced by more than 50%. However, SATO is able to handle ∼50% of the SA1 faults. SAs are very costly and occupy large area, which using SAs to shift data, increases power consumption and leads to higher energy consumption. Power and Energy consumption of SATO with different number of SA1 faults is reported in Table VII.

B. FTV/FTG Simulation Results
SM is the most important parameter when FTV is applied to DCIM. Increased number of LRS RRAMs connected to V DD (ground) on a BL worsens the SM when '0' ('1') is the output. Considering N AN D − 2, the worst case '0' occurs when one of the operands is '0' and the other operand is '1'. In this case, there is a voltage division is between one LRS RRAM connected to '0' and one LRS RRAM which is connected to V DD . When there is a faulty LRS RRAM on the BL and it is forced to V DD , the worst case '0' is when two LRS RRAMs are connected to V DD and one LRS RRAM is connected to '0' which lead to increased output '0' voltage on the BL. Increased BL voltage for the worst case '0' degrades the SM as shown in Fig. 11 (a).
The degradation in worst case SM happens when the bitcell is made of only a RRAM (i.e., no selector diode). However, DCIM employs a bidirectional diode in series with the RRAM. This series-connected bidirectional diode is included to reduce power consumption by dropping 0.5V across the 2 terminals. When the voltage difference between a BL and a WL is less then the selector diode threshold voltage there is no current between the BL and the WL. So, increased number of LRS inputs connected to V DD (ground) does not affect the SM of DCIM ( Fig. 11 (b)). The current of HRS RRAMs increases with temperature which results in higher sneak path currents. In an AN D (OR) array, the higher sneak path currents pull up (down) BL voltage to degrade the SM. However, when number of faults increases, sneak paths currents become negligible compared LRS RRAMs which are connected to V DD (ground). Simulation results (Fig. 11 (a)) show that the SMs in different temperatures become equal when the number of faults is more than 20.
Compared to the fault-free situation, FTV reduces power and energy consumption by >54% and >7% respectively (since, SAs consume a lot of power and in the case of FTV, SAs connected to faulty BLs are deactivate in the first cycle and SAs connected to fault-free BLs are deactivated in the second cycle). This is due to inactive BLs and SAs and a longer time of operation. However, the performance reduces by 50% due to two cycle operations. Average power and energy for the four consecutive AN D operations in the 64*64 array are reported in Table VIII.

C. Process Variation Simulations
The most important parameter to consider in a crossbar array under process variation is SM. We ran 1000-point MC simulations at −10°C, and 90°C on DCIM by considering the bitcell consisting of only a RRAM and RRAM and a selector diode with different number of SA1 faults. Simulation results for RRAM bitcell and RRAM and selector diode bitcell are shown in Fig. 9 and 10, respectively.
As shown in Fig. 9, variations do not affect SM significantly due to the presence of selector diode which stabilizes BL voltage. However, as demonstrated in Fig. 10, variations affect the SM when only RRAM is used in the crossbar. This is due to large changes in the RRAM resistance for a small change in RRAM gap when 1.2V is applied across it. Worst case SM with the number of failures for both w/ and w/o selector diode is reported in Table. IX.

VI. CONCLUSIONS
We proposed FAME for in-memory FP arithmetic computation. FAME implements single precision FP adder/subtractor using RRAM crossbar and evaluated two flavors with N AN D − N AN D and N OR − N OR compute arrays. We also proposed a novel SA based shift circuit for frequent shifting needed in FP operation. Compared to MAGIC-based implementation, FAME achieves 828X and 3.7X latency and energy improvement over MAGIC and compared to processing units (e.g. CPU, FPGA, GPU) it also reduces energy consumption and delay by 93% and 70%, respectively. FAME achieves lower power and energy consumption compared to MAGIC and processing units at low area overhead to the memory arrays. FAME uses 3KB memory to implement single precision FP operations (V). Furthermore, two approaches to mitigate HRS to LRS retention and stuck-at-1 failures in RRAM-based compute memories are proposed along with a test approach to identify faulty RRAMs. Forcing-to-V DD (FTV) can mitigate 99% of the faults while reducing the power consumption by >50% and energy consumption by >7%.
Shifting-at-the-Output (SATO) technique increases power consumption slightly but increases energy consumption by >50%.