A Novel Electrical Bias Technique to Recover Degraded ReRAM Arrays for Deep Learning

Abstract

Resistive Random Access Memory (ReRAM) arrays based on $HfO_x$have shown promise for in-memory computing in deep neural network (DNN) training. A modified SGD algorithm [1, 2, 3], together with a co-optimized ReRAM material, can achieve respectable accuracy on a reduced Modified National Institute of Standards and Technology (MNIST) database classification task, approaching a floating point baseline [4]. However, as these arrays are used over time, they begin to degrade, which reduces their efficacy and impacts the performance of the systems that rely on them. As these arrays are leveraged for complex tasks like deep neural network training, the degradation issue becomes increasingly acute. In an effort to better understand the degradation phenomena, we turned towards employing a 3D atomistic simulator that enables the simulation of processes such as bond breakage, ion and vacancy diffusion, and trap-assisted electronic tunneling between vacancies. The filament formation and subsequent switching are also simulated, providing valuable insights into the degradation process and the potential for recovery. Building on this foundation, a unique electrical bias technique has been proposed to address the degradation issue and restore the precision of a deteriorated ReRAM array. We noticed that post-fatigue, the devices in the ReRAM array exhibited increased resistance and a diminished number of observable states. Our proposed electrical bias technique efficiently addresses both these challenges, reducing resistance and amplifying the number of states, thereby restoring the ReRAM array's accuracy to its initial pre-fatigue level. In practice, this carefully controlled recovery method has demonstrated impressive results. The accuracy after recovery using the biasing technique reaches 98% on a reduced MNIST classification task, closely approaching the results from a floating point baseline, a notable benchmark in the field. In summary, the proposed method offers a promising solution to a pervasive issue in neuromorphic computing. It underscores the potential for rejuvenating the performance of worn-out ReRAM crossbar arrays employed in in-memory deep neural network training. In addressing the immediate problem, it also significantly enhances the durability and reliability of these indispensable systems, promising more precise and reliable results over time. Moreover, this breakthrough sets a new direction for future developments in neuromorphic computing, paving the way for more resilient and enduring memory solutions that can withstand the rigorous demands of advanced computing tasks. Reference: [1] Gokmen, Tayfun, and Wilfried Haensch. "Algorithm for training neural networks on resistive device arrays." <i>Frontiers in neuroscience</i> 14 (2020): 103. [2] Gokmen, Tayfun. "Enabling training of neural networks on noisy hardware." <i>Frontiers in Artificial Intelligence</i>4 (2021): 699148. [3] Kim, Youngseok, et al. "Neural network learning using non-ideal resistive memory devices." Frontiers in Nanotechnology (2022). [4] Gong, Nanbo et al. “Deep learning acceleration in 14nm CMOS compatible ReRAM array: device, material and algorithm co-optimization.” 2022 International Electron Devices Meeting (IEDM) (2022): 33.7.1-33.7.4. Acknowledgment: This work is supported by the IBM Research AI Hardware Center. The authors thank IBM Research and TEL Technology Center members in Albany for device fabrication.