深度學習為機器學習的分支，隨著科技的日新月異，深度學習已成為人工智慧中成長最快速的技術，其主要功能是讓機器模擬人類神經網路的運作方式，而深度神經網路又為深度學習最常使用的架構，已經被廣泛應用在許多領域中，舉凡像是圖像識別、語音識別與自動化設備等。
深度神經網路最主要的運算特性為向量內積與矩陣相乘運算，也因為這項特性使得運算所需的資料量非常龐大。傳統的范紐曼 (Von Neumann) 計算機架構在執行運算時，會花費許多的資料搬運時間以及大量的能量消耗，成為神經網路運算的一大瓶頸。隨著科技的進步，記憶體內運算 (Computing in Memory) 的提出可以解決這方面的問題，而電阻式記憶體 (RRAM) 除了應用在高儲存系統或是隨機存取記憶體以外，更因為本身具有向量內積與矩陣相乘的運算特性，可以做為記憶體內運算之硬體架構。
然而由於製程尚未成熟，在電阻式記憶體中仍會有許多的故障發生，導致矩陣計算時的誤差，進而影響神經網路之推論正確率 (Accuracy)。因此本篇論文特別針對固定型故障及變異故障，提出權重位址重映射技術以減輕故障對於正確率的影響，而本篇所提出之技術相較於傳統容錯技術 [9]，不須增加複雜的繞線器即可實現，因而減少硬體成本；另外也針對權重位元重要性進行分析，分析故障對每個位元的影響程度，以優先保護重要位元。
本研究以 Pytorch [11] 深度學習框架開發一個模擬器，分析不同深度學習模型之正確率以評估權重位址重映射技術之效果，此外也分析了在不同的錯誤比率下實際上使用權重位址重映射技術可遮蔽錯誤效應的數量以及修復率 (Repair Rate)，最後也針對硬體成本進行分析。

Deep Learning is one of the most important machine learning techniques. With the
rapid development of technology, deep learning has become the fastest growing
technology in artificial intelligence (AI). Its main function is to allow machines to
imitate the operations of human neural networks. Deep neural networks (DNN) are the
most widely used architectures in deep learning. It has been widely used in many fields, such as image recognition, speech recognition, and automation equipment.
The operation characteristic of DNN is the vector inner product and matrix
multiplication operations. Due to this characteristic, the amount of data required for these operations are usually very huge. The traditional Von Neumann architecture has the bottlenecks of power consumption and bandwidth issues for providing DNN
operands. With the advancement of emerging memory technologies, computing in
memory is considered a promising solution to deal with these issues. As mentioned in
other research results, resistive random access memory (RRAM) can not only be used
in high storage systems or can be used as random access memories (RAMs). It is also
suitable for implementation of vector inner product and matrix multiplications.
Therefore, it can also be used as a feasible hardware architecture for computing in
memory.
However, due to the immature manufacturing process, RRAM manufacturing
defects, such as stuck-at faults and resistance variations may lead to serious accuracy degradation for DNN applications. Therefore, in order to simultaneously mitigate the impacts of stuck-at faults and variation faults, we propose the weight address remapping technique to reduce the accuracy degradation. As compared with the
traditional fault-tolerant techniques [9], our proposed technique does not require
complicated router. Therefore, the hardware overhead can be greatly reduced. We also
analyze the significance of weight bits which reveal the influence of faults on each
individual bit. Thereafter, the most significant weight bits are protected.
In this thesis, we use the deep learning framework⎯pytorch [11] for evaluating
the accuracy of different DNN models. Also, a simulator is developed for evaluating
repair rate and hardware overhead. According to experimental results, we can improve
the accuracy of DNN models with negligible hardware overhead.

[1] N. Papandreou, H. Pozidis, A. Pantazi, A. Sebastian, M. Breitwischt, C. Lamt, and
E. Eleftheriou, “Programming algorithms for multilevel phase-change memory,”
in Proc. IEEE Int’l Symp. on Circuits and Systems, pp. 329–332, May 2011.
[2] H. S. P. Wong, H. Y. Lee, S. Yu, Y. S. Chen, Y. Wu, P. S. Chen, B. Lee, F. T.
Chen, and M. J. Tsai, “Metal-Oxide RRAM,” Proc. of the IEEE, vol. 100, no. 6,
pp. 1951–1970, June 2012.
[3] M. N. Baibich, et al., “Giant magnetoresistance of (001)Fe/(001)Cr magnetic
superlattices,” The American Physical Society, vol. 61, no. 21, pp. 2472–2475, Nov.
1988.
[4] F. Zahoor, T. Z. A. Zulkifli, and F. A. Khanday, ‘‘Resistive random access memory
(RRAM): An overview of materials, switching mechanism, performance,
multilevel cell (MLC) storage, modeling, and applications,’’ Nanosc. Res. Lett.,
vol. 15, no. 1, pp. 1–26, Dec. 2020.
[5] Z. R. Wang et al., “Efficient implementation of Boolean and full-adder functions
with 1T1R RRAMs for beyond Von Neumann in-memory computing,” IEEE
Trans. Electron Devices, vol. 65, no. 10, pp. 4659–4666, Sep. 2018.
[6] W. A. Wulf and S. A. McKee, “Hitting the memory wall: implications of the
obvious,” ACM SIGARCH Computer Architecture News, vol. 23, no.1, pp. 20-24,
Mar. 1995.
[7] R. Dekker, F. Beenker, and L. Thijssen, “Fault modeling and test algorithm
development for static random-access memories,” in Proc. IEEE Int’l Test Conf.,
pp. 343–352, Sep. 1988.
[8] B. Liu, H. Li, Y. Chen, X. Li, Q. Wu, and T. Huang, “Vortex: variation-aware
training for memristor x-bar,” in Proc. IEEE Design Automation Conference
(DAC), pp. 1–6, June 2015.
[9] F. Zhang and M. Hu, “Defects mitigation in resistive crossbars for analog vector
matrix multiplication”, in Proc.IEEE Asia and South Pacific Design Automation
Conference (ASP-DAC), pp. 187-192, 2020.
[10]Ma, Chang, et al. “Go unary: A novel synapse coding and mapping scheme for
reliable ReRAM-based neuromorphic computing,” in Proc. IEEE Design,
Automation & Test in Europe Conference & Exhibition (DATE), 2020.
[11]A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A.
Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in Pytorch,” in
Proc. Conf. Neural Info. Process. Syst. Workshop, pp. 1–4, Oct. 2017.
[12]J. Y. Hu, K. W. Hou, C. Y. Lo, Y. F. Chou, and C. W. Wu, “RRAM-based
neuromorphic hardware reliability improvement by self-healing and error
correction,” in Proc. IEEE International Test Conference in Asia (ITC-Asia), Aug.
2018.
[13]X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks,” in
Proc. Conf. Artificial Intelligence and Statistics, vol. 15, pp. 315-323, Apr. 2011.
[14]T. Tang, et al., "Binary convolutional neural network on RRAM", in Proc. IEEE
Asia and South Pacific Design Automation Conference (ASP-DAC), pp. 782-787,
2017.
[15]F. F. Li, R. Krishna, D. Xu, A. Byun, W. Shen, J. Braatz, D. Cai, J. Gwak, De-An
Huang, A. Kondrich, F. Yu Lin, D. Mrowca, B. Pan, N. Rai, L. P. Tchapmi, C.
Waites, R. Wang, Yi Wen, K. Yang, B. Yi, C. Yuan, K. Zakka, and Y. Zhang,
“CS231n Convolutional neural networks for visual recognition,” Jan. 2015.
[Online]. Available: https://cs231n.github.io/convolutional-networks/
[16]A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep
convolutional neural networks,” in NIPS, 2012.
[17]T. W. Hickmott, “Low-frequency negative resistance in thin anodic oxide films,”
Journal of Applied Physics, vol. 33, no. 9, pp. 2669-2682, Sep.1962.
[18]G. Dearnale, A. M. Stoneham, and D. V. Morgan, “Electrical phenomena in
amorphous oxide films,” Rep. Progr. Phys., vol. 33, pp. 1129–1191, 1970.
[19]J. F. Gibbons and W. E. Beadle, “Switching properties of thin NIO films,” SolidState Electron, vol. 7, no. 11, pp. 785–790, 1964.
[20]C. Rohde, B. J. Choi, D. S. Jeong, S. Choi, J. S. Zhao, and C. S. Hwang,
“Identification of a determining parameter for resistive switching of TiO2 thin
films,” Appl. Phys. Lett., vol. 86, pp. 262907-1–262907-3, June 2005.
[21]S. Seo, M. J. Lee, D. H. Seo, E. J. Jeoung, and D. S. Suh, et al., “Reproducible
resistance switching in polycrystalline NiO films,” Appl. Phys. Lett., vol. 85, no.
23, pp. 5655–5657, Dec. 2004.
[22]L.O. Chua, “Memristor – the missing circuit element,” IEEE Trans. on Circuit
Theory, vol. 18, no. 5, pp. 507–519, Sep. 1971.
[23]C. Y. Chen et al., “RRAM defect modeling and failure analysis based on march
test and a novel squeeze-search scheme,” IEEE Trans. Comput., vol. 64, no. 1, pp.
180–190, Jan. 2015.
[24]Tseng, Yi-Ting, et al. "Complementary resistive switching behavior induced by
varying forming current compliance in resistance random access memory," Applied
Physics Letters, 106.21 (2015): 213505.
[25]S. Yu and P. Y. Chen, “Emerging memory technologies: Recent trends and
prospects,” IEEE Solid-State Circuits Magazine, vol. 8, no. 2, pp. 43–56, June 2016.
[26]J. C. Liu, C. W. Hsu, I. T. Wang, and T. H. Hou, “Categorization of multilevel-cell
storage-class memory: An RRAM example,” IEEE Trans. Electron Devices, vol.
62, no. 8, pp. 2510–2516, Aug. 2015.
[27]Yu, Muxi, et al, "Novel vertical 3D structure of TaOx-based RRAM with selflocalized switching region by sidewall electrode oxidation," Scientific reports, 6.1
(2016): 1-10.
[28]Meng, Ziqi, et al. "Digital offset for RRAM-based neuromorphic computing: A
novel solution to conquer cycle-to-cycle variation," in Proc. IEEE Design,
Automation & Test in Europe Conference & Exhibition (DATE), Feb. 2021.
[29]K. Sachhidh, J. Rajendran, R. Karri, and O. Sinanoglu, “Sneak path testing of
metal-oxide memristor-based memories,” in Proc. 26th Int’l Conf. VLSI Design,
pp. 386–391, Jan. 2013.
[30]T. Karnik, P. Hazucha, and J. Patel, “Characterization of soft errors caused by
single event upsets in CMOS processes,” IEEE Trans. Depend. Secure Computing,
vol. 1, no. 2, pp. 128–143, Apr.–June 2004.
[31]W. Kuo, W. T. K. Chien, and T. Kim, “Reliability, yield, and stress burn-in,”
Kluwer Academic Publishers, Jan. 1998.
[32]C. Liu, M. Hu, J. P. Strachan, and H. Li, “Rescuing memristor-based neuromorphic
design with high defects,” in Proc. IEEE Design Automation Conference (DAC),
pp. 18-22, Oct. 2017.
[33]Y. Zhu, X. Zhao, and K. Qiu, "Insights and optimizations on IR-drop induced
sneak-Path for RRAM crossbar-based convolutions," in Proc. IEEE Asia and South
Pacific Design Automation Conference (ASP-DAC), pp. 506-511, Jan. 2020.
[34]Y. LeCun, C. Cortes, and C. Burges, “The MNIST database of handwritten digits,”
2012. Available online: http://yann.lecun.com/exdb/mnist/.
[35]Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied
to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.
[36]M. Bocquet, et al., "In-memory and error-immune differential RRAM
implementation of binarized deep neural networks," in Proc. IEEE Int’l Electron
Devices Meeting (IEDM), pp. 20.6.1-20.6.4, Dec. 2018.
[37]Y. BABACAN, “Memristor: Three MOS transistors and one capacitor,” in Proc.
INES, pp. 1–7, Dec. 2017.
[38]S. K. Lu, S. C. Yu, C. L. Hsu, C. T. Sun, M. Hashizume and H. Yotsuyanagi,
"Fault-aware dependability enhancement techniques for flash memories," IEEE
Trans. on Very Large Scale Integration (VLSI) Systems, vol. 28, no. 3, pp. 634-645,
March 2020.