研究生: |
穆明蘭 Muhammad Bintang Gemintang Sulaiman |
---|---|
論文名稱: |
基於脈衝神經網絡的 MNIST 數字識別 FPGA 實現 FPGA Implementation of MNIST Digit Recognition by using Spiking Neural Network |
指導教授: |
陳伯奇
Poki Chen 鄭桂忠 Kea-Tiong Tang |
口試委員: |
鄭桂忠
Kea-Tiong Tang 陳伯奇 Poki Chen 鍾勇輝 Yung-Hui Chung 徐浩桓 Hao-Huan Hsu |
學位類別: |
碩士 Master |
系所名稱: |
電資學院 - 電子工程系 Department of Electronic and Computer Engineering |
論文出版年: | 2019 |
畢業學年度: | 107 |
語文別: | 英文 |
論文頁數: | 74 |
中文關鍵詞: | 尖端神經網路 、電場可程式化邏輯陣列 、硬體實現 、尖端可塑性 |
外文關鍵詞: | spiking neural network, field-programmable gate array, hardware implementation, spike-timing-dependent plasticity |
相關次數: | 點閱:184 下載:0 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
本論文提出一實現於FPGA之尖端神經網路硬體應用,其功能為識別數位MNIST資料集。採用FPGA實現本應用是因其具有極大的彈性和重構性能加速軟體演算法的快速原型實現。
本論文所採用之尖端神經網路架構可分為兩個層面:輸入端與處理端,在處理端有100個以興奮性神經元構成之處理神經單元以及100個以支持性神經元構成之抑制單元兩部分所組成,其在神經網路中提供側抑制之作用。神經網路中利用尖端可塑性機制達到無監督學習訓練,在產生模擬流程可分為兩個階段,第一階段使用全程式化之模擬,採用Brian simulator從訓練中之網路獲得參數,第二階段將模擬獲得之參數部屬至硬體之中並開始執行內部數位辨識。此架構在可程式化之精確度模擬結果為82.81%,而實現在FPGA上精確度結果達到82.62%。
This thesis provides the hardware implementation of spiking neural networks by using FPGA. The implemented network is to recognize the digit of MNIST dataset. We use the FPGA as the hardware since it offers great flexibility and reconfigurability for fast prototyping acceleration of software algorithms. The proposed architecture of the spiking neural network consists of two layers which are input and processing layer. The processing layer itself consists of 100 excitatory neurons as the main processing neuron unit and 100 inhibitory neuron as the supportive neuron unit to provide the lateral inhibition in the network. The network is trained by using unsupervised learning with spike-timing-dependent plasticity mechanism. There is two-phase of flow process of simulation. The first phase is to simulate the process fully on the programmable simulation. We use Brian simulator to obtain the parameters from the trained network. The second phase of the flow process is to deploy the parameters of the trained network into the hardware and do the digit recognition inside. The testing accuracy performance from the programmable simulation result by using proposed architecture resulting 82.81%, whereas the FPGA implementation result reaches 82.62%.
1. R. Rojas. "Neural Networks: A Systematic Introduction," Berlin: Springer, 1996.
2. A. Tavanaei, M. Ghodrati, S. R. Kheradpisheh, T. Masquelier and A. S. Maida, "Deep learning in spiking neural networks," in Neural Networks, vol. 111, Pages 47-63, 2019.
3. L. O. Slepova and A. A. Zhilenkov, "Synthesis of model of hardware realization of LIF-model of biological neuron on the basis of FPGA," in 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), Moscow, pp. 992-996, 2018.
4. Melissa Flagg, "Structure of a Neuron," Owlcation Medical Science, 2017. Available: https://owlcation.com/stem/Structure-of-a-Neuron [Accessed: 15-Jun-2019]
5. Khan Academy, "The Synapse," Khan Academy, 2017. Available: https://www.khanacademy.org/science/biology/human-biology/neuron-nervous-system/a/the-synapse [Accessed: 3-Jun-2019]
6. OpenStax, "The Action Potential," Anatomy and Physiology by Rice University, 2013. Available: https://opentextbc.ca/anatomyandphysiology/chapter/12-4-the-action-potential/ [Accessed: 26-May-2019]
7. Donald O. Hebb, "The Organization of Behavior," New York: Wiley and Sons, 1949.
8. W. Gerstner and W. Kistler, "Spiking Neuron Models: Single neurons, populations, plasticity," Cambridge University Press, 2002.
9. W. Maass, "Networks of Spiking Neurons: The Third Generation of Neural network Models," in Technische Universitt Graz, 1997.
10. W. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," in Bulletin of Mathematical Biophysics, 5:115133, 1943.
11. Dr. Mark Humphrys, "Sigmoid activation function," School of Computing. Dublin City University, 1987. Available: https://computing.dcu.ie/~humphrys/Notes/Neural/sigmoid.html [Accessed: 4-Jun-2019]
12. H. Paugam-Moisy and S. Bohte, "Computing with Spiking Neuron Networks," in Berlin: Springer, 2012.
13. G. Q. Bi and M. M. Poo, "Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type," in Journal of Neuroscience, 18, 10464-10472, 1998.
14. A. Morrison, A. Aeertsen and M. Diesmann, "Spike-timing dependent plasticity in balanced random networks," in Neural Comput., vol. 19, Pages 1437-1467, 2007.
15. D. Heeger, “Poisson model of spike generation,” in Handout, University of Standford, vol. 5, 2000.
16. Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
17. D. Goodman and R. Brette, "The Brian simulator," in Frontiers in Neuroscience, Vol. 3, p. 26, 2009. doi:10.3389/neuro.01.026.2009
18. N. Rathi, P. Panda and K. Roy, "STDP-Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy-Efficient Recognition," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 38, no. 4, pp. 668-677, April 2019.
19. T. Iakymchuk, A. Rosado-Muñoz, J. F. Guerrero-Martínez, M. Bataller-Mompeán and J. V. Francés-Víllora, "Simplified spiking neural network architecture and STDP learning algorithm applied to image classification," in EURASIP Journal on Image and Video Processing, 2015.
20. Q. Wang, Y. Li, B. Shao, S. Dey and Peng Li, "Energy efficient parallel neuromorphic architectures with approximate arithmetic on FPGA," in Neurocomputing, vol. 221, Pages 146-158, 2017.
21. P. U. Diehl and M. Cook, "Unsupervised learning of digit recognition using spike-timing-dependent plasticity," in Frontiers in computational neuroscience, vol. 9, 2015.