基於脈衝神經網絡的 MNIST 數字識別 FPGA 實現｜國立臺灣科技大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	穆明蘭 Muhammad Bintang Gemintang Sulaiman
論文名稱：	基於脈衝神經網絡的 MNIST 數字識別 FPGA 實現 FPGA Implementation of MNIST Digit Recognition by using Spiking Neural Network
指導教授：	陳伯奇 Poki Chen 鄭桂忠 Kea-Tiong Tang
口試委員:	鄭桂忠 Kea-Tiong Tang 陳伯奇 Poki Chen 鍾勇輝 Yung-Hui Chung 徐浩桓 Hao-Huan Hsu
學位類別：	碩士 Master
系所名稱：	電資學院 - 電子工程系 Department of Electronic and Computer Engineering
論文出版年：	2019
畢業學年度：	107
語文別：	英文
論文頁數：	74
中文關鍵詞：	尖端神經網路、電場可程式化邏輯陣列、硬體實現、尖端可塑性
外文關鍵詞：	spiking neural network, field-programmable gate array, hardware implementation, spike-timing-dependent plasticity
相關次數：	點閱：181 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

上一筆

本論文提出一實現於FPGA之尖端神經網路硬體應用，其功能為識別數位MNIST資料集。採用FPGA實現本應用是因其具有極大的彈性和重構性能加速軟體演算法的快速原型實現。
本論文所採用之尖端神經網路架構可分為兩個層面：輸入端與處理端，在處理端有100個以興奮性神經元構成之處理神經單元以及100個以支持性神經元構成之抑制單元兩部分所組成，其在神經網路中提供側抑制之作用。神經網路中利用尖端可塑性機制達到無監督學習訓練，在產生模擬流程可分為兩個階段，第一階段使用全程式化之模擬，採用Brian simulator從訓練中之網路獲得參數，第二階段將模擬獲得之參數部屬至硬體之中並開始執行內部數位辨識。此架構在可程式化之精確度模擬結果為82.81%，而實現在FPGA上精確度結果達到82.62%。

This thesis provides the hardware implementation of spiking neural networks by using FPGA. The implemented network is to recognize the digit of MNIST dataset. We use the FPGA as the hardware since it offers great flexibility and reconfigurability for fast prototyping acceleration of software algorithms. The proposed architecture of the spiking neural network consists of two layers which are input and processing layer. The processing layer itself consists of 100 excitatory neurons as the main processing neuron unit and 100 inhibitory neuron as the supportive neuron unit to provide the lateral inhibition in the network. The network is trained by using unsupervised learning with spike-timing-dependent plasticity mechanism. There is two-phase of flow process of simulation. The first phase is to simulate the process fully on the programmable simulation. We use Brian simulator to obtain the parameters from the trained network. The second phase of the flow process is to deploy the parameters of the trained network into the hardware and do the digit recognition inside. The testing accuracy performance from the programmable simulation result by using proposed architecture resulting 82.81%, whereas the FPGA implementation result reaches 82.62%.

Acknowledgement    iii
摘要                iv
Abstract             v
Table of Contents   vi
List of Figures      x
List of Tables     xiv

Chapter 1
Introduction             1
1 Introduction         1
2 Motivation           2
3 Thesis Organization  3

Chapter 2
Spiking Neural Network                           4
1 Biological Neuron                            4
1.1 Neuron                                     5
1.2 Synapses                                   5
1.3 Membrane Potential                         6
1.4 Synaptic Plasticity and Learning           7
2 Neural Network Development                   8
2.1 First Generation                           9
2.2 Second Generation                          9
2.3 Third Generation                          10
2.4 Purpose of Neural Network Implementation  11
3 Spiking Neural Network                      12
3.1 Leaky-Integrate-and-Fire Neuron           12
3.2 Synapse and STDP Learning                 14
3.3 Lateral Inhibition                        16
3.4 Dynamic Threshold                         17
4 Poisson Spike Distribution                  17
4.1 Probability and Random Processes          18
4.2 Homogeneous Poisson Process               18
4.3 Poisson Spike Trains Generation           19

Chapter 3
Design of Spiking Neural Network              21
1 Spiking Neural Network Architecture       21
1.1 Input Layer                             22
1.2 Excitatory Layer                        22
1.3 Inhibitory Layer                        23
1.4 Synapses                                23
1.5 Membrane Potential and Conductance      25
2 Flow Process of Simulation                26
3 Programmable Simulation Method            28
3.1 Brian Simulator                         28
3.2 Input Encoding                          28
3.3 Training, Classification and Testing    29
3.4 Quantization                            30
4 Design of Spiking Neural Network on FPGA  31
4.1 Simplified Spiking Neural Network       33
4.2 Input Encoding                          34
4.3 Neuron Unit                             36
4.4 Classifier Unit                         37

Chapter 4
Simulation Result                    39
1 Programmable Simulation Results  39
1.1 Training Accuracy              40
1.2 Training Result                41
1.3 Weights Quantization           42
1.4 Dynamic Threshold              44
1.5 Membrane Potential             45
1.6 Testing Accuracy               47

Chapter 5
Spiking Neural Network Implementation on FPGA  49
1 Block Diagram of Implementation            50
1.1 UART                                     50
1.2 Input Layer                              51
1.3 Excitatory Layer                         52
1.4 Inhibitory Layer                         54
1.5 Threshold                                55
1.6 Weights                                  55
1.7 Classifier                               56
2 Mapping from Brian Simulator into FPGA     57
2.1 Weights Mapping                          57
2.2 Threshold Mapping                        60
2.3 Assigned-Class Neuron                    61
3 FPGA Behavior Simulation Results           63
3.1 Poisson Input Spike Test                 63
3.2 Membrane Potential                       65
3.3 Recognition Result                       66
4 Accuracy Results and Comparison            68

Chapter 6
Conclusion  70

References  72

                                

1. R. Rojas. "Neural Networks: A Systematic Introduction," Berlin: Springer, 1996.

2. A. Tavanaei, M. Ghodrati, S. R. Kheradpisheh, T. Masquelier and A. S. Maida, "Deep learning in spiking neural networks," in Neural Networks, vol. 111, Pages 47-63, 2019.

3. L. O. Slepova and A. A. Zhilenkov, "Synthesis of model of hardware realization of LIF-model of biological neuron on the basis of FPGA," in 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), Moscow, pp. 992-996, 2018.

4. Melissa Flagg, "Structure of a Neuron," Owlcation Medical Science, 2017. Available: https://owlcation.com/stem/Structure-of-a-Neuron [Accessed: 15-Jun-2019]

5. Khan Academy, "The Synapse," Khan Academy, 2017. Available: https://www.khanacademy.org/science/biology/human-biology/neuron-nervous-system/a/the-synapse [Accessed: 3-Jun-2019]

6. OpenStax, "The Action Potential," Anatomy and Physiology by Rice University, 2013. Available: https://opentextbc.ca/anatomyandphysiology/chapter/12-4-the-action-potential/ [Accessed: 26-May-2019]

7. Donald O. Hebb, "The Organization of Behavior," New York: Wiley and Sons, 1949.

8. W. Gerstner and W. Kistler, "Spiking Neuron Models: Single neurons, populations, plasticity," Cambridge University Press, 2002.

9. W. Maass, "Networks of Spiking Neurons: The Third Generation of Neural network Models," in Technische Universitt Graz, 1997.

10. W. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," in Bulletin of Mathematical Biophysics, 5:115133, 1943.

11. Dr. Mark Humphrys, "Sigmoid activation function," School of Computing. Dublin City University, 1987. Available: https://computing.dcu.ie/~humphrys/Notes/Neural/sigmoid.html [Accessed: 4-Jun-2019]

12. H. Paugam-Moisy and S. Bohte, "Computing with Spiking Neuron Networks," in Berlin: Springer, 2012.

13. G. Q. Bi and M. M. Poo, "Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type," in Journal of Neuroscience, 18, 10464-10472, 1998.

14. A. Morrison, A. Aeertsen and M. Diesmann, "Spike-timing dependent plasticity in balanced random networks," in Neural Comput., vol. 19, Pages 1437-1467, 2007.

15. D. Heeger, “Poisson model of spike generation,” in Handout, University of Standford, vol. 5, 2000.

16. Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.

17. D. Goodman and R. Brette, "The Brian simulator," in Frontiers in Neuroscience, Vol. 3, p. 26, 2009. doi:10.3389/neuro.01.026.2009

18. N. Rathi, P. Panda and K. Roy, "STDP-Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy-Efficient Recognition," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 38, no. 4, pp. 668-677, April 2019.

19. T. Iakymchuk, A. Rosado-Muñoz, J. F. Guerrero-Martínez, M. Bataller-Mompeán and J. V. Francés-Víllora, "Simplified spiking neural network architecture and STDP learning algorithm applied to image classification," in EURASIP Journal on Image and Video Processing, 2015.

20. Q. Wang, Y. Li, B. Shao, S. Dey and Peng Li, "Energy efficient parallel neuromorphic architectures with approximate arithmetic on FPGA," in Neurocomputing, vol. 221, Pages 146-158, 2017.

21. P. U. Diehl and M. Cook, "Unsupervised learning of digit recognition using spike-timing-dependent plasticity," in Frontiers in computational neuroscience, vol. 9, 2015.

全文公開日期 2024/08/19 (校內網路)
全文公開日期 2024/08/19 (校外網路)
全文公開日期 2024/08/19 (國家圖書館：臺灣博碩士論文系統)

簡易檢索 / 詳目顯示

相關論文