本論文目的在於利用三維人臉重構來提升人臉辨識系統的準確率，並將其實現於FPGA Xilinx ZCU104達到邊緣運算，結合此塊開發板的兩大系統PS和PL，在讀取二維人臉影像後，透過影像預處理、人臉偵測、三維人臉重構、特徵提取以及與人臉資料庫比對完成人臉辨識。
首先我們會對各種三維人臉重構技術做研究並比較，本論文使用基於3D Morphable Model的重建方法得到輸入影像對應的三維人臉模型，再透過此模型旋轉投影得到正面人臉，最後以此來進行人臉辨識。
在FPGA我們使用Xilinx所開發之PYNQ系統，其映像檔是基於Ubuntu加上Python建構而成，PYNQ內建了Jupyter Notebook，使開發者可以直接在上面撰寫Python程式及除錯，直接使用豐富的資料庫就可對FPGA硬體進行功能的開發，例如：使用ZCU104透過HDMI輸出影像至顯示器上，只需調用API即可，不需要另外自行對PL層設計。
最後我們設計一系列實驗，採用Color FERET和CMU PIE人臉資料庫，皆被廣泛使用在多角度人臉辨識研究，因其提供大量人臉及不同角度人臉照，角度包含0度、 15度、 22.5度、 45度、 67.5度及 90度。
實驗結果分析三維人臉重構對於人臉辨識系統確實有改善，並且在不同的資料庫下，準確率是可複製的，不因資料庫的差異影響準確率，而運用三維人臉重建彌補大角度人臉比對失敗的問題，使整體準確率可確實提升，在大角度(大於45度的人臉影像)下的提升，皆有至少18%的改善。

The purpose of this thesis is to enhance accuracy of facial recognition system through 3D face reconstruction and reach edge computing while realizing thus in FPGA Xilinx ZCU104 in order to integrate PS and PL of the evaluation board. After reading 2D facial image, we use image preprocessing, face detection, 3D face reconstruction, feature extraction and compare it with existing facial image database to complete recognition process.
First, we will look into and compare various kinds of 3D face recognition. We used 3D Morphable Model reconstruction to acquire corresponding 3D face model, then use the model to rotate the projection. The projection would be a frontal face available for making comparison with database.
We use PYNQ system on FPGA. The image file was built by Ubuntu and Python. PYNQ, along with built-in Jupyter Notebook, enable developers to code and debug on it directly. With enormous database we may develop all kinds of functions on FPGA hardware.
Finally, we designed a series of experiment applied with Color FERET and CMU PIE face database. Both databases are commonly used in research of face recognition across poses. Face recognition across poses provides huge amount of facial data and face image of various angles including 0°, 15°, 22.5°, 45°, 67.5° and 90°.
The result of experiments showed that 3D face reconstruction does improve the accuracy of facial recognition system, especially on compensating big angle (more than 45°) comparison failure. General accuracy is improved at least 18% more.

[1] V. Blanz and T.Vetter , "Face recognition based on fitting a 3D morphable model," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, pp. 1063-1074, 2003.
[2] Carlos D. Castillo and David W. Jacobs, "Using Stereo Matching for 2-D Face Recognition Across Pose," IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[3] D. Jiang, Y. Hu,S.Yan, L. Zhang, H. Zhang and W. Gao, "Efficient 3D reconstruction for face recognition," Elsevier Pattern Recognition, vol. 38, pp. 787-798, 2005.
[4] Xiangyu Zhu, Xiaoming Liu, Zhen Lei and Stan Z. Li, "Face Alignment in Full Pose Range: A 3D Total Solution," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, pp. 78-92, 2017.
[5] Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang and Xi Zhou., "Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 534-551, 2018.
[6] Kaiming He, "Deep Residual Learning for Image Recognition," Proc. Computer Vision and Pattern Recognition (CVPR), IEEE., pp. 1-9, 2016.
[7] Ze Lu, Xudong Jiang and Alex Kot, "Deep Coupled ResNet for Low-Resolution Face Recognition," IEEE Signal Processing Letters, vol.25, pp. 526-530, 2018.
[8] H. M. Rehan Afzal, Suhuai Luo, M. Kamran Afzal, Gopal Chaudhary, Manju Khari and Sathish A. P. Kumar, " 3D Face Reconstruction From Single 2D Image Using Distinctive Features," IEEE Access, pp. 180681-180689, 2020
[9] Tao Wu, Fei Zhou and Qingmin Liao, " Real-time 3D face reconstruction from one single image by displacement mapping," IEEE International Conference on Image Processing (ICIP), pp. 1-5, 2017.
[10] Patrik Huber, Philipp Kopp, William Christmas, Matthias Rätsch and Josef Kittler, "Real-Time 3D Face Fitting and Texture Fusion on In-the-Wild Videos," IEEE Signal Processing Letters, vol. 24, pp. 437-441, 2017.
[11] P. Huber et al., "A multiresolution 3D morphable face model and fitting framework," Proc. Int. Conf. Comput. Vision Theory Appl., vol.4, pp. 79-86, 2016.
[12] Chen Cao, Yanlin Weng, Shun Zhou, Yiying Tong and Kun Zhou, " FaceWarehouse: A 3D Facial Expression Database for Visual Computing," IEEE Transactions on Visualization and Computer Graphics, vol. 20, pp. 413-425, 2014.
[13] Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia and Xin Tong, " Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set," IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1-11, 2019.
[14] Mingliang Xu, Fuhai Chen, Lu Li, Chen Shen, Pei Lv, Bing Zhou and Rongrong Ji, "Bio-Inspired Deep Attribute Learning Towards Facial Aesthetic Prediction," IEEE Transactions on Affective Computing, vol.12, pp. 227-238, 2021.
[15] Yikui Zhai, Yu Huang, Ying Xu and Junying Gan, "Asian Female Facial Beauty Prediction Using Deep Neural Networks via Transfer Learning and Multi-Channel Feature Fusion," IEEE Access, vol. 8, pp. 56892-56907, 2005.
[16] Yudong Guo, juyong zhang, Jianfei Cai, Boyi Jiang and Jianmin Zheng," CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.41, pp. 1294-1307, 2019.
[17] Wenqi Wu, Yingjie Yin, Xingang Wang and De Xu, " Face Detection With Different Scales Based on Faster R-CNN," IEEE Transactions on Cybernetics, vol. 49, pp. 4017-4028, 2018.
[18] Sanghoon Kang, Jinmook Lee, Kyeongryeol Bong, Changhyeon Kim, Youchang Kim and Hoi-Jun Yoo, "Low-Power Scalable 3-D Face Frontalization Processor for CNN-Based Face Recognition in Mobile Devices," IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 8, pp. 873-883, 2019.
[19] Zhen Zheng, Bingting Zha, Hailu Yuan and Youshi Xuchen, " Edge Detection Algorithm based on Morphology and Grey Relation Analysis," IEEE International Conference on Mechatronics and Automation (ICMA), pp. 1-6, 2019.
[20] P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features,"Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2001.
[21] Xiangyu Zhu, Xiaoming Liu, Zhen Lei and Stan Z. Li, "Face Alignment Across Large Poses: A 3D Solution," IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-10, 2016.
[22] N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2005.
[23] T. Heseltine, N. Pears, J. Austin and Z. Chen, "Face Recognition: A Comparison of Appearance-Based Approaches," Proc. VIIth Digital Image Computing: Techniques and Applications, pp. 59-68, 2003.
[24] P.N. Belhumeur,J.P. Hespanha and D.J. Kriegman, "Eigenfaces vs. Fisherfaces: recognition using class specific linear projection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, pp. 711-720, 1997.
[25] Andrea F. Abate, Paola Barra, Silvio Barra, Cristiano Molinari, Michele Nappi and Fabio Narducci, "Clustering Facial Attributes: Narrowing the Path From Soft to Hard Biometrics," IEEE Access, vol. 8, pp. 9037-9045, 2020.
[26] Luis Cárabe and Eduardo Cermeño, " Stegano-morphing: Concealing attacks on face identification algorithms," IEEE Access, pp. 1-21, 2021.
[27] Jie Sun, Siyu Xia, Zhilin Sun and Shengli Lu, " Cross-Model Deep Feature Fusion for Face Detection," IEEE Sensors Letters, vol. 4, pp. 1-4, 2020.
[28] Baoliang Chen, Wenhan Yang, Haoliang Li, Shiqi Wang and Sam Kwong, "Camera Invariant Feature Learning for Generalized Face Anti-Spoofing," IEEE Transactions on Information Forensics and Security, vol.16, pp. 2477-2492, 2021.
[29] Si-Qi Liu, Xiangyuan Lan and Pong C. Yuen, "Multi-Channel Remote Photoplethysmography Correspondence Feature for 3D Mask Face Presentation Attack Detection," IEEE Transactions on Information Forensics and Security, vol.16, pp. 2683-2692, 2021.
[30] Matthew Turk and Alex Pentland, "Eigenfaces for Recognition," Journal of Cognitive Neuroscience, vol. 3, pp. 71-86, 1991.
[31] Yanhong Zhou, Shukai Cao, Dong Wen, Huiyang Zhang and Liqiang Zhao, "The study of face recognition based on hybrid principal components analysis and independent component analysis," International Conference on Electronics, Communications and Control, pp. 1-4, 2011.
[32] TaiLin Han, Guangw Wen Liu, Hua Cai and Bo Wang, "The face detection and location system based on Zynq," in Proc. Int. Conference on Fuzzy Systems and Knowledge Discovery, pp. 835-839, 2014.
[33] Feng Liu, Qijun Zhao, Xiaoming Liu and Dan Zeng, " Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, pp. 664-678, 2020.
[34] Jian Zhao, Lin Xiong, Jianshu Li, Junliang Xing, Shuicheng Yan and Jiashi Feng, " 3D-Aided Dual-Agent GANs for Unconstrained Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, pp. 2380-2394, 2019.
[35] T. Sim, S. Baker and M. Bsat, "The CMU Pose, Illumination, and Expression (PIE) database," IEEE International Conference on Automatic Face Gesture Recognition, pp. 46-51, 2002.
[36] Xiaozheng Zhang and Yongsheng Gao, "Face recognition across pose: A review," Pattern Recognition, vol. 42, pp. 2876-2896, 2009.
[37] R. Gross, I. Matthews and S. Baker, "Appearance-based face recognition and light-fields," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, pp. 449-465, 2004.
[38] Xiaozheng Zhang and Yongsheng Gao, "Face recognition across pose: A review," Pattern Recognition, vol. 42, pp. 2876-2896, 2009.
[39] Yoav Freund and Robert E Schapire, "A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting," Journal of Computer and System Sciences,vol. 55, pp. 119-139, 1997.
[40] T. Ahonen,A. Hadid and M. Pietikainen, "Face Description with Local Binary Patterns: Application to Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, pp. 2037-2041, 2006.
[41] Irfan Mehmood, Amin Ullah, Khan Muhammad, Der-Jiunn Deng and Weizhi Meng, " Efficient Image Recognition and Retrieval on IoT-Assisted Energy-Constrained Platforms From Big Data Repositories," IEEE Internet of Things Journal, vol. 6, pp. 9246-9255, 2019.