簡易檢索 / 詳目顯示

研究生: 武亭忠
Dinh-Trung Vu
論文名稱: 使用深度學習技術的掌靜脈辨識系統之研究
A Study on a Palm Vein Recognition System Using Deep Learning Techniques
指導教授: 洪西進
Shi-Jinn Horng
口試委員: 趙涵捷
楊竹星
楊昌彪
李正吉
葉佐任
范欽雄
戴文凱
吳怡樂
洪西進
學位類別: 博士
Doctor
系所名稱: 電資學院 - 資訊工程系
Department of Computer Science and Information Engineering
論文出版年: 2023
畢業學年度: 112
語文別: 英文
論文頁數: 73
中文關鍵詞: 掌靜脈辨識飽和度掌部RGB 圖像深度學習掌紋和掌靜脈融合
外文關鍵詞: palm vein recognition, saturation, palm RGB images, deep learning, palm print and palm vein fusion
相關次數: 點閱:32下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

  • Acknowledgment I 摘要 II ABSTRACT III Table of Contents IV List of Figures VI List of Tables VII List of Abbreviations VIII List of Symbols X Chapter 1 Introduction 1 1.1 Background of Image Processing 1 1.2 Dissertation Contribution 2 1.3 Dissertation Organization 3 Chapter 2 Rotation-Invariant Palm ROI Extraction 4 2.1 Overview 4 2.2 Related Work 5 2.3 Proposed Palm ROI Extraction Technique 8 2.3.1 Hand Segmentation 9 2.3.2 Convex Hull Finding and Correcting 10 2.3.3 Key Vector Candidates Finding 11 2.3.4 Optimal Key Vectors Selection 12 2.3.5 ROI Extraction 13 2.4 Experiment Results and Discussion 15 2.4.1 Test on Tongji Dataset 15 2.4.2 Test on a Self-Collected Dataset 16 Chapter 3 Recognizing Palm Vein in Smartphones Using RGB Images 18 3.1 Overview 18 3.2 Related Work 21 3.3 Proposed Method 21 3.3.1 Palm Vein on Saturation Channel 22 3.3.2 ROI Extraction 22 3.3.3 Palm Vein Enhancement 29 3.3.4 Proposed Architecture of Feature Extractor: MPSNet 30 3.4 Experimental Results and Discussion 33 3.4.1 Databases 33 3.4.2 Experiment Settings 35 3.4.3 Evaluate Biometric Performance 39 Chapter 4 Hybrid Deep Learning-Based Palm Fusion Recognition System 44 4.1 Overview 44 4.2 Related Work 46 4.3 Proposed Methodology 49 4.3.1 Pretrained Feature Extraction Module 49 4.3.2 Hybrid Fusion Module 49 4.4 Experimental Results and Discussion 51 4.4.1 Databases 51 4.4.2 Experiment Settings 53 4.4.3 Effectiveness Evaluation of Saturation Channel in MPFNet 54 4.4.4 Evaluation of Hybrid Fusion Strategy 55 4.4.5 Comparison with Other Related Work 55 4.4.6 Real-time Performance Evaluation 61 Chapter 5 Conclusions and Future Work 63 5.1 Conclusions 63 5.2 Future Work 64

    [1] G. Bradski, A. Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library, O’Reilly Media, Inc, 2008.
    [2] S. Brindha, “Finger vein recognition,” International Journal of Renewable Energy Research (IJRER), 2017, vol. 4, pp. 1298-1300.
    [3] W. Damak, R. B. Trabelsi, M.A. Damak, D. Sellami, “Dynamic roi extraction method for hand vein images,” IET Computer Vision, 2018, vol. 12, no. 5, pp. 586-595.
    [4] A. El-Sallam, F. Sohel and M. Bennamoun, “Robust pose invariant shape-based hand recognition,” 2011 6th IEEE Conference on Industrial Electronics and Applications, Beijing, China, 2011, pp. 281-286, doi: 10.1109/ICIEA.2011.5975595.
    [5] H. Han, Z. Sun, F. Wang, and T. Tan, “Palmprint recognition under unconstrained scenes,” Computer Vision–ACCV 2007: 8th Asian Conference on Computer Vision, Tokyo, Japan, November 18-22, 2007, Proceedings, Part II 8, Springer Berlin Heidelberg, pp. 1-11.
    [6] A. Horváth, et al., “Preprocessing endoscopic images of colorectal polyps,” Acta Technica Jaurinensis, 2016, vol. 9, no. 1, pp. 65-82.
    [7] W. L. Jhinn, et al., “Contactless palm vein roi extraction using convex hull algorithm,” Computational Science and Technology: 5th ICCST 2018, Kota Kinabalu, Malaysia, 29-30 August 2018, Springer Singapore, 2019, pp. 25-35.
    [8] W. Kang and Q. Wu, “Contactless palm vein recognition using a mutual foreground-based local binary pattern,” in IEEE Transactions on Information Forensics and Security, vol. 9, no. 11, pp. 1974-1985, Nov. 2014, doi: 10.1109/TIFS.2014.2361020.
    [9] L. Leng, G. Liu, M. Li, M. K. Khan and A. M. Al-Khouri, “Logical conjunction of triple-perpendicular-directional translation residual for contactless palmprint preprocessing,” 2014 11th International Conference on Information Technology: New Generations, Las Vegas, NV, USA, 2014, pp. 523-528, doi: 10.1109/ITNG.2014.18.
    [10] D. Maltoni, D. Maio, A. K. Jain, S. Prabhakar, Handbook of fingerprint recognition, London, Springer, 2009.
    [11] G. K. O. Michael, T. Connie and A. Teoh Beng Jin, “Robust palm print and knuckle print recognition system using a contactless approach,” 2010 5th IEEE Conference on Industrial Electronics and Applications, Taichung, Taiwan, 2010, pp. 323-329, doi: 10.1109/ICIEA.2010.5516864.
    [12] S. Pal, U. Pal, M. Blumenstein, “Signature-based biometric authentication,” Computational Intelligence in Digital Forensics: Forensic Investigation and Applications, 2014, pp. 285-314.
    [13] V. Mura, et al., “Livdet 2017 fingerprint liveness detection competition 2017,” 2018 International Conference on Biometrics (ICB), Gold Coast, QLD, Australia, 2018, pp. 297-302, doi: 10.1109/ICB2018.2018.00052.
    [14] N. Otsu, “A threshold selection method from gray-level histograms,” IEEE transactions on systems, man, and cybernetics, 1979, vol. 9, no. 1, pp. 62-66.
    [15] C. S. Ouyang, R. C. Wu, C. C. Wang, “An improved neural network-based palm biometric system with rotation detection mechanism,” 2010 International Conference on Machine Learning and Cybernetics, Qingdao, China, 2010, pp. 2927-2932, doi: 10.1109/ICMLC.2010.5580754.
    [16] E. Perumal, S. Ramachandran, “A multimodal biometric system based on palmprint and finger knuckle print recognition methods,” International Arab Journal of Information Technology (IAJIT), 2015, vol. 12, no. 2.
    [17] H. Qin, et al., “An iterative deep neural network for hand-vein verification,” IEEE Access, 2019, vol. 7, pp. 34823-34837.
    [18] J. Sklansky, “Finding the convex hull of a simple polygon,” Pattern Recognition Letters, 1982, vol. 1, no. 2, pp. 79-83.
    [19] S. Suzuki, et al., “Top logical structural analysis of digitized binary images by border following,” Computer vision, graphics, and image processing, 1985, vol. 30, no. 1, pp. 32-46.
    [20] J. G. Wang, et al., “Person recognition by fusing palmprint and palm vein images based on “Laplacianpalm” representation,” Pattern Recognition, 2008, vol. 41, no. 5, pp. 1514-1527.
    [21] P. D. Wellner, “Adaptive thresholding for the digitaldesk,” Xerox, EPC1993-110, 1993, pp. 1-19.
    [22] A.L.N. Wong, P. Shi, “Peg-free hand geometry recognition using hierarchical geometry and shape matching,” MVA, 2002, pp. 281–284.
    [23] D. Zhang, et al., “Online palmprint identification,” IEEE Transactions on pattern analysis and machine intelligence, 2003, vol.25, no. 9, pp.1041-1050.
    [24] L. Zhang, et al., “Towards contactless palmprint recognition: A novel device, a new benchmark, and a collaborative representation based identification approach,” Pattern Recognition, 2017, vol. 69, pp.199-212.
    [25] L. Zhang, Z. Cheng, Y. Shen, and D. Wang, “Palmprint and palmvein recognition based on DCNN and a new large–scale contactless palmvein database,” Symmetry, 2018, vol. 10, no. 4: 78.
    [26] M. D. Putro, L. Kurnianggoro and K. -H. Jo, “High performance and efficient real-time face detector on central processing unit based on convolutional neural network,” in IEEE Transactions on Industrial Informatics, vol. 17, no. 7, pp. 4449-4457, July 2021, doi: 10.1109/TII.2020.3022501.
    [27] Z. Xi, et al., “Facial expression recognition of industrial internet of things by parallel neural networks combining texture feature,” IEEE Transactions on Industrial Informatics, vol. 17, issue 4, April 2021, pp. 2784-2793.
    [28] S. Cho, B. Oh, K. Toh and Z. Lin, “Extraction and cross-matching of palm vein and palmprint from the RGB and the NIR spectrums for identity verification,” in IEEE Access, vol. 8, pp. 4005-4021, 2020.
    [29] Dinh-Trung Vu, Thi-Van Nguyen, and Shi-Jinn Horng, “Rotation-invariant palm ROI extraction for contactless recognition,” in Advances in Computer Vision and Computational Biology: Proceedings from IPCV'20, HIMS'20, BIOCOMP'20, and BIOENG'20, Cham: Springer International Publishing, 2021, pp.701-715.
    [30] D. Zhong, S. Liu, W. Wang, and X. Du, “Palm vein recognition with deep hashing network,” in Pattern Recognition and Computer Vision: First Chinese Conference, PRCV 2018, Guangzhou, China, November 23-26, 2018, Proceedings, Part I 1, pp. 38–49.
    [31] D. Thapar, G. Jaswal, A. Nigam and V. Kanhangad, “PVSNet: Palm vein authentication siamese network trained using triplet loss and adaptive hard mining by learning enforced domain specific features,” 2019 IEEE 5th International Conference on Identity, Security, and Behavior Analysis (ISBA), Hyderabad, India, 2019, pp. 1-8, doi: 10.1109/ISBA.2019.8778623.
    [32] A. G. Howard, et al., “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
    [33] CASIA-MS-PalmprintV1, Accessed: Aug. 22, 2020. [Online]. Available: http://biometrics.idealtest.org/dbDetailForUser.do?id=6.
    [34] PolyU Multispectral Palmprint Database, Accessed: Aug. 22, 2020. [Online]. Available: http://www.comp.polyu.edu.hk/~biometrics/MultispectralPalmprint/MSP.htm.
    [35] Xian Jiaotong University Unconstrained Palmprint (XJTU-UP) Database, Accessed: August 22, 2020. [Online]. Available: http://gr.xjtu.edu.cn/web/bell/resource.
    [36] G. K. Michael, T. Connie, and A. B. Teoh, “A contactless biometric system using palm print and palm vein features,” Advanced Biometric Technologies, 2011, pp. 155-177.
    [37] P. Babakhani, P. Zarei, “Automatic gamma correction based on average of brightness,” Advances in Computer Science: an International Journal (ACSIJ), Nov. 2015, vol. 4, no. 6, pp. 156-159.
    [38] S. Pizer, et al., “Adaptive histogram equalization and its variations,” Computer vision, Graphics, and Image Processing, 1987, vol. 39, no. 3, pp. 355-368.
    [39] M. Sandler, et al., “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018. p. 4510-4520.
    [40] A. Howard, et al., “Searching for mobilenetv3,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1314-1324.
    [41] K. He, X. Zhang, S. Ren and J. Sun, “Spatial pyramid pooling in deep convolutional networks for visual recognition,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1904-1916, 1 Sept. 2015, doi: 10.1109/TPAMI.2015.2389824.
    [42] S. Chen, et al., “Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices,” in Biometric Recognition: 13th Chinese Conference, CCBR 2018, Urumqi, China, August 11-12, 2018, Proceedings 13, Springer International Publishing, 2018, pp. 428-438. .
    [43] X. Zhang, et al., “Adacos: Adaptively scaling cosine logits for effectively learning deep face representations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 10823-10832.
    [44] K. He, et al., “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1026-1034.
    [45] A. D. Mora, J. Soares and J. M. Fonseca, “A template matching technique for artifacts detection in retinal images,” 2013 8th International Symposium on Image and Signal Processing and Analysis (ISPA), Trieste, Italy, 2013, pp. 717-722, doi: 10.1109/ISPA.2013.6703831.
    [46] H. A. Nugroho, et al., “Segmentation of plasmodium using saturation channel of HSV color space,” 2019 International Conference on Information and Communications Technology (ICOIACT), Yogyakarta, Indonesia, 2019, pp. 91-94, doi: 10.1109/ICOIACT46704.2019.8938471.
    [47] R. Ranjan, C. D. Castillo, and R. Chellappa, “L2-constrained softmax loss for discriminative face verification,” arXiv preprint arXiv:1703.09507, 2017.
    [48] H. Wang, et al., “Cosface: Large margin cosine loss for deep face recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5265-5274.
    [49] J. Deng, et al., “Arcface: Additive angular margin loss for deep face recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4690-4699.
    [50] A Cost Effective Biometric for Security Application using Infra-Red Vein Imaging with Conventional Computing Platforms, European Commission, Accessed: Sep. 12, 2021. [Online]. Available: https://cordis.europa.eu/project/id/IN10217I.
    [51] Pervasive and UseR Focused BiomeTrics BordEr ProjeCT, European Commission, Accessed: Sep. 12, 2021. [Online]. Available: https://cordis.europa.eu/project/id/700259.
    [52] B. Bhanu; V. Govindaraju, Multibiometrics for human identification, Cambridge University Press, 2011.
    [53] A. Karpathy, et al., “Large-scale video classification with convolutional neural networks,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, pp. 1725-1732.
    [54] X. Yang, et al., “Multilayer and multimodal fusion of deep neural networks for video classification,” in Proceedings of the 24th ACM international conference on Multimedia, 2016, pp. 978-987.
    [55] A. Owens, and AA. Efros., “Audio-visual scene analysis with self-supervised multisensory features,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 631-648.
    [56] Y. Kim, H. Lee, and E. M. Provost, “Deep learning for robust feature generation in audiovisual emotion recognition,” in 2013 IEEE international conference on acoustics, speech and signal processing, IEEE, 2013, pp. 3687-3691.
    [57] S. E. Kahou, et al., “Emonets: Multimodal deep learning approaches for emotion recognition in video,” Journal on Multimodal User Interfaces, 2016, vol. 10, pp. 99-111.
    [58] D. Ramachandram and G. W. Taylor, “Deep multimodal learning: A survey on recent advances and trends,” in IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 96-108, Nov. 2017, doi: 10.1109/MSP.2017.2738401.
    [59] C. H. Chan, M. A. Tahir, J. Kittler and M. Pietikainen, “Multiscale local phase quantization for robust component-based face recognition using kernel fusion of multiple descriptors,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 5, pp. 1164-1177, May 2013, doi: 10.1109/TPAMI.2012.199.
    [60] Y. Huang, H. Ma and M. Wang, “Multimodal finger recognition based on asymmetric networks with fused similarity,” in IEEE Access, vol. 11, pp. 17497-17509, 2023, doi: 10.1109/ACCESS.2023.3242984.
    [61] M. Liu; J. Yuan, “Recognizing human actions as the evolution of pose estimation maps,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 1159-1168.
    [62] M. Abavisani, H. R. Joze, V. M. Patel, “Improving the performance of unimodal dynamic hand-gesture recognition with multimodal training,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 1165-1174.
    [63] A. K. Katsaggelos, S. Bahaadini and R. Molina, “Audiovisual fusion: Challenges and new approaches,” in Proceedings of the IEEE, vol. 103, no. 9, pp. 1635-1653, Sept. 2015, doi: 10.1109/JPROC.2015.2459017.
    [64] K. Simonyan, A. Zisserman, “Two-stream convolutional networks for action recognition in videos,” Advances in neural information processing systems, 2014, vol. 27, pp. 568-576.
    [65] H. R. Joze, et al., “MMTM: Multimodal transfer module for CNN fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 13289-13299.
    [66] S. J. Horng, D. T. Vu, T. V. Nguyen, W. Zhou and C. T. Lin, “Recognizing palm vein in smartphones using RGB images,” in IEEE Transactions on Industrial Informatics, vol. 18, no. 9, pp. 5992-6002, Sept. 2022, doi: 10.1109/TII.2021.3134016.
    [67] Nan Luo, Zhenhua Guo, Gang Wu and Changjiang Song, “Joint palmprint and palmvein verification by Dual Competitive Coding,” 2011 3rd International Conference on Advanced Computer Control, Harbin, 2011, pp. 538-542, doi: 10.1109/ICACC.2011.6016471.
    [68] T. Eglitis, M. Pudzs and M. Greitans, “Bimodal palm biometric feature extraction using a single RGB image,” 2014 International Conference of the Biometrics Special Interest Group (BIOSIG), Darmstadt, Germany, 2014, pp. 1-7.
    [69] O. Nikisins, T. Eglitis, M. Pudzs and M. Greitans, “Algorithms for a novel touchless bimodal palm biometric system,” 2015 International Conference on Biometrics (ICB), Phuket, Thailand, 2015, pp. 436-443, doi: 10.1109/ICB.2015.7139107.
    [70] Q. Li, X. Li, Z. Guo, and J. You, “Online personal verification by palmvein image through palmprint-like and palmvein information,” Neurocomputing, 2015, vol. 147, pp. 364-371.
    [71] T. Wu, L. Leng, M. K. Khan and F. A. Khan, “Palmprint-palmvein fusion recognition based on deep hashing network,” in IEEE Access, vol. 9, pp. 135816-135827, 2021, doi: 10.1109/ACCESS.2021.3112513.
    [72] S. Chen, Z. Guo, J. Feng and J. Zhou, “An improved contact-based high-resolution palmprint image acquisition system,” in IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 9, pp. 6816-6827, Sept. 2020, doi: 10.1109/TIM.2020.2976081.
    [73] Z. Yang, L. Leng and W. Min, “Extreme downsampling and joint feature for coding-based palmprint recognition,” in IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-12, 2021, Art no. 5005112, doi: 10.1109/TIM.2020.3038229.
    [74] S. Cho, B. -S. Oh, K. -A. Toh and Z. Lin, “Extraction and cross-matching of palm-vein and palmprint from the RGB and the NIR spectrums for identity verification,” in IEEE Access, vol. 8, pp. 4005-4021, 2020, doi: 10.1109/ACCESS.2019.2963078.
    [75] X. Dong, M. K. Khan, L. Leng and A. B. J. Teoh, “Co-learning to hash palm biometrics for flexible IoT deployment,” in IEEE Internet of Things Journal, vol. 9, no. 23, pp. 23786-23794, 1 Dec.1, 2022, doi: 10.1109/JIOT.2022.3190020.
    [76] A. Singh, S. Gupta, “Learning to hash: a comprehensive survey of deep learning-based hashing methods,” Knowledge and Information Systems, 2022 Oct, vol. 64, no. 10, pp. 2565-2597.
    [77] Yong-Yi Fanjiang, Cheng-Chi Lee, Yan-Ta Du, and Shi-Jinn Horng, “Palm Vein Recognition Based on Convolutional Neural Network,” Informatica, 2021, vol. 32, no. 4, pp. 687-708.
    [78] Shi-Jinn Horng, Minh-Tuong Le, Dinh-Trung Vu, and Thi-Van Nguyen, “Shadow detection and removal from hand images using synthetic dataset,” 16th International Conference on Pattern Recognition and Information Processing, Belarus, pp. 29-34, 17-19 October 2023.
    [79] T. V. Nguyen, S. J. Horng, D. T. Vu, H. Chen and T. Li, “LAWNet: A lightweight attention-based deep learning model for wrist vein verification in smartphones using RGB images,” in IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1-10, Oct. 2023, doi: 10.1109/tim.2023.3328702.

    無法下載圖示 全文公開日期 2029/01/22 (校內網路)
    全文公開日期 2034/01/22 (校外網路)
    全文公開日期 2034/01/22 (國家圖書館:臺灣博碩士論文系統)
    QR CODE