研究生: |
卡齊 Qazi Mazhar Ul Haq |
---|---|
論文名稱: |
一種基於邊緣感知的雙目圖像立體匹配特徵提取技術 An Edge-aware Based Feature set Extraction and Stereo Matching of Binocular Images Under Radiometric Variation |
指導教授: |
阮聖彰
Shanq-Jang Ruan |
口試委員: |
林昌鴻
Chang-Hong Lin 呂政修 Jenq-Shiou Leu 魏榮宗 Rong-Jong Wai 彭文志 Wen-Chih Peng 彭彥璁 Yan-Tsung Peng |
學位類別: |
博士 Doctor |
系所名稱: |
電資學院 - 電子工程系 Department of Electronic and Computer Engineering |
論文出版年: | 2021 |
畢業學年度: | 110 |
語文別: | 英文 |
論文頁數: | 90 |
中文關鍵詞: | 立體匹配 、功能集 、直方圖均衡 、双目影像 、梯度模型 、中值濾波 |
外文關鍵詞: | Stereo Matching, Feature sets, Histogram Equalization, Binocular images, Gradient Models, Median filtering |
相關次數: | 點閱:250 下載:4 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
物件立體視覺通常是電腦視覺中深入研究的領域之一。立體匹配用於許多現代應用,包括機器人導航、增強現實和汽車應用。儘管它有著悠久的研究歷史,但對於輻射變化下的無紋理、不連續和遮擋區域的邊緣仍然具有挑戰性。這篇研究文章提出了一種改進的長條圖均衡化、一種新穎的特徵提取、一種空間梯度模型和匹配成本,它對在不同輻射變化下拍攝的圖像具有魯棒性和穩定性。所提出的方法將不良圖元的平均百分比降低到 3.35,並將 Middlebury 資料集上差異照明和曝光值的相對均方誤差 (RMSE) 降低到 30.08。對所提出方法的定量和定性評估表明,在增加 PSNR 和降低壞圖元百分比方面對輻射變化和最先進的局部立體匹配演算法有顯著改善。
Object Stereo Vision has conventionally been one of the deeply examined areas in computer vision. Stereo matching is employed in numerous modern applications, including robot navigation, augmented reality, and automotive applications. Even though it has a long research history, it is still challenging for the edges of textureless, discontinues, and occluded regions under radiometric variation. This research article proposes a modified histogram equalization, a novel feature extraction, a spatial gradient model, and matching cost, which is robust and stable to images taken in different radiometric variations. The proposed method reduced the average percentage of bad pixels to 3.35 and reduced the relative mean square error (RMSE) up to 30.08 on the Middlebury dataset for different illumination and exposure values. Quantitative and qualitative evaluation of the proposed method demonstrates significant improvement in increasing PSNR and decreasing bad pixel percentage against radiometric variation and stateoftheart local stereo matching algorithms.
[1] Q.M. U.Haq, M. A.Haq, S.J. Ruan, P.J.Liang, and D.Q.Gao, “3d object detection based on proposal generation network utilizing monocular images,” IEEE Consumer Electronics Magazine, pp. 1–1, DOI: 10.1109/MCE.2021.3059565, 2021.
[2] P. Chondro, Z.R. Yao, and S.J. Ruan, “Depthbased dynamic lightness adjustment powersaving algorithm for AMOLED in headmounted display,”Optics express,vol. 26, no. 25, pp. 33158–33165, 2018.
[3] Q. M. U. Haq, S.J. Ruan, M. A. Haq, S. Karam, J. L. Shieh, P. Chondro, and D.Q.Gao, “An incremental learning of yolov3 without catastrophic forgetting for smart city applications,” IEEE Consumer Electronics Magazine, pp. 1–1, DOI:10.1109/MCE.2021.3096376, 2021.
[4] Y. Xu, M. Li, L. Cui, S. Huang, F. Wei, and M. Zhou, “Layoutlm: Pretrainingof text and layout for document image understanding,” in Proceedings of the 26th ACMSIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1192–1200, 2020.
[5] T. Kanade and M. Okutomi, “A stereo matching algorithm with an adaptive window: Theory and experiment,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 9, pp. 920–932, 1994.
[6] K.J. Yoon and I. S. Kweon, “Adaptive supportweight approach for correspondence search,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 650–656, 2006.
[7] M. Gong and Y.H. Yang, “Near realtime reliable stereo matching using programmable graphics hardware,” in2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 924–931, IEEE,2005.
[8] E. Gudis, G. van der Wal, S. Kuthirummal, S. Chai, S. Samarasekera, R. Kumar, and V. Branzoi, “Stereo vision embedded system for augmented reality,” in201282
IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 15–20, IEEE, 2012.
[9] Z. Zhang, X. Ai, N. Canagarajah, and N. Dahnoun, “Local stereo disparity estimation with novel cost aggregation for subpixel accuracy improvement in automotive applications,” in 2012 IEEE Intelligent Vehicles Symposium, pp. 99–104, IEEE, 2012.
[10] J. M. LópezValles, M. A. Fernández, A. FernándezCaballero, M. T. López, J. Mira,and A. E. Delgado, “Motionbased stereo vision method with potential utility in robot navigation,” in International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 16–25, Springer, 2005.
[11] X. Zhou and P. Boulanger, “Radiometric invariant stereo matching based on relative gradients,” in 2012 19th IEEE International Conference on Image Processing, pp. 2989–2992, IEEE, 2012.
[12] H. Hirschmuller and D. Scharstein, “Evaluation of stereo matching costs on images with radiometric differences,” IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 31, no. 9, pp. 1582–1599, 2008.
[13] N. Gac, S. Mancini, M. Desvignes, and D. Houzet, “High-speed 3d tomography CPU, GPU, and FPGA,” EURASIP Journal on Embedded systems, vol. 2008, pp. 1–12,2009.
[14] G. Zhao, Y. K. Du, and Y. D. Tang, “A new extension of the rank transform for stereo matching,” inAdvanced Engineering Forum, vol. 2, pp. 182–187, Trans Tech Publ,2012.
[15] X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang, and X. Zhang, “On building an accurate stereo matching system on graphics hardware,” in 2011 IEEE InternationalConference on Computer Vision Workshops (ICCV Workshops), pp. 467–474, IEEE,2011.
[16] C. Cigla and A. A. Alatan, “Information permeability for stereo matching,” Signal Processing: Image Communication, vol. 28, no. 9, pp. 1072–1088, 2013.83
[17] K. R. Vijayanagar, M. Loghman, and J. Kim, “Realtime refinement of Kinect depth map using multiresolution anisotropic diffusion,” Mobile Networks and Applications, vol. 19, no. 3, pp. 414–425, 2014.
[18] M. Michael, J. Salmen, J. Stallkamp, and M. Schlipsing, “Realtime stereo vision: Optimizing semiglobal matching,” in2013 IEEE Intelligent Vehicles Symposium(IV), pp. 1197–1202, IEEE, 2013.
[19] Z. Ma, K. He, Y. Wei, J. Sun, and E. Wu, “Constant time-weighted median filtering for stereo matching and beyond,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 49–56, 2013.
[20] A. Banno and K. Ikeuchi, “Disparity map refinement and 3d surface smoothing via directed anisotropic diffusion,” Computer Vision and Image Understanding, vol. 115, no. 5, pp. 611–619, 2011.
[21] S. Zhang, C. Wang, and S. Chan, “A new high-resolution depth map estimation system using stereo vision and depth-sensing device,” in 2013 IEEE 9th InternationalColloquium on Signal Processing and its Applications, pp. 49–53, IEEE, 2013.
[22] E. Z. Psarakis and G. D. Evangelidis, “An enhanced correlationbased method for stereo correspondence with subpixel accuracy,” in Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, vol. 1, pp. 907–912, IEEE, 2005.[23] B. Jia, S. Liu, and Z. Du, “A progressive framework for dense stereo matching,” Pattern Recognition and Image Analysis, vol. 26, no. 2, pp. 294–301, 2016.
[24] S. Mukherjee and R. M. R. Guddeti, “A hybrid algorithm for disparity calculation from sparse disparity estimates based on stereo vision,” in2014 International Conference on Signal Processing and Communications (SPCOM), pp. 1–6, IEEE, 2014.
[25] K. Zhang, J. Li, Y. Li, W. Hu, L. Sun, and S. Yang, “Binary stereo matching,” inProceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 356–359, IEEE, 2012.
[26] G. A. Kordelas, D. S. Alexiadis, P. Daras, and E. Izquierdo, “Enhanced disparity estimation in stereo images,” Image and Vision Computing, vol. 35, pp. 31–49, 2015.84
[27] S. Lee, J. H. Lee, J. Lim, and I. H. Suh, “Robust stereo matching using adaptive random walk with restart algorithm,” Image and Vision Computing, vol. 37, pp. 1–11, 2015.
[28] M. Yang, Y. Liu, Y. Cai, and Z. You, “Stereo matching based on classification of materials,” Neurocomputing, vol. 194, pp. 308–316, 2016.
[29] Y. S. Heo, K. M. Lee, and S. U. Lee, “Robust stereo matching using adaptive normalized crosscorrelation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 4, pp. 807–822, 2010.
[30] I.L. Jung, T.Y. Chung, J.Y. Sim, and C.S. Kim, “Consistent stereo matching under varying radiometric conditions,” IEEE Transactions on Multimedia, vol. 15, no. 1, pp. 56–69, 2012.
[31] Y. S. Heo, K. M. Lee, and S. U. Lee, “Joint depth map and color consistency estimation for stereo images with different illuminations and cameras,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 5, pp. 1094–1106, 2012.
[32] T. Mounts, N. Aouf, and M. A. Richardson, “A novel image representation via local frequency analysis for illumination invariant stereo matching,” IEEE Transactions on Image Processing, vol. 24, no. 9, pp. 2685–2700, 2015.
[33] V. D. Nguyen, D. D. Nguyen, S. J. Lee, and J. W. Jeon, “Local density encoding for robust stereo matching,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, no. 12, pp. 2049–2062, 2014.
[34] Y. Qu, J. Jiang, X. Deng, and Y. Zheng, “Robust local stereo matching under varying radiometric conditions,” IET Computer Vision, vol. 8, no. 4, pp. 263–276, 2014.
[35] V. Q. Dinh, V. D. Nguyen, and J. W. Jeon, “Robust matching cost function for stereo correspondence using matching by tone mapping and adaptive orthogonal integral image,” IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5416–5431,2015.85
[36] L. Xu, O. C. Au, W. Sun, L. Fang, F. Zou, and J. Li, “Stereo matching with optimal local adaptive radiometric compensation,” IEEE Signal Processing Letters, vol. 22, no. 2, pp. 131–135, 2014.
[37] L. DeMaeztu, A. Villanueva, and R. Cabeza, “Stereo matching using gradient similarity and locally adaptive supportweight,” Pattern Recognition Letters, vol. 32, no. 13, pp. 1643–1651, 2011.
[38] I.L. Jung, J.Y. Sim, C.S. Kim, and S.U. Lee, “Robust stereo matching under radiometric variations based on cumulative distributions of gradients,” in 2013 IEEE International Conference on Image Processing, pp. 2082–2085, IEEE, 2013.
[39] Y.H. Kim, J. Koo, and S. Lee, “Adaptive descriptorbased robust stereo matching under radiometric changes,” Pattern Recognition Letters, vol. 78, pp. 41–47, 2016
.[40] C.K. Liang, C.C. Cheng, Y.C. Lai, L.G. Chen, and H. H. Chen, “Hardwareefficient belief propagation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 5, pp. 525–537, 2011.
[41] M. Z. Brown, D. Burschka, and G. D. Hager, “Advances in computational stereo,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 993–1008, 2003.
[42] H. Hirschmuller, “Stereo processing by semi-global matching and mutual information,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328–341, 2007.
[43] K. Yamaguchi, D. McAllester, and R. Urtasun, “Efficient joint segmentation, occlusion labeling, stereo, and flow estimation,” in European Conference on ComputerVision, pp. 756–771, Springer, 2014.
[44] C. Zhang, Z. Li, Y. Cheng, R. Cai, H. Chao, and Y. Rui, “Meshstereo: A global stereo model with mesh alignment regularization for view interpolation,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 2057–2065, 2015.86
[45] D.Scharsteinand R.Szeliski, “A taxonomy and evaluation of dense twoframestereocorrespondence algorithms,” International Journal of Computer Vision, vol. 47, no. 1, pp. 7–42, 2002.
[46] X. Zhang and Z. Chen, “Sadbased stereo vision machine on a systemonprogrammablechip (sopc),” Sensors, vol. 13, no. 3, pp. 3014–3027, 2013.
[47] R. K. Gupta and S.Y. Cho, “Windowbased approach for fast stereo correspondence,” IET Computer Vision, vol. 7, no. 2, pp. 123–134, 2013.
[48] K. Sharma, K.y. Jeong, and S.G. Kim, “Vision-based autonomous vehicle navigation with selforganizing map feature matching technique,” in 2011 11th International Conference on Control, Automation, and Systems, pp. 946–949, IEEE, 2011.
[49] X. Song, X. Zhao, L. Fang, H. Hu, and Y. Yu, “Edgestereo: An effective multitasklearning network for stereo matching and edge detection,” International Journal of Computer Vision, vol. 128, no. 4, pp. 910–930, 2020.
[50] F. Ekstrand, C. Ahlberg, M. Ekström, and G. Spampinato, “Highspeedsegmentationdriven highresolution matching,” in Seventh International Conference on Machine Vision (ICMV 2014), vol. 9445, p. 94451Y, International Society for Optics and Photonics, 2015.
[51] D.Scharsteinand R.Szeliski, “A taxonomy and evaluation of dense twoframestereocorrespondence algorithms,” International Journal of Computer Vision, vol. 47, no. 1, pp. 7–42, 2002.
[52] J. Fang, A. L. Varbanescu, J. Shen, H. Sips, G. Saygili, and L. Van Der Maaten, “Accelerating cost aggregation for realtime stereo matching,” in2012 IEEE 18thInternational Conference on Parallel and Distributed Systems, pp. 472–481, IEEE,2012.
[53] A. Hosni, M. Bleyer, and M. Gelautz, “Secrets of adaptive support weight techniques for local stereo matching,” Computer Vision and Image Understanding, vol. 117, no. 6, pp. 620–632, 2013.87
[54] L. Nalpantidis and A. Gasteratos, “Stereo vision for robotic applications in the presence of nonideal lighting conditions,” Image and Vision Computing, vol. 28, no. 6, pp. 940–951, 2010.
[55] K. Zhang, J. Lu, Q. Yang, G. Lafruit, R. Lauwereins, and L. Van Gool, “Realtimeand accurate stereo: A scalable approach with bitwise fast voting on Cuda,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 7, pp. 867–878, 2011.
[56] Z. Lee, J. Juang, and T. Q. Nguyen, “Local disparity estimation with threemodedcross census and advanced support weight,” IEEE Transactions on Multimedia, vol. 15, no. 8, pp. 1855–1864, 2013.
[57] H.q. Wang, M. Wu, Y.b. Zhang, and L. Zhang, “Effective stereo matching using reliable points-based graph cut,” in 2013 Visual Communications and Image Processing (VCIP), pp. 1–6, IEEE, 2013.
[58] A. Arranz, Á. Sánchez, and M. Alvar, “Multiresolution energy minimization framework for stereo matching,” IET Computer Vision, vol. 6, no. 5, pp. 425–434, 2012.
[59] Q. Yang, L. Wang, R. Yang, H. Stewénius, and D. Nistér, “Stereo matching withcolorweighted correlation, hierarchical belief propagation, and occlusion handling,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 492–504, 2008.
[60] A. Fusiello, U. Castellani, and V. Murino, “Relaxing symmetric multiple windows stereo using Markov random fields,” inInternationalWorkshoponEnergyMinimization Methods in Computer Vision and Pattern Recognition, pp. 91–105, Springer,2001.
[61]R. Yang and M. Pollefeys, “Multiresolution realtime stereo on commodity graphics hardware,” in 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., vol. 1, pp. I–I, IEEE, 2003.
[62]B. J. Tippetts, D.J. Lee, J. K. Archibald, and K. D. Lillywhite, “Dense disparityrealtime stereo vision algorithm for resourcelimited systems,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 10, pp. 1547–1555, 2011.88
[63]S. H. Lee and S. Sharma, “Realtime disparity estimation algorithm for stereo camera systems,” IEEE Transactions on Consumer Electronics, vol. 57, no. 3, pp. 1018–1026, 2011.
[64]M. Humenberger, C. Zinner, M. Weber, W. Kubinger, and M. Vincze, “A fast stereo matching algorithm suitable for embedded realtime systems,” Computer Vision and Image Understanding, vol. 114, no. 11, pp. 1180–1202, 2010.
[65]L. Ma, J. Li, J. Ma, and H. Zhang, “A modified census transform based on the neighborhood information for the stereo matching algorithm,” in 2013 Seventh InternationalConference on Image and Graphics, pp. 533–538, IEEE, 2013.
[66]H. Hirschmuller and D. Scharstein, “Evaluation of cost functions for stereo matching,” in2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, IEEE, 2007.
[67]S. Satoh, “Simple lowdimensional features are approximating nccbased image matching,” Pattern Recognition Letters, vol. 32, no. 14, pp. 1902–1911, 2011.
[68]D. Scharstein and C. Pal, “Learning conditional random fields for stereo,” in 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, IEEE, 2007.
[69] L. DeMaeztu, A. Villanueva, and R. Cabeza, “Stereo matching using gradient similarity and locally adaptive supportweight,” Pattern Recognition Letters, vol. 32, no. 13, pp. 1643–1651, 2011.
[70]I.L. Jung, J.Y. Sim, C.S. Kim, and S.U. Lee, “Robust stereo matching under radiometric variations based on cumulative distributions of gradients,” in2013 IEEE International Conference on Image Processing, pp. 2082–2085, IEEE, 2013.
[71] A. Banno and K. Ikeuchi, “Disparity map refinement and 3d surface smoothing via directed anisotropic diffusion,” Computer Vision and Image Understanding, vol. 115, no. 5, pp. 611–619, 2011.
[72]H. Liu, R. Wang, Y. Xia, and X. Zhang, “Improved cost computation and adaptive shape guided filter for local stereo matching of low texture stereo images,” Applied Sciences, vol. 10, no. 5, p. 1869, 2020.89
[73] J. Lim and S. Lee, “Patchmatchbased robust stereo matching under radiometric changes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 5, pp. 1203–1212, 2018.
[74] S. Katsigiannis, J. Scovell, N. Ramzan, L. Janowski, P. Corriveau, M. A. Saad, and G. Van Wallendael, “Interpreting mos scores, when can users see a difference? understanding user experience differences for photo quality,” Quality and User Experience, vol. 3, no. 1, pp. 1–14, 2018.
[75] A. Hore and D. Ziou, “Image quality metrics: Psnr vs. ssim,” in 2010 20th International Conference on Pattern Recognition, pp. 2366–2369, IEEE, 2010.
[76] L. DeMaeztu, S. Mattoccia, A. Villanueva, and R. Cabeza, “Linear stereo matching,” in2011 International Conference on Computer Vision, pp. 1708–1715, IEEE,2011.[77] K.H. Thung and P. Raveendran, “A survey of image quality measures,” in 2009 International Conference for Technical Postgraduates (TECHPOS), pp. 1–4, IEEE,2009.
[78] Z. Wang and A. C. Bovik, “A universal image quality index,” IEEEsignalprocessingletters, vol. 9, no. 3, pp. 81–84, 2002.90