Restoration of Underwater Images using Multi-Color Spaces Encoder and Adaptive Tone Mapping Enhancement

簡易檢索 / 詳目顯示

回結果列表

研究生：	何泰山 Ha Thai Son
論文名稱：	Restoration of Underwater Images using Multi-Color Spaces Encoder and Adaptive Tone Mapping Enhancement Restoration of Underwater Images using Multi-Color Spaces Encoder and Adaptive Tone Mapping Enhancement
指導教授：	胡國瑞 Kuo-Jui Hu
口試委員:	高聖龍 Sheng-Long Kao 孫沛立 Pei-Li Sun 胡國瑞 Kuo-Jui Hu
學位類別：	碩士 Master
系所名稱：	應用科技學院 - 色彩與照明科技研究所 Graduate Institute of Color and Illumination Technology
論文出版年：	2022
畢業學年度：	110
語文別：	英文
論文頁數：	87
中文關鍵詞：	underwater images 、image dehazing 、deep learning 、adaptive-tone mapping
外文關鍵詞：	underwater images, image dehazing, deep learning, adaptive-tone mapping
相關次數：	點閱：249 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

上一筆

The task of restoring and enhancing underwater images is particularly challenging
due to the physical and chemical properties of underwater environments. Issues such as green-bluish color cast and background light scattering make images taken underwater often have unnatural colors, low dynamic range, or blurry details. Inspired by some ofthe latest restoration methods, our approach to this problem makes use of two components:one supervised encoder-decoder network called Ucolor-improve aimed to learn the image degrading model and one enhancement module to further improve the dynamic range of the images. In the network, an input image is encoded in RGB, CIELAB and Y’CbCr color spaces to enrich feature representations, after which the most contributive features are highlighted using the attention mechanism. The network is further guided towards more severely degraded regions of the images with the help of medium transmission maps.Perceptual loss is also used with the per-pixel loss for optimization to highlight the difference in high-level semantic features, of which the impact is analyzed in the thesis as well. Objective and subjective comparisons indicate that Ucolor-improve produced relatively good results in the Underwater Image Enhancement Benchmark (UIEB) dataset as well as in challenging cases when compared to other methods. As for the enhancement module, adaptive tone-mapping is used in combination with bilateral filters. As a result,the enhanced output produces a gamut with a larger volume and is more evenly distributed when visualized in the CIELAB color space showing the improvement in dynamic range and contrast while still managing to avoid over-enhancement and distortion.

ABSTRACT ..................................................................................................................... I
ACKNOWLEDGMENT ............................................................................................... II
LIST OF TABLES ......................................................................................................... V
LIST OF FIGURES ..................................................................................................... VI
INTRODUCTION ...................................................................................................... 1
1 RESTORATION AND ENHANCEMENT OF UNDERWATER IMAGING ....................................................... 1
2 OBJECTIVE OF THESIS ........................................................................................................................ 5
3 ORGANIZATION OF THESIS ................................................................................................................. 6
LITERARY REVIEW ................................................................................................ 7
1 HARDWARE-BASED METHODS ............................................................................................................ 8
2 CONVENTIONAL IMPROVEMENT METHODS ......................................................................................... 9
2.1 Underwater image enhancement methods ................................................................................. 9
2.2 Underwater Image Restoration Methods ................................................................................ 10
3 DEEP LEARNING-BASED METHODS ................................................................................................... 12
BACKGROUND ....................................................................................................... 15
1 IMAGE FORMATION MODEL ............................................................................................................. 15
1.1 Definition ................................................................................................................................. 15
1.2 Transmission Estimation using Dark Channel Prior .............................................................. 16
1.3 Generalization of the Dark Channel Prior .............................................................................. 18
2 COLOR SPACES ................................................................................................................................. 21
2.1 RGB ......................................................................................................................................... 21
2.2 CIELAB ................................................................................................................................... 22
2.3 Y’CbCr .................................................................................................................................... 23
2.4 HSV ......................................................................................................................................... 24
3 DEEP LEARNING IN COMPUTER VISION ............................................................................................ 27
3.1 Convolutional Neural Network................................................................................................ 27
3.2 Residual Enhancement Network .............................................................................................. 33
3.3 Squeeze-and-Excitation module .............................................................................................. 34
3.4 Perceptual Loss ....................................................................................................................... 36
PROPOSED METHODS ......................................................................................... 39
1 NETWORK ARCHITECTURE ............................................................................................................... 40
IV
1.1 Multi-color spaces encoding module ....................................................................................... 42
1.2 Residual-learning module: ...................................................................................................... 42
1.3 Channel attention module ....................................................................................................... 43
1.4 Medium Transmission Guidance module: ............................................................................... 43
1.5 Loss function ........................................................................................................................... 44
2 ENHANCEMENT USING ROBUST ADAPTIVE TONE MAPPING ............................................................... 46
EXPERIMENTAL RESULTS................................................................................. 51
1 TRAINING PROCESS .......................................................................................................................... 51
1.1 Training Datasets .................................................................................................................... 51
1.2 Choice of hyperparameters and implementation platform ...................................................... 54
2 EXPERIMENT SETTINGS .................................................................................................................... 58
2.1 Experiments Dataset................................................................................................................ 58
2.2 Compared methods .................................................................................................................. 58
2.3 Evaluation Metrics .................................................................................................................. 59
3 QUALITATIVE EVALUATION ............................................................................................................. 60
4 QUANTITATIVE EVALUATION........................................................................................................... 63
5 FAILURE CASES ................................................................................................................................ 64
6 ABLATION STUDY ............................................................................................................................ 67
CONCLUSIONS AND DISCUSSIONS .................................................................. 69
1 CONCLUSIONS .................................................................................................................................. 69
2 DISCUSSIONS AND FUTURE RESEARCH ............................................................................................. 70
REFERENCES .............................................................................................................. 72
                                

[1] Y. Wang, W. Song, G. Fortino, L.-Z. Qi, W. Zhang, and A. Liotta, "An
Experimental-Based Review of Image Enhancement and Image Restoration
Methods for Underwater Imaging," IEEE Access, vol. 7, pp. 140233-140251,
2019, doi: 10.1109/access.2019.2932130.
[2] C. Tan, A. Sluzek, G. S. GL, and T. Jiang, "Range gated imaging system for
underwater robotic vehicle," in OCEANS 2006-Asia Pacific, 2006: IEEE, pp. 1-
6.
[3] T. Treibitz and Y. Y. Schechner, "Active polarization descattering," IEEE
Trans. Pattern Anal. Mach. Intell., vol. 31, no. 3, pp. 385-399, 2008.
[4] C. O. Ancuti, C. Ancuti, C. De Vleeschouwer, and P. Bekaert, "Color Balance
and Fusion for Underwater Image Enhancement," (in English), Ieee T Image
Process, vol. 27, no. 1, pp. 379-393, Jan 2018, doi: 10.1109/Tip.2017.2759252.
[5] P. L. Drews, E. R. Nascimento, S. S. Botelho, and M. F. M. Campos,
"Underwater depth estimation and image restoration based on single images,"
IEEE computer graphics and applications, vol. 36, no. 2, pp. 24-35, 2016.
[6] A. Galdran, D. Pardo, A. Picón, and A. Alvarez-Gila, "Automatic red-channel
underwater image restoration," Journal of Visual Communication and Image
Representation, vol. 26, pp. 132-145, 2015.
[7] Y.-T. Peng and P. C. Cosman, "Underwater image restoration based on image
blurriness and light absorption," Ieee T Image Process, vol. 26, no. 4, pp. 1579-
1594, 2017. [Online]. Available: https://ieeexplore.ieee.org/document/7840002/.
[8] J. Li, K. A. Skinner, R. M. Eustice, and M. Johnson-Roberson, "WaterGAN:
Unsupervised Generative Network to Enable Real-time Color Correction of
Monocular Underwater Images," IEEE Robotics and Automation Letters, pp. 1-
1, 2017, doi: 10.1109/lra.2017.2730363.
[9] Y. Guo, H. Li, and P. Zhuang, "Underwater image enhancement using a
multiscale dense generative adversarial network," IEEE Journal of Oceanic
Engineering, vol. 45, no. 3, pp. 862-870, 2019.
[10] C. Li, S. Anwar, and F. Porikli, "Underwater scene prior inspired deep
underwater image and video enhancement," Pattern Recognition, vol. 98, p.
107038, 2020.
[11] C. Li et al., "An Underwater Image Enhancement Benchmark Dataset and
Beyond," IEEE Trans Image Process, Nov 28 2019, doi:
10.1109/TIP.2019.2955241.
[12] C. Li, S. Anwar, J. Hou, R. Cong, C. Guo, and W. Ren, "Underwater Image
Enhancement via Medium Transmission-Guided Multi-Color Space
Embedding," IEEE Trans Image Process, vol. 30, pp. 4985-5000, 2021, doi:
10.1109/TIP.2021.3076367.
[13] J. Johnson, A. Alahi, and L. Fei-Fei, "Perceptual losses for real-time style
transfer and super-resolution," in European conference on computer vision,
2016: Springer, pp. 694-711.
[14] K.-J. Hu, M.-Y. Lu, J.-C. Wang, T.-I. Hsu, and T.-T. Chang, "Using adaptive
tone mapping to enhance edge-preserving color image automatically," EURASIP
Journal on Image and Video Processing, vol. 2010, pp. 1-11, 2010.
[15] N. Otsu, "A threshold selection method from gray-level histograms," IEEE
73
transactions on systems, man, and cybernetics, vol. 9, no. 1, pp. 62-66, 1979.
[16] Y. Y. Schechner and N. Karpel, "Recovery of underwater visibility and structure
by polarization analysis," IEEE Journal of oceanic engineering, vol. 30, no. 3,
pp. 570-587, 2005.
[17] C. O. Ancuti, C. Ancuti, C. De Vleeschouwer, and P. Bekaert, "Color balance
and fusion for underwater image enhancement," Ieee T Image Process, vol. 27,
no. 1, pp. 379-393, 2017. [Online]. Available:
https://ieeexplore.ieee.org/ielx7/83/8071125/08058463.pdf?tp=&arnumber=805
8463&isnumber=8071125&ref=.
[18] K. He, J. Sun, and X. Tang, "Single image haze removal using dark channel
prior," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 2341-2353,
2010. [Online]. Available: https://ieeexplore.ieee.org/document/5567108/.
[19] N. Carlevaris-Bianco, A. Mohan, and R. M. Eustice, "Initial results in
underwater single image dehazing," in Oceans 2010 Mts/IEEE Seattle, 2010:
IEEE, pp. 1-8.
[20] C. Li, J. Guo, and C. Guo, "Emerging from water: Underwater image color
correction based on weakly supervised color transfer," IEEE Signal processing
letters, vol. 25, no. 3, pp. 323-327, 2018.
[21] H. Li, X. Wang, T. Bai, W. Jin, Y. Huang, and K. Ding, "Speckle noise
suppression of range gated underwater imaging system," in Applications of
Digital Image Processing XXXII, 2009, vol. 7443: International Society for
Optics and Photonics, p. 74432A.
[22] M. Roser, M. Dunbabin, and A. Geiger, "Simultaneous underwater visibility
assessment, enhancement and improved stereo," in 2014 IEEE International
Conference on Robotics and Automation (ICRA), 2014: IEEE, pp. 3840-3847.
[23] A. Galdran, "Image dehazing by artificial multiple-exposure image fusion,"
Signal Processing, vol. 149, pp. 135-147, 2018.
[24] Y. Li, H. Lu, K.-C. Li, H. Kim, and S. Serikawa, "Non-uniform de-scattering
and de-blurring of underwater images," Mobile Networks and Applications, vol.
23, no. 2, pp. 352-362, 2018.
[25] R. Hummel, "Image enhancement by histogram transformation," Unknown,
1975.
[26] C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert, "Enhancing underwater
images and videos by fusion," in 2012 IEEE conference on computer vision and
pattern recognition, 2012: IEEE, pp. 81-88.
[27] L. A. Torres-Méndez and G. Dudek, "Color correction of underwater images for
aquatic robot inspection," in International Workshop on Energy Minimization
Methods in Computer Vision and Pattern Recognition, 2005: Springer, pp. 60-
73.
[28] K. Iqbal, R. A. Salam, A. Osman, and A. Z. Talib, "Underwater Image
Enhancement Using an Integrated Colour Model," IAENG International Journal
of computer science, vol. 34, no. 2, 2007.
[29] M. S. Hitam, E. A. Awalludin, W. N. J. H. W. Yussof, and Z. Bachok, "Mixture
contrast limited adaptive histogram equalization for underwater image
enhancement," in 2013 International conference on computer applications
technology (ICCAT), 2013: IEEE, pp. 1-5.
[30] D. Huang, Y. Wang, W. Song, J. Sequeira, and S. Mavromatis, "Shallow-water
image enhancement using relative global histogram stretching based on adaptive
parameter acquisition," in International conference on multimedia modeling,
2018: Springer, pp. 453-465.
74
[31] X. Fu, P. Zhuang, Y. Huang, Y. Liao, X.-P. Zhang, and X. Ding, "A retinexbased
enhancing approach for single underwater image," in 2014 IEEE
International Conference on Image Processing (ICIP), 2014: IEEE, pp. 4572-
4576.
[32] H. Lu, Y. Li, L. Zhang, and S. Serikawa, "Contrast enhancement for images in
turbid water," JOSA A, vol. 32, no. 5, pp. 886-893, 2015.
[33] C. Li, J. Quo, Y. Pang, S. Chen, and J. Wang, "Single underwater image
restoration by blue-green channels dehazing and red channel correction," in
2016 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP), 2016: IEEE, pp. 1731-1735.
[34] C.-Y. Li, J.-C. Guo, R.-M. Cong, Y.-W. Pang, and B. Wang, "Underwater
image enhancement by dehazing with minimum information loss and histogram
distribution prior," Ieee T Image Process, vol. 25, no. 12, pp. 5664-5677, 2016.
[35] Y.-T. Peng, X. Zhao, and P. C. Cosman, "Single underwater image enhancement
using depth estimation based on blurriness," in 2015 IEEE International
Conference on Image Processing (ICIP), 2015: IEEE, pp. 4952-4956.
[36] Y. T. Peng, K. Cao, and P. C. Cosman, "Generalization of the Dark Channel
Prior for Single Image Restoration," IEEE Trans Image Process, vol. 27, no. 6,
pp. 2856-2868, Jun 2018, doi: 10.1109/TIP.2018.2813092.
[37] M. Yang, K. Hu, Y. Du, Z. Wei, Z. Sheng, and J. Hu, "Underwater image
enhancement based on conditional generative adversarial network," Signal
Processing: Image Communication, vol. 81, p. 115723, 2020.
[38] https://homepages.inf.ed.ac.uk/rbf/Fish4Knowledge/ (accessed May 25, 2022).
[39] J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba, "Sun database:
Large-scale scene recognition from abbey to zoo," in 2010 IEEE computer
society conference on computer vision and pattern recognition, 2010: IEEE, pp.
3485-3492.
[40] "MARIS - Marine Radioactivity Information System."
https://maris.iaea.org/datasets (accessed May 25, 2022).
[41] A. Duarte, F. Codevilla, J. D. O. Gaya, and S. S. Botelho, "A dataset to evaluate
underwater image restoration methods," in OCEANS 2016-Shanghai, 2016:
IEEE, pp. 1-6.
[42] H. Blasinski, T. Lian, and J. Farrell, "Underwater image systems simulation," in
Imaging Systems and Applications, 2017: Optical Society of America, p. ITh3E.
3.
[43] K. He, J. Sun, and X. Tang, "Guided image filtering," IEEE Trans. Pattern
Anal. Mach. Intell., vol. 35, no. 6, pp. 1397-1409, 2012.
[44] X. Dong and J. Wen, "Low lighting image enhancement using local maximum
color value prior," Frontiers of Computer Science, vol. 10, no. 1, pp. 147-156,
2016.
[45] E. Reinhard, E. A. Khan, A. O. Akyuz, and G. Johnson, Color imaging:
fundamentals and applications. CRC Press, 2008.
[46] Wikipedia. "RGB to YCbCr conversion."
https://en.wikipedia.org/wiki/YCbCr#/media/File:CCD.png (accessed June 29,
2022).
[47] M. D. Fairchild, Color appearance models. John Wiley & Sons, 2013.
[48] Wikipedia. "HSL and HSV." Wikipedia.
https://en.wikipedia.org/wiki/HSL_and_HSV#Basic_principle (accessed.
[49] C. A. Brewer, "Color use guidelines for data representation," in Proceedings of
the Section on Statistical Graphics, American Statistical Association, 1999, pp.
75
55-60.
[50] C. Poynton, "Frequently asked questions about color," Retrieved June, vol. 19,
no. 449, p. 2004, 1997.
[51] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning
applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp.
2278-2324, 1998.
[52] K. Simonyan and A. Zisserman, "Very deep convolutional networks for largescale
image recognition," arXiv preprint arXiv:1409.1556, 2014.
[53] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with
deep convolutional neural networks," Advances in neural information
processing systems, vol. 25, 2012.
[54] C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the
IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.
[55] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image
recognition," in Proceedings of the IEEE conference on computer vision and
pattern recognition, 2016, pp. 770-778.
[56] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A
large-scale hierarchical image database," in 2009 IEEE conference on computer
vision and pattern recognition, 2009: Ieee, pp. 248-255.
[57] J. Hu, L. Shen, and G. Sun, "Squeeze-and-excitation networks," in Proceedings
of the IEEE conference on computer vision and pattern recognition, 2018, pp.
7132-7141.
[58] K. Simonyan and A. Zisserman, "Very deep convolutional networks for largescale
visual recognition," arXiv preprint arXiv:1409.1556, 2014.
[59] X. Fu, Z. Fan, M. Ling, Y. Huang, and X. Ding, "Two-step approach for single
underwater image enhancement," in 2017 international symposium on
intelligent signal processing and communication systems (ISPACS), 2017:
IEEE, pp. 789-794.
[60] C. Li, J. Guo, C. Guo, R. Cong, and J. Gong, "A hybrid method for underwater
image correction," Pattern Recognition Letters, vol. 94, pp. 62-67, 2017.
[61] W. Ren, S. Liu, H. Zhang, J. Pan, X. Cao, and M.-H. Yang, "Single image
dehazing via multi-scale convolutional neural networks," in European
conference on computer vision, 2016: Springer, pp. 154-169.
[62] https://apps.apple.com/us/app/dive-video-color-correction/id1251506403
(accessed February 22 2022.
[63] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, "Indoor segmentation and
support inference from rgbd images," in European conference on computer
vision, 2012: Springer, pp. 746-760.
[64] J. Y. Chiang and Y.-C. Chen, "Underwater image enhancement by wavelength
compensation and dehazing," Ieee T Image Process, vol. 21, no. 4, pp. 1756-
1769, 2011.
[65] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv
preprint arXiv:1412.6980, 2014.
[66] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality
assessment: from error visibility to structural similarity," IEEE Trans Image
Process, vol. 13, no. 4, pp. 600-12, Apr 2004, doi: 10.1109/tip.2003.819861.
[67] D. Berman, D. Levy, S. Avidan, and T. Treibitz, "Underwater single image
color restoration using haze-lines and a new quantitative dataset," IEEE Trans.
Pattern Anal. Mach. Intell., vol. 43, no. 8, pp. 2822-2837, 2020.

簡易檢索 / 詳目顯示

相關論文