研究生: |
楊為城 Wei-Cheng Yang |
---|---|
論文名稱: |
基於生成對抗網路的夜間道路圖像白晝化轉換 Night-to-Day Road Image Tranformation Based on Generative Adversarial Network |
指導教授: |
孫沛立
Pei-Li Sun |
口試委員: |
林宗翰
Tzung-han Lin 胡國瑞 Kuo-Jui Hu 陳怡永 Yi-Yung Chen |
學位類別: |
碩士 Master |
系所名稱: |
應用科技學院 - 色彩與照明科技研究所 Graduate Institute of Color and Illumination Technology |
論文出版年: | 2021 |
畢業學年度: | 109 |
語文別: | 中文 |
論文頁數: | 47 |
中文關鍵詞: | 深度學習 、生成對抗網路 、圖像到圖像轉換 、電腦視覺 |
外文關鍵詞: | Deep Learning, Generative Adversarial Networks, Image-to-image Transformation, Computer Vision |
相關次數: | 點閱:367 下載:2 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
「圖像到圖像轉換」是一種將圖像從原始場景轉換至目標場景的下,並能夠保留原始圖像内容的技術。這些年來,由於該技術在電腦視覺、圖像分割、風格轉換等問題上的廣泛應用,引起越來越多的關注。而在自動駕駛場景中,日夜轉換的是個很重要的應用領域:因爲夜間圖像的對比度與能見度低,導致對於夜間檢測性能的下降。
現今圖像轉換的做法主要是基於GAN(Generative Adversarial Nets,生成對抗網路)的架構,GAN可以通過監督式學習或非監督式學習進行訓練。如果使用監督式學習來對夜間圖像進行轉換,主要的問題在於缺少夜間和白天圖像的配對數據。而非監督學習不需要配對數據,可以減少對數據的依賴。
本研究提出了一個基於GAN的圖像轉換系統,將夜間道路圖像轉換至白天場景。主要利用亮度增强網路和U-Net的架構來保留夜間圖像的顔色和細節,並使用一系列的損失函數來協助網路的訓練。實驗結果優於經典的ToDayGAN。
"Image-to-image Transformation" is a CNN-based deep learning method that converts an image from the original scene to the target scene and retains the original image content. The method gains more and more attention as it can be applied to many different applications such as computer vision, image segmentation and image style transfer. In an advanced driver assistance system, night-to-day transformation becomes a very important one as the contrast and visibility of road images at night are low which reduces the performance of the automatic car driving.
Today, image-to-image transformation is mainly based on the architecture of GAN (Generative Adversarial Networks). GANs can be trained by either supervised learning or unsupervised learning. If you use the supervised learning to transform the night images, the main problem is the lack of great amount of day and night road-image pairs. However, the unsupervised learning doesn’t need such paired image bank.
This research proposes a GAN-based image conversion system, which converts night road-images to their daytime scenes. It uses a brightness boosting network to enhance the image contrast, a U-Net to preserve the color and details of the night images, and a series of loss functions to help the network training. The experimental results are better than the classic ToDayGAN.
[1] C.Guo et al., “Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1780-1789, 2020 [Online]. Available: https://li-chongyi.github.io/Proj_Zero-DCE.html/.
[2] G.Buchsbaum, “A Spatial Processor Model for Object Colour Perception,” J. Franklin Inst., vol. 310, no. 1, 1980, doi: 10.1016/0016-0032(80)90058-7.
[3] I. J.Goodfellow et al., “Generative Adversarial Networks,” 2014, [Online]. Available: http://arxiv.org/abs/1406.2661.
[4] J.-Y.Zhu, T.Park, P.Isola, andA. A.Efros, “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks,” Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2223-2232, 2017, [Online]. Available: http://arxiv.org/abs/1703.10593.
[5] X.Mao, Q.Li, H.Xie, R. Y. K.Lau, Z.Wang, andS. P.Smolley, “Least Squares Generative Adversarial Networks,” Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2794-2802, 2017, [Online]. Available: http://arxiv.org/abs/1611.04076.
[6] X.Huang andS.Belongie, “Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization,” Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1501-1510, 2017 [Online]. Available: http://arxiv.org/abs/1703.06868.
[7] L.Mescheder, A.Geiger, andS.Nowozin, “Which Training Methods for GANs do actually Converge?,” Proceedings of the 35 th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018, [Online]. Available: http://arxiv.org/abs/1801.04406.
[8] A.Anoosheh, T.Sattler, R.Timofte, M.Pollefeys, andL.VanGool, “Night-to-Day Image Translation for Retrieval-based Localization.” 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 5958-5964, 2019.
[9] A.Jolicoeur-Martineau, “The Relativistic Discriminator: A Key Element Missing from Standard GAN,” ICLR 2019 Conference, 2019, [Online]. Available: http://arxiv.org/abs/1807.00734.
[10] T.-C.Wang, M.-Y.Liu, J.-Y.Zhu, A.Tao, J.Kautz, andB.Catanzaro, “High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8798-8807, 2018, [Online]. Available: http://arxiv.org/abs/1711.11585.
[11] P.Isola, J.-Y.Zhu, T.Zhou, andA. A.Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” CVPR 2017 , [Online]. Available: http://arxiv.org/abs/1611.07004.
[12] M.Liu et al., “Few-Shot Unsupervised Image-to-Image Translation.” Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10551-10560, 2019.
[13] I.Anokhin et al., “High-Resolution Daytime Translation Without Domain Labels,” CVPR 2020, [Online]. Available: http://arxiv.org/abs/2003.08791.
[14] F.Yu et al., “BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020, [Online]. Available: http://arxiv.org/abs/1805.04687.
[15] M.Heusel, H.Ramsauer, T.Unterthiner, B.Nessler, andS.Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” 31st Conference on Neural Information Processing Systems (NIPS 2017), 2017, [Online]. Available: http://arxiv.org/abs/1706.08500.
[16] Thalles Silva, “An intuitive introduction to Generative Adversarial Networks (GANs), ”[Online]. Available: https://www.freecodecamp.org/news/an-intuitive-introduction-to-generative-adversarial-networks-gans-7a2264a81394/.