在拍攝有霧霾的影像時，常常因為霧霾及光源分布不均勻的地方導致整體影像失真及產生色彩偏差。在這些情況下，對於傳統的除霧演算法來說會受限於白天及黑夜的大氣光及暗通道假設之理想模型不同，進而影響到傳輸率的估測結果，導致在使用傳統演算去除霧霾時會過度除霧，造成雜訊放大的現象以及人工偽影的產生。因此本論文提出一個使用深度學習之除霧演算法，透過修改密集連結神經網絡的架構，並結合不同的損失函數，其中包括平方差損失函數、絕對誤差損失函數、感知損失函數，及總變差損失函數來達成無霧霾影像重建的目的，其中由於深度密集連接網絡架構具有特徵圖重用的特點，因此我們可以在網絡深度加深的情況下，反而使用較少的參數，藉此來穩定整體網路訓練的過程。
相較於傳統及深度學習的其他除霧演算法，需先具備輸入影像的深度資訊或是針對輸入影像估測深度抑或是傳輸率資訊，本文僅需對有霧霾及無霧霾的影像進行監督式訓練，本文亦開發了一種有效的深度學習架構，結合深度密集連接網絡的概念，可以提取霧霾影像的重要特徵並將這些重要特徵重建。此外，我們使用了混合式的損失函數，結合其優點選擇適當權重，以增強重建圖像的紋理和細節。最後我們在實驗結果證明，基於深度學習的解決方案明顯優於以往的方法，並且發現是一種可行且更具發展性的方法。

Image acquisition in a bad weather often results in visible distortions and color imbalance due to haze and imbalance light sources. In this scenario, the conventional dehazing algorithms try to estimate the transmission map or image prior. However, the condition of the transmission map is altered during different weather conditions and time. Consequently, the removal of haze with conventional strategies exist many limitations, and it needs improvements. In this thesis, a dehazing algorithm using the deep learning approach is proposed and the architecture is based on the U-Net. The model comprises of dense block networks, and incorporates different loss functions such as L1, L2, perceptual and total-variation. Dense Block is able to reuse feature map, and thus parameters can be reduced and also the network is more stable during training time.
Compared with former dehazing algorithms, the input depth map is required to be estimated to acquire transmission rate map. Conversely, the proposed method only needs image pairs of the haze and haze-free images by applying supervised learning. This study also developed an effective deep learning architecture, combined with the concept of Deep DenseNet, which can extract important features of smog images and reconstruct these important features. In addition, we use a hybrid loss function combining their advantages, and then pick the appropriate weights to enhance the texture and detail of the reconstructed image. Finally, experimental results show that the deep learning based solution is significantly superior to the previous methods, and thus it can be a very practical method to address dehazing.

[1] S. Narasimhan and S. Nayar, "Contrast restoration of weather degraded images", in IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 25, no. 6, pp. 713-724, 2003.
[2] S. K. Nayar and S. G. Narasimhan, "Vision in bad weather," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), vol. 2. Sep. 1999, pp. 820–827.
[3] Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, "Instant dehazing of images using polarization," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2001, pp. I-325–I-332.
[4] S. Shwartz, E. Namer, and Y. Y. Schechner, " Blind haze separation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), vol. 2. 2006, pp. 1984–1991.
[5] S. G. Narasimhan and S. K. Nayar, " Interactive (de) weathering of an image using physical models," in Proc. IEEE Workshop Color Photometric Methods Comput. Vis., vol. 6. France, 2003, p. 1.
[6] N. Hautiere, J. Tarel, and D. Aubert, " Towards fog -free in-vehicle vision systems through contrast restoration," Proc. IEEE Conf. Computer Vision, Pattern Recognition, pp. 1-8, June 2007.
[7] H. Koschmider, "Theorie der horizontalen Sichtweite," in Proc. Beiträgezur Phys. der Freien Atmos., 1924, pp. 171–181
[8] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," Proc. of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[9] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” advances in Neural Information Processing Systems (NIPS), 2012.
[10] V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” Inter. Conf. on Machine Learning, pp. 807-814, Haifa, Israel, 21-24, Jun. 2010
[11] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” IEEE Inter. Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016
[12] Long, J., Shelhamer, E., & Darrell, T. “Fully convolutional networks for semantic segmentation.” Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431-3440.
[13] O. Ronneberger and P.Fischer and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation”, Medical Image Computing and Computer-Assisted Intervention (MICCAI), vol 9351, pp. 234-241, 2015
[14] M. Drozdzal, E. Vorontsov, G. Chartrand, S. Kadoury, and C. Pal, “The Importance of Skip Connections in Biomedical Image Segmentation”, Workshop on Deep Learning in Medical Image Analysis (DLMIA), 2016
[15] CS231n: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture9.pdf
[16] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2015
[17] G. Huang, Z. Liu, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2017.
[18] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C. and Ghemawat, S. “Tensorflow: Large-scale machine learning on heterogeneous distributed systems.”, arXiv preprint arXiv:1603.04467, 2016.
[19] Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., and Darrell, T., “Caffe: Convolutional architecture for fast feature embedding.”, Proc. of the 22nd ACM international conference on Multimedia, ACM, pp. 675-678, 2014
[20] Zeiler, M. D., Krishnan, D., Taylor, G. W., & Fergus, R., “Convolutional networks.” in Computer Vision and Pattern Recognition (CVPR), pp. 2528-2535, 2010.
[21] K. He, J. Sun, and X. Tang, "Single Image Haze Removal Using Dark Channel Prior," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 2341–2353, Dec. 2011
[22] B. Cai, X. Xu, K. Jia, C. Qing, and D. Tao. “Dehazenet: An end-to-end system for single image haze removal.”, IEEE Trans. on Image Processing, vol. 25, no. 11, pp. 5187-5198, 2016.
[23] D. Yang, J. Sun, “Proximal Dehaze-Net: A Prior Learning-Based Deep Network for Single Image Dehazing.”, Proc. European Conf. Computer Vision(ECCV), pp 729-746, 2018
[24] W. Ren, S. Liu, H. Zhang, X. C. J. Pan, and M.-H. Yang. “Single image dehazing via multi-scale convolutional neural networks.”, Proc. European Conf. Computer Vision(ECCV), 2016.
[25] C. Ancuti and C. Ancuti., “Single image dehazing by multi-scale fusion.”, in IEEE Trans. on Image Processing, vol. 22, no. 8, pp. 3271-3282, 2013.
[26] C. Ancuti, C. O. Ancuti, A. Bovik, and C. De Vleeschouwer., “Night time dehazing by fusion.”, IEEE Inter. Conf. on Image Processing(ICIP), pp. 2256-2260, 2016.
[27] R. Fattal., “Single image dehazing.”, SIGGRAPH, 2008.
[28] G. Meng, Y. Wang, J. Duan, S. Xiang, and C. Pan., “Efficient image dehazing with boundary constraint and contextual regularization.”, in IEEE Inter. Conf. on Computer Vision, pp. 617-624, 2013.
[29] K. Nishino, L. Kratz, and S. Lombardi., “Bayesian defogging.”, Inter. Jour. on Computer Vision, 2012.
[30] R. T. Tan., “Visibility in bad weather from a single image.”, IEEE Inter. Conf. on Computer Vision and Pattern Recognition(CVPR), 2008.
[31] K. Tang, J. Yang, and J. Wang., “Investigating haze-relevant features in a learning framework for image dehazing.”, IEEE Inter. Conf.on Computer Vision and Pattern Recognition(CVPR), 2014.
[32] J.-P. Tarel and N. Hautiere., “Fast visibility restoration from a single color or gray level image.”, IEEE Inter. Conf. on Computer Vision(ICCV), 2009.
[33] A. Mittal, A. K. Moorthy, and A. C. Bovik., “No-reference image quality assessment in the spatial domain.” IEEE Trans. on Image Processing, 2012.
[34] J.-P. Tarel, N. Hautire, L. Caraffa, A. Cord, H. Halmaoui, and D. Gruyer., “Vision enhancement in homogeneous and heterogeneous fog.” IEEE Intelligent Transportation Systems Magazine, 2012
[35] J. E. Khoury(B), J.-B. Thomas, and A. Mansouri., “A color image database for haze model and dehazing methods evaluation.”, Inter. Conf. on Image and Signal Processing, 2016
[36] Li, B., Peng, X., Wang, Z., Xu, J., Feng, D., “Aod-net: All-in-one dehazing network.”, IEEE Inter. Conf. on Computer Vision(ICCV), 2017.
[37] Li, R., Pan, J., Li, Z., Tang, J., “Single image dehazing via conditional generative adversarial network.”, IEEE Inter. Conf. on Computer Vision and Pattern Recognition(CVPR), 2018.
[38] Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., Yang, M.H., “Gated fusion network for single image dehazing.”, IEEE Inter. Conf. on Computer Vision and Pattern Recognition(CVPR), 2018.
[39] Zhang, H., Patel, V.M., “Densely connected pyramid dehazing network.”, IEEE Inter. Conf. on Computer Vision and Pattern Recognition(CVPR), 2018.
[40] B. Li et al., “Benchmarking Single-Image Dehazing and Beyond.”, IEEE Transactions on Image Processing, vol. 28, no. 1, pp. 492-505, 2019.
[41] J. Bruna, P. Sprechmann, and Y. LeCun., “Superresolution with deep convolutional sufficient statistics.”, IEEE Inter. Conf. on Learning Representations (ICLR), 2016.
[42] S. H. Khan, M. Bennamoun, F. Sohel and R. Togneri, “Automatic Feature Learning for Robust Shadow Detection.”, IEEE Inter. Conf. on Computer Vision and Pattern Recognition(CVPR), 2014, pp. 1939-1946.
[43] H. A. Aly and E. Dubois, “Image up-sampling using total-variation regularization with a new observation model.”, IEEE Trans. on Image Processing, vol. 14, no. 10, pp. 1647-1659, 2005.
[44] D. P. Kingma and J. Ba. Adam, “A method for stochastic optimization.”, arXiv preprint arXiv:1412.6980v9, 2014.
[45] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for realtime style transfer and super-resolution.”, European Conf. on Computer Vision(ECCV). pp. 694–711, 2016.
[46] Simonyan, K., & Zisserman, A., “Very Deep Convolutional Networks for Large-Scale Image Recognition.”, arXiv preprint arXiv:1409.1556v6, 2015.
[47] R. Fattal, “Dehazing using color-lines.”, ACM Transactions on Graphics (TOG), vol. 34, no. 1, p. 13, 2014
[48] K. Ma, W. Liu, and Z. Wang, “Perceptual evaluation of single image dehazing algorithms.”, IEEE Inter. Conf. on Image Processing(ICIP), pp. 3600–604, 2015.
[49] C. Sakaridis, D. Dai, and L. Van Gool, “Semantic foggy scene understanding with synthetic data.”, arXiv preprint arXiv:1708.07819, 2017.
[50] Y. Zhang, L. Ding, and G. Sharma, “Hazerd: An outdoor scene dataset and benchmark for Single image dehazing.”, IEEE Inter. Conf. on Image Processing(ICIP), pp. 3205–3209, 2017.
[51] T. K. Kim, J. K. Paik, and B. S. Kang., “Contrast enhancement system using spatially adaptive histogram equalization with temporal filtering.”, IEEE Trans. on Consumer Electronics, vol. 44, no. 1, pp. 82–87, 1998.
[52] J. A. Stark., “Adaptive image contrast enhancement using generalizations of histogram equalization.”, IEEE Trans. on image processing, vol. 9, no. 5, pp. 889–896, 2000.
[53] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity.”, IEEE Trans. on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
[54] G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks.”, Science, vol. 313, no. 5786, pp. 504–507, 2006.
[55] V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines.”, in Inter. Conf. on Machine Learning (ICML), pp. 807–814, 2010
[56] M. Xia and TT. Wong, “Deep Inverse Halftoning via Progressively Residual Learning.”, Asian Conference on Computer Vision, (ACCV), vol. 11366, pp 523-539, 2019.
[57] P. Isola, J. Y. Zhu, T. H. Zhou, A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks.”, arXiv preprint arXiv:1611.07004v3, 2018
[58] S. J´egou, M. Drozdzal, D, Vazque et al., “The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation.”, arXiv preprint arXiv: 1611.09326v3, 2017.
[59] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting.”, Jour. of Machine Learning Research, vol. 15, pp1929-1958, 2014.
[60] H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss Functions for Image Restoration with Neural Networks.”, in IEEE Trans. on Computational Imaging, vol. 3, no. 1, 2017.
[61] D. Engin, A. Genc and H. K. Ekenel, “Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing,” IEEE Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 938-9388, 2018.
[62] J. Guo, J. Syue, V. R. Radzicki and H. Lee, “An Efficient Fusion-Based Defogging,” in IEEE Trans. on Image Processing, vol. 26, no. 9, pp. 4217-4228, 2017.
[63] A. Mittal, A. K. Moorthy and A. C. Bovik, "No-Reference Image Quality Assessment in the Spatial Domain," in IEEE Trans. on Image Processing, vol. 21, no. 12, pp. 4695-4708, Dec. 2012.