近幾年深度學習隨著硬體的發展和DCNN(Deep convolutional neural network)的技術突破，讓許多傳統複雜的影像處理方法，逐漸被深度學習取代。運用深度學習技術於圖片增強(Image enhancement)是目前重要的研究方法之一。深度學習模型可以透過大量圖片對圖片的學習，讓網路自己找到一個可以去除雜訊的函數，來提升整張圖片的品質，達到圖片除霧、除雨、去模糊以及本篇論文所研究的圖片超解析的目的。
在各種模型架構中，殘差網路(Residual network)是最適合使用在圖片超解析的應用上。我們提出一個基於殘差網路的輕量化圖片超解析模型架構，和使用post-upsampling的超解析策略，減少整體模型的運算參數量，並且在實驗結果中也超越了在NTIRE 2017 Super-Resolution這個影像增強比賽中圖片超解析項目的最高分數。同時也將本論文所提出的模型應用在人臉辨識系統上，透過使用不同超解析放大尺度的圖片對人臉辨識系統做準確率之實驗，得知不同尺度對人臉辨識系統的影響，同時也在訓練模型的過程中，觀察到不同的子像素卷積(Sub-pixel convolution)操作對整個模型在訓練驗證時對預測圖片的影響。

In recent years, the development of hardware and the technological breakthrough of DCNN (Deep Convolution Neural Network) have flourished. Deep learning has gradually replaced many traditional complex image processing methods. The application of deep learning in image enhancement is one of major researches. We can train the network model by a large number of image pairs, so that the network model can find a function to remove the noise, the haze, the rain and the blur of image and also do image super-resolution as proposed in this research.
Among the various model architectures, the residual architecture is the most suitable for the image super-resolution application. We propose a lightweight model based on residual network and we follow the post-upsampling strategy to design the network for reducing the overall parameters of the model. Experimental results show that our method has the higher score than that of NTIRE 2017 Super-Resolution workshop. At the same time, we apply our model in the face recognition system, and get the results from using multi-scale models to test the origin pretrain face recognition system. It let us know the influence in using multi-scale models. Moreover, while training the super-resolution model, we observe about the effect of Sub-pixel convolution operations from the prediction images

[1]R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, and L. Zhang, "Ntire 2017 challenge on single image super-resolution: Methods and results," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 114-125.
[2]E. Agustsson and R. Timofte, "Ntire 2017 challenge on single image super-resolution: Dataset and study," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 126-135.
[3]C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 295-307, 2015.
[4]K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
[5]C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.
[6]C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818-2826.
[7]C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, "Inception-v4, inception-resnet and the impact of residual connections on learning," in Thirty-first AAAI conference on artificial intelligence, 2017.
[8]K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[9]G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700-4708.
[10]C. Dong, C. C. Loy, and X. Tang, "Accelerating the super-resolution convolutional neural network," in European conference on computer vision, 2016: Springer, pp. 391-407.
[11]W. Shi et al., "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1874-1883.
[12]J. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1646-1654.
[13]C. Ledig et al., "Photo-realistic single image super-resolution using a generative adversarial network," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4681-4690.
[14]Z. Wang, J. Chen, and S. C. Hoi, "Deep learning for image super-resolution: A survey," IEEE Trans. Pattern Anal. Mach. Intell., 2020.
[15]J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431-3440.
[16]F. Schroff, D. Kalenichenko, and J. Philbin, "Facenet: A unified embedding for face recognition and clustering," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 815-823.
[17]OpenCV, "Cascade Classifier", https://docs.opencv.org/4.5.2/db/d28/tutorial_cascade_classifier.html
[18]K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, "Joint face detection and alignment using multitask cascaded convolutional networks," IEEE Signal Process Lett., vol. 23, no. 10, pp. 1499-1503, 2016.
[19]J. Yu et al., "Wide activation for efficient and accurate image super-resolution," arXiv preprint arXiv:1808.08718, 2018.
[20]F. Chollet, "Xception: Deep learning with depthwise separable convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251-1258.
[21]J.-B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 5197-5206.
[22]Berkeley University of California Computer Vision Group, https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/
[23]Labeled Faces in the Wild, http://vis-www.cs.umass.edu/lfw/index.html
[24]Tensorflow API Documentation, https://www.tensorflow.org/versions/r2.1/api_docs
[25]H. Zhao, O. Gallo, I. Frosio, and J. Kautz, "Loss functions for image restoration with neural networks," IEEE Trans. Comput. Imaging, vol. 3, no. 1, pp. 47-57, 2016.
[26]B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, "Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 136-144.
[27]Facenet pretrain model, https://github.com/nyoki-mtl/keras-facenet