人臉影像超解析：一個強化輪廓資訊之多重拉普拉斯金字塔生成對抗網路

簡易檢索 / 詳目顯示

回結果列表

研究生：	戈善磊 Shan-Lei Ko
論文名稱：	人臉影像超解析：一個強化輪廓資訊之多重拉普拉斯金字塔生成對抗網路 Face Super Resolution: a MultiLaplacian GAN with Edge Enhancement
指導教授：	戴碧如 Bi-Ru Dai
口試委員:	戴志華 Chih-Hua Tai 帥宏翰 Hong-Han Shuai 陳怡伶 Yi-Ling Chen
學位類別：	碩士 Master
系所名稱：	電資學院 - 資訊工程系 Department of Computer Science and Information Engineering
論文出版年：	2020
畢業學年度：	108
語文別：	英文
論文頁數：	47
中文關鍵詞：	影像超解析、深度學習、電腦視覺
外文關鍵詞：	Super Resolution, Deep Learning, Computer Vision
相關次數：	點閱：341 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

人臉影像超解析已成為影響處領域的熱門主題。如今，越來越多的研究增加了人臉特徵點，身份等其他信息，以從低解析度圖像中重建高分辨率圖像，並且在定量方面和感知質量上都有良好的表現。但是，在許多情況下很難獲得這些附加資訊。在本文中，我們專注於通過直接從圖像中提取有用的信息而不是使用附加資訊來重建人臉圖像。通過觀察人臉圖像各個尺度的輪廓訊息，我們提出了一種增強輪廓資訊而重建高分辨率人臉圖像的方法。此外，通過提出的訓練程序，我們的方法可以在8倍的放大倍數上重建極逼真的人臉圖像，並且在量與值方面均優於最新方法。

Face image super resolution has become a research hotspot in the field of image processing. Nowadays, more and more researches add additional information, such as landmark,identity, to reconstruct high resolution images from low resolution ones, and have a good performance in quantitative terms and perceptual quality. However, these additional information is hard to obtain in many cases. In this work, we focus on reconstructing face images by extracting useful information from images directly rather than using additional information. By observing edge information in each scale of face images, we propose a method to reconstruct high resolution face images with enhanced edge information. In additional, with the proposed training procedure, our method reconstructs photo realistic images in upscaling factor 8× and outperforms state-of-the-art methods both in quantitative terms.

Abstract in Chinese . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Abstract in English . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
List of Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1 Deep Learning for Image Super resolution
. . . . . . . . . . . . . . . . . 3
2.2 Face Image Super resolution
. . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 Edge Detection for Human Face . . . . . . . . . . . . . . . . . . . . . . 5
3 Proposed Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1 Generative Branch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Discriminative Branch . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Intertwined Training Procedure . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4.1 Upsampling Cell . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4.2 Extraction Cell . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4.3 Squeeze and Extraction Cell . . . . . . . . . . . . . . . . . . . . 14
4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.2 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.3 Ablation Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.4 The Convergence of Losses in CAMLGE . . . . . . . . . . . . . . . . . 20
4.5 Quantitative Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.6 Qualitative Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.7 Multi-scale SR Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.7.1 The Quantitative Comparison for Multi-scale
SR Tasks . . . . . 27
4.7.2 The Qualitative Comparison for Multi-scale
SR Tasks . . . . . . 28
5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Letter of Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

                                

[1]F.Lin,C.Fookes,V.Chandran,andS.Sridharan,“Superresolvedfacesforimprovedfacerecognitionfrom surveillance video,” inInternational Conference on Biometrics, pp. 1–10, Springer, 2007.
[2]E. AtaerCansizoglu, M. Jones, Z. Zhang, and A. Sullivan, “Verification of very lowresolution facesusing an identitypreserving deep face superresolution network,”arXiv preprint arXiv:1903.10974,2019.
[3]S. Peled and Y. Yeshurun, “Superresolution in mri: application to human white matter fiber tractvisualization by diffusion tensor imaging,”Magnetic Resonance in Medicine: An Official Journal ofthe International Society for Magnetic Resonance in Medicine, vol. 45, no. 1, pp. 29–35, 2001.[4]K. Malczewski and R. Stasiński, “Super resolution for multimedia, image, and video processing applications,” inRecent Advances in Multimedia Signal Processing and Communications, pp. 171–208,Springer, 2009.
[5]A. Z. S. Peleg, “Efficient superresolution and applications to mosaics,” inPattern Recognition, International Conference on, vol. 1, pp. 1579–1579, 2000.[6]A. Bulat and G. Tzimiropoulos, “Superfan: Integrated facial landmark localization and superresolutionofrealworldlowresolutionfacesinarbitraryposeswithgans,” inProceedings of the IEEEConference on Computer Vision and Pattern Recognition, pp. 109–117, 2018.
[7]X. Yu, B. Fernando, B. Ghanem, F. Porikli, and R. Hartley, “Face superresolution guided by facial component heatmaps,” inProceedings of the European Conference on Computer Vision (ECCV),pp. 217–233, 2018.
[8]Y. Chen, Y. Tai, X. Liu, C. Shen, and J. Yang, “Fsrnet: Endtoend learning face superresolution withfacial priors,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,pp. 2492–2501, 2018.
[9]X. Yu, B. Fernando, R. Hartley, and F. Porikli, “Superresolving very lowresolution face images withsupplementary attributes,” inProceedings of the IEEE Conference on Computer Vision and PatternRecognition, pp. 908–917, 2018.
[10]C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz,Z.Wang,et al.,“Photorealisticsingleimagesuperresolutionusingagenerativeadversarialnetwork,”inProceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690,2017.[11]C.Dong,C.C.Loy,K.He,andX.Tang,“Imagesuperresolutionusingdeepconvolutionalnetworks,”IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp. 295–307, 2015.
[12]J. Kim, J. Kwon Lee, and K. Mu Lee, “Accurate image superresolution using very deep convolutional networks,” inProceedings of the IEEE conference on computer vision and pattern recognition,pp. 1646–1654, 2016.
[13]W.S. Lai, J.B. Huang, N. Ahuja, and M.H. Yang, “Deep laplacian pyramid networks for fast andaccurate superresolution,” inProceedings of the IEEE conference on computer vision and patternrecognition, pp. 624–632, 2017.
[14]W.S. Lai, J.B. Huang, N. Ahuja, and M.H. Yang, “Fast and accurate image superresolution withdeep laplacian pyramid networks,”IEEE transactions on pattern analysis and machine intelligence,vol. 41, no. 11, pp. 2599–2613, 2018.
[15]X.YuandF.Porikli,“Ultraresolvingfaceimagesbydiscriminativegenerativenetworks,”inEuropeanconference on computer vision, pp. 318–333, Springer, 2016.[16]X.Xu,D.Sun,J.Pan,Y.Zhang,H.Pfister,andM.H.Yang,“Learningtosuperresolveblurryfaceandtext images,” inProceedings of the IEEE international conference on computer vision, pp. 251–260,2017.
[17]S.Zhu,S.Liu,C.C.Loy,andX.Tang,“Deepcascadedbinetworkforfacehallucination,”inEuropeanconference on computer vision, pp. 614–630, Springer, 2016.
[18]K. Zhang, Z. Zhang, C.W. Cheng, W. H. Hsu, Y. Qiao, W. Liu, and T. Zhang, “Superidentity convolutional neural network for face hallucination,” inProceedings of the European Conference on Computer Vision (ECCV), pp. 183–198, 2018.
[19]R. Kalarot, T. Li, and F. Porikli, “Component attention guided face superresolution network:Cagface,” inThe IEEE Winter Conference on Applications of Computer Vision, pp. 370–380, 2020.
[20]G. Meishvili, S. Jenni, and P. Favaro, “Learning to have an ear for face superresolution,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1364–1374,2020.
[21]M.T.B.Iqbal,M.AbdullahAlWadud,B.Ryu,F.Makhmudkhujaev,andO.Chae,“Facialexpressionrecognitionwithneighborhoodawareedgedirectionalpattern(nedp),”IEEE Transactions on AffectiveComputing, 2018.
[22]A.R. Azar and F.Khalilzadeh, “Real time eyedetection using edge detection andeuclidean distance,”in 2015 2nd International Conference on KnowledgeBased Engineering and Innovation (KBEI),pp. 43–48, IEEE, 2015.
[23]S. Xie and Z. Tu, “Holisticallynested edge detection,” inProceedings of the IEEE international conference on computer vision, pp. 1395–1403, 2015.
[24]Z. Yu, W. Liu, Y. Zou, C. Feng, S. Ramalingam, B. Vijaya Kumar, and J. Kautz, “Simultaneous edgealignment and learning,” inProceedings of the European Conference on Computer Vision (ECCV),pp. 388–404, 2018.
[25]D. Acuna, A. Kar, and S. Fidler, “Devil is in the edges: Learning semantic boundaries from noisyannotations,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,pp. 11075–11083, 2019.
[26]G. S. Robinson, “Color edge detection,”Optical Engineering, vol. 16, no. 5, p. 165479, 1977.
[27]J. Canny, “A computational approach to edge detection,”IEEE Transactions on pattern analysis andmachine intelligence, no. 6, pp. 679–698, 1986.
[28]V. Torre and T. A. Poggio, “On edge detection,”IEEE Transactions on Pattern Analysis and MachineIntelligence, no. 2, pp. 147–163, 1986.
[29]I. Sobel, “Camera models and machine perception,” tech. rep., Computer Science Department, Technion, 1972.
[30]K. Simonyan and A. Zisserman, “Very deep convolutional networks for largescale image recognition,”arXiv preprint arXiv:1409.1556, 2014.
[31]P. Isola, J.Y. Zhu, T. Zhou, and A. A. Efros, “Imagetoimage translation with conditional adversarial networks,” inProceedings of the IEEE conference on computer vision and pattern recognition,pp. 1125–1134, 2017.
[32]K. Roth, A. Lucchi, S. Nowozin, and T. Hofmann, “Stabilizing training of generative adversarialnetworks through regularization,” inAdvances in neural information processing systems, pp. 2018–2028, 2017.
[33]A. Odena, V. Dumoulin, and C. Olah, “Deconvolution and checkerboard artifacts,”Distill, vol. 1,no. 10, p. e3, 2016.
[34]Z. Liu, P. Luo, X. Wang, and X. Tang, “Largescale celebfaces attributes (celeba) dataset,”RetrievedAugust, vol. 15, p. 2018, 2018.
[35]V. Le, J. Brandt, Z. Lin, L. Bourdev, and T. S. Huang, “Interactive facial feature localization,” inEuropean conference on computer vision, pp. 679–692, Springer, 2012.
[36]T. Karras, S. Laine, and T. Aila, “A stylebased generator architecture for generative adversarial networks,”inProceedings of the IEEE conference on computer vision and pattern recognition,pp.4401–4410, 2019.
[37]A. Hore and D. Ziou, “Image quality metrics: Psnr vs. ssim,” in2010 20th international conferenceon pattern recognition, pp. 2366–2369, IEEE, 2010.
[38]Z. Wang and A. C. Bovik, “A universal image quality index,”IEEE signal processing letters, vol. 9,no. 3, pp. 81–84, 2002.
[39]M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two timescale update rule converge to a local nash equilibrium,” inAdvances in neural information processingsystems, pp. 6626–6637, 2017.36
[40]C.Szegedy,V.Vanhoucke,S.Ioffe,J.Shlens,andZ.Wojna,“Rethinkingtheinceptionarchitectureforcomputer vision,” inProceedings of the IEEE conference on computer vision and pattern recognition,pp. 2818–2826, 2016.
[41]T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniquesfor training gans,” inAdvances in neural information processing systems, pp. 2234–2242, 2016.
[42]I. Goodfellow, J. PougetAbadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair, A. Courville, andY. Bengio, “Generative adversarial nets,” inAdvances in neural information processing systems,pp. 2672–2680, 2014.
[43]A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutionalgenerative adversarial networks,”arXiv preprint arXiv:1511.06434, 2015.[44]R.R.Selvaraju,M.Cogswell,A.Das,R.Vedantam,D.Parikh,andD.Batra,“Gradcam: Visualexplanationsfromdeepnetworksviagradientbasedlocalization,”inProceedings of the IEEE internationalconference on computer vision, pp. 618–626, 2017.
[45]R.Keys, “Cubicconvolutioninterpolationfordigitalimageprocessing,”IEEE transactions on acoustics, speech, and signal processing, vol. 29, no. 6, pp. 1153–1160, 1981.
[46]I.T.Jolliffe,“Principalcomponentsinregressionanalysis,”inPrincipal component analysis,pp.129–155, Springer, 1986.
[47]H. J. Nussbaumer, “The fast fourier transform,” inFast Fourier Transform and Convolution Algorithms, pp. 80–111, Springer, 1981.
[48]J. W. Soh, S. Cho, and N. I. Cho, “Metatransfer learning for zeroshot superresolution,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3516–3525,2020.
[49]Z. Wang, J. Chen, and S. C. Hoi, “Deep learning for image superresolution: A survey,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[50]K. Cho, A. Courville, and Y. Bengio, “Describing multimedia content using attentionbased encoderdecoder networks,”IEEE Transactions on Multimedia, vol. 17, no. 11, pp. 1875–1886, 2015.
[51]D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”arXiv preprintarXiv:1412.6980, 2014.
[52]I. Goodfellow, “Nips 2016 tutorial: Generative adversarial networks,”arXiv preprintarXiv:1701.00160, 2016.
[53]A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acousticmodels,” inProc. icml, vol. 30, p. 3, 2013.37

全文公開日期 2025/08/21 (校內網路)
全文公開日期本全文未授權公開 (校外網路)
全文公開日期本全文未授權公開 (國家圖書館：臺灣博碩士論文系統)

簡易檢索 / 詳目顯示

相關論文