簡易檢索 / 詳目顯示
| 研究生： |
Jose Jaena Mari Romabiles Ople Jose Jaena Mari Ople |
|---|---|
| 論文名稱： |
Augmenting Super-Resolution using Neural Texture Transfer Augmenting Super-Resolution using Neural Texture Transfer |
| 指導教授： |
花凱龍
Kai-Lung Hua |
| 口試委員: |
Arnulfo Azcarraga
Arnulfo Azcarraga 楊傳凱 Chuan-Kai Yang 陳駿丞 Jun-Cheng Chen |
| 學位類別： |
碩士 Master |
| 系所名稱： |
電資學院 - 資訊工程系 Department of Computer Science and Information Engineering |
| 論文出版年： | 2020 |
| 畢業學年度： | 108 |
| 語文別： | 英文 |
| 論文頁數： | 41 |
| 中文關鍵詞： | super-resolution 、machine learning 、computer vision 、deep learning 、texture transfer |
| 外文關鍵詞： | super-resolution, machine learning, computer vision, deep learning, texture transfer |
| 相關次數： | 點閱：328 下載：7 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
Recent deep learning approaches in Single image superresolution (SISR) can generate highdefinition textures for superresolved (SR) images. However, they tend to hallucinate fake textures and even produce artifacts. Alternative to SISR, Referencebased SR (RefSR) approaches use highresolution (HR) reference images to provide HR details that are missing in the low-resolution input image. We propose a novel framework that leverages existing SISR approaches and augment them with RefSR. Specifically, we refine the output of SISR methods using neural texture transfer, where HR features are queried from Ref images. The query is conducted by computing the similarity between the features of the input lowresolution (LR) image and the Ref images. The most similar HR features, patchwise, is used to augment the output image of the SISR approach. Different from past RefSR approaches, our method does not impose limitations on the Ref images. We showcase that our method drastically improves the performance of the base SISR approach.
Recent deep learning approaches in Single image superresolution (SISR) can generate highdefinition textures for superresolved (SR) images. However, they tend to hallucinate fake textures and even produce artifacts. Alternative to SISR, Referencebased SR (RefSR) approaches use highresolution (HR) reference images to provide HR details that are missing in the low-resolution input image. We propose a novel framework that leverages existing SISR approaches and augment them with RefSR. Specifically, we refine the output of SISR methods using neural texture transfer, where HR features are queried from Ref images. The query is conducted by computing the similarity between the features of the input lowresolution (LR) image and the Ref images. The most similar HR features, patchwise, is used to augment the output image of the SISR approach. Different from past RefSR approaches, our method does not impose limitations on the Ref images. We showcase that our method drastically improves the performance of the base SISR approach.
[1] K. Simonyan and A. Zisserman, “Very deep convolutional networks for largescale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[2] Z. Zhang, Z. Wang, Z. Lin, and H. Qi, “Image superresolution by neural texture transfer,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7982–7991, 2019.
[3] D. G. Lowe et al., “Object recognition from local scaleinvariant features.,” in iccv, vol. 99, pp. 1150–1157, 1999.
[4] B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image
superresolution,” in Proceedings of the IEEE conference on computer vision and pattern recognition
workshops, pp. 136–144, 2017.
[5] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al., “Photorealistic single image superresolution using a generative adversarial network,”in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690, 2017.
[6] C.Y. Yang, C. Ma, and M.H. Yang, “Singleimage superresolution: A benchmark,” in European Conference on Computer Vision, pp. 372–386, Springer, 2014.
[7] J. Kim, J. Kwon Lee, and K. Mu Lee, “Accurate image superresolution using very deep convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646–1654, 2016.
[8] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al., “Photorealistic single image superresolution using a generative adversarial network,”in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690, 2017.
[9] M. S. Sajjadi, B. Scholkopf, and M. Hirsch, “Enhance-net: Single image superresolution through automated texture synthesis,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 4491–4500, 2017.
[10] J. Johnson, A. Alahi, and L. FeiFei, “Perceptual losses for realtime style transfer and super-resolution,” in European conference on computer vision, pp. 694–711, Springer, 2016.
[11] I. Goodfellow, J. PougetAbadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, pp. 2672–2680, 2014.
[12] A. Shocher, N. Cohen, and M. Irani, ““zeroshot”superresolution using deep internal learning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3118–3126, 2018.
[13] R. Timofte, V. De Smet, and L. Van Gool, “Anchored neighborhood regression for fast examplebased super-resolution,” in Proceedings of the IEEE international conference on computer vision, pp. 1920–1927, 2013.
[14] L. Sun and J. Hays, “Superresolution from internetscale scene matching,” in 2012 IEEE International Conference on Computational Photography (ICCP), pp. 1–12, IEEE, 2012.
[15] G. Freedman and R. Fattal, “Image and video upscaling from local selfexamples,” ACM Transactions on Graphics (TOG), vol. 30, no. 2, p. 12, 2011.
[16] H. Chang, D.Y. Yeung, and Y. Xiong, “Superresolution through neighbor embedding,” in Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., vol. 1, pp. I–I, IEEE, 2004.
[17] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
[18] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.
[19] V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international conference on machine learning (ICML10), pp. 807–814, 2010.
[20] J. Johnson, A. Alahi, and L. FeiFei, “Perceptual losses for realtime style transfer and superresolution,” in European conference on computer vision, pp. 694–711, Springer, 2016.
[21] K. Simonyan and A. Zisserman, “Very deep convolutional networks for largescale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[22] E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image superresolution: Dataset and study,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 126–135, 2017.
[23] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv: 1412.6980, 2014.
[24] L. Gatys, A. S. Ecker, and M. Bethge, “Texture synthesis using convolutional neural networks,” in Advances in neural information processing systems, pp. 262–270, 2015.
[25] M. Mori, K. F. MacDorman, and N. Kageki, “The uncanny valley [from the field],” IEEE Robotics & Automation Magazine, vol. 19, no. 2, pp. 98–100, 2012.
