簡易檢索 / 詳目顯示

回結果列表
研究生： 劉子璞
TZU-PU LIU
論文名稱： 基於特徵強化架構之二階段單一影像除模糊網路
Two-Stage Single Image Deblurring Network Based on Feature Enhancement Architecture
指導教授： 林昌鴻
Chang-Hong Lin
口試委員: 阮聖彰
Shanq-Jang Ruan
吳晋賢
Chin-Hsien Wu
林淵翔
Yuan-Hsiang Lin
學位類別： 碩士
Master
系所名稱： 電資學院 - 電子工程系
Department of Electronic and Computer Engineering
論文出版年： 2021
畢業學年度： 109
語文別： 中文
論文頁數： 73
中文關鍵詞： 影像除模糊 、影像品質改善 、深度學習 、卷積神經網路 、通道注意力機制
外文關鍵詞： Image Deblurring, Image Quality Improvement, Deep Learning, Convolution Neural Network, Channel Attention
相關次數： 點閱：165  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 在計算機視覺領域上，影像除模糊是一項常見也具有實用性的問題，比如在照相、錄影、或是工廠用來偵測產線問題的儀器、其他影像處理的前處理都能用到。在進行影像處理應用時，若影像模糊就可能會影響它的結果，藉由影像除模糊讓模糊圖片能得到一個較有可信度的清晰圖片供其後續使用，因此影像除模糊是很有實用性的項目。近年來深度學習越來越廣泛應用在各種影像處理，隨著深度學習的發展出現很多不同的解決問題的方法，也有越來越多種數據集出現提供研究者使用，為了提高準確率，也開始發展多層的卷積神經網路(Convolutional Neural Network，CNN)方法。
    本論文提出一個兩階段除模糊系統。第一階段先使用較小區域原始模糊圖片在子網路中利用注意力機制(Channel Attention)萃取資訊，之後再加總到下一階段較大區域的原始模糊圖片中再利用同樣架構提取最後的特徵加強圖。第二階時將前一級的特徵加強圖聯接到原始模糊圖像，經由卷積神經網路直接預測清晰的圖片。本論文的方法在GOPRO數據集中訓練，並在GOPRO及Su的模糊數據集上評估我們的方法，也比較了僅卷積神經網路、卷積神經網路加入一層注意力機制的特徵圖及卷積神經網路加入兩層注意力機制的加強特徵圖，在GOPRO數據集上實驗的結果：PSNR為32.35、SSIM為0.9558；在Su的數據集上的結果：PSNR為31.64，SSIM為0.9352。這些結果表明，從小區域提取特徵圖後再連結較大的圖片的特徵強化，在卷積神經網路處理後會有較佳的結果，和現有技術相比，也有更好的結果。

    Image deblurring is a common and practical computer vision technique in the deep learning field. For example, it can be used in photography, video recording, equipment used in factories to detect production line problems, and other pre-processing of image processing.
    If the images are blurred, this could worsen the performance of other image processing methods. Therefore, the image deblurring can make the reliable sharp images for subsequent applications. So, image deblurring is a practical application. In the recent years, the deep learning has become more widely used in various image processing applications. With the development of deep learning, many different image deblurring methods have emerged, and datasets of wider applicability have appeared for researchers to use. In order to improve the accuracy, the multi-hierarchy convolutional neural network methods have also been developed.
    Therefore, this thesis proposed a two-stage deblurring system. The first stage uses the Channel Attention subnetwork on smaller regions of the original blurred image to obtain the information, and then summate them to the larger regions of original blurred image with the same subnetwork. Finally, the final feature enhancement result images have been obtained. The second stage concatenates the result images of the first stage to the original images, and then use the Convolutional Neural Network (CNN) to predict the sharp images. The proposed method is trained on the GOPRO dataset, and is evaluated on both the GOPRO and the Su’s datasets to compare with other state-of-the-art methods. The results on the GOPRO dataset have the PSNR of 32.35 and the SSIM of 0.9558, and the results on the Su’s dataset have the PSNR of 31.64 and the SSIM of 0.9352. Those results show the feature enhancement processing from small to large would make the CNN predict better results.

    摘要 I ABSTRACT II 致謝 III LIST OF CONTENTS IV LIST OF FIGURES VI LIST OF TABLES VIII CHAPTER 1 INTRODUCTIONS 1 1.1 Motivation 1 1.2 Contributions 3 1.3 Thesis Organization 4 CHAPTER 2 RELATED WORKS 5 2.1 The Generated Blurred Images 5 2.2 Uniform and Non-Uniform Deblurring Methods 6 2.3 Single-Stage and Multi-Stage Architectures 7 CHAPTER 3 PROPOSED METHOD 8 3.1 Data Augmentation 9 3.1.1 Random Crop 9 3.1.2 Random Flip 11 3.1.3 Random Rotation 12 3.1.4 Random Saturation Adjustment 13 3.1.5 Random Hue Adjustment 15 3.2 First Stage Architecture 17 3.2.1 Channel Attention Network 18 3.2.2 Residual Learning [30] 22 3.2.3 Input Image Splitting 23 3.3 Second Stage Architecture 27 3.3.1 U-net 28 3.4 Training Setting 30 3.4.1 Optimizer 30 3.4.2 Learning Rate Decay 35 3.4.3 Joint Learning 36 3.4.4 Loss Functions 37 CHAPTER 4 EXPERIMENTAL RESULTS 40 4.1 Experimental Environment 40 4.2 GOPRO Dataset [9] 41 4.3 Evaluation Methods 44 4.3.1 PSNR [52] 44 4.3.2 SSIM [53] 45 4.4 Performance Evaluation 47 4.4.1 GOPRO test dataset [9] 47 4.4.2 Su et al’s Quantitative Dataset [27] 52 CHAPTER 5 CONCLUSIONS and Future works 56 5.1 Conclusions 56 5.2 Future Works 57 REFERENCES 58

    [1] C. Szegedy, L. Wei, J. Yangqing, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going Deeper with Convolutions," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June 2015, pp. 1-9.
    [2] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, "The Cityscapes Dataset for Semantic Urban Scene Understanding," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June 2016, pp. 3213-3223.
    [3] H. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, and R. M. Summers, "Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning," IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1285-1298, 2016.
    [4] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising," IEEE Transactions on Image Processing (TIP), vol. 26, no. 7, pp. 3142-3155, 2017.
    [5] F. Milletari, N. Navab, and S. Ahmadi, "V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation," in 2016 Fourth International Conference on 3D Vision (3DV), 25-28 Oct. 2016, pp. 565-571.
    [6] W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June 2016, pp. 1874-1883.
    [7] D. Ciregan, U. Meier, and J. Schmidhuber, "Multi-Column Deep Neural Networks for Image Classification," in 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 16-21 June 2012, pp. 3642-3649.
    [8] L. A. Gatys, A. S. Ecker, and M. Bethge, "Image Style Transfer Using Convolutional Neural Networks," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June 2016, pp. 2414-2423.
    [9] S. Nah, T. H. Kim, and K. M. Lee, "Deep Multi-Scale Convolutional Neural Network for Dynamic Scene Deblurring," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July 2017, pp. 257-265.
    [10] C. J. Schuler, M. Hirsch, S. Harmeling, and B. Schölkopf, "Learning to Deblur," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 38, no. 7, pp. 1439-1451, 2016.
    [11] J. Sun, C. Wenfei, X. Zongben, and J. Ponce, "Learning a Convolutional Neural Network for Non-Uniform Motion Blur Removal," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June 2015, pp. 769-777.
    [12] X. Tao, H. Gao, X. Shen, J. Wang, and J. Jia, "Scale-Recurrent Network for Deep Image Deblurring," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 18-23 June 2018, pp. 8174-8182.
    [13] H. Zhang, Y. Dai, H. Li, and P. Koniusz, "Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019, pp. 5971-5979.
    [14] H. Gao, X. Tao, X. Shen, and J. Jia, "Dynamic Scene Deblurring with Parameter Selective Sharing and Nested Skip Connections," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019, pp. 3843-3851.
    [15] H. Sim and M. Kim, "A Deep Motion Deblurring Network Based on Per-Pixel Adaptive Kernels with Residual Down-Up and Up-Down Modules," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 16-17 June 2019, pp. 2140-2149.
    [16] S. Lim, J. Kim, and W. Kim, "Deep Spectral-Spatial Network for Single Image Deblurring," IEEE Signal Processing Letters (SPL), vol. 27, pp. 835-839, 2020.
    [17] M. Suin, K. Purohit, and A. N. Rajagopalan, "Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June 2020, pp. 3603-3612.
    [18] S. Anwar and N. Barnes, "Real Image Denoising with Feature Attention," in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 27 Oct.-2 Nov. 2019, pp. 3155-3164.
    [19] Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image Super-Resolution Using Very Deep Residual Channel Attention Networks," in 2018 European Conference on Computer Vision (ECCV), Cham, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds., 2018: Springer International Publishing, pp. 286-301.
    [20] W. Yang, R. T. Tan, J. Feng, J. Liu, Z. Guo, and S. Yan, "Deep Joint Rain Detection and Removal from a Single Image," in 2017 IEEE Conference On Computer Vision And Pattern Recognition (CVPR), 21-26 July 2017, pp. 1685-1694.
    [21] T. Dai, J. Cai, Y. Zhang, S. Xia, and L. Zhang, "Second-Order Attention Network for Single Image Super-Resolution," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019, pp. 11057-11066.
    [22] T. Brooks, B. Mildenhall, T. Xue, J. Chen, D. Sharlet, and J. T. Barron, "Unprocessing Images for Learned Raw Denoising," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019, pp. 11028-11037.
    [23] H. Zhang, V. Sindagi, and V. M. Patel, "Image De-Raining Using a Conditional Generative Adversarial Network," IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), vol. 30, no. 11, pp. 3943-3956, 2020.
    [24] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July 2017, pp. 105-114.
    [25] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman, "Understanding and Evaluating Blind Deconvolution Algorithms," in 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 20-25 June 2009, pp. 1964-1971.
    [26] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman, "Efficient Marginal Likelihood Optimization in Blind Deconvolution," in 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 20-25 June 2011, pp. 2657-2664.
    [27] S. Su, M. Delbracio, J. Wang, G. Sapiro, W. Heidrich, and O. Wang, "Deep Video Deblurring for Hand-Held Cameras," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July 2017, pp. 237-246.
    [28] S. Cho and S. Lee, "Fast Motion Deblurring," in Association for Computing Machinery (ACM) Special Interest Group in Graphics (SIGGRAPH) Asia 2009 papers, 2009, pp. 1-8.
    [29] Q. Shan, J. Jia, and A. Agarwala, "High-Quality Motion Deblurring from a Single Image," Association for Computing Machinery (ACM) Transactions on Graphics (TOG), vol. 27, no. 3, pp. 1-10, 2008.
    [30] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June 2016, pp. 770-778.
    [31] K. Jiang, Z. Wang, P. Yi, C. Chen, B. Huang, Y. Luo, J. Ma, and J. Jiang, "Multi-Scale Progressive Fusion Network for Single Image Deraining," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June 2020, pp. 8343-8352.
    [32] O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas, "DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 18-23 June 2018, pp. 8183-8192.
    [33] O. Kupyn, T. Martyniuk, J. Wu, and Z. Wang, "DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better," in IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 8878-8887.
    [34] Y. Wei, H. Xiao, H. Shi, Z. Jie, J. Feng, and T. S. Huang, "Revisiting Dilated Convolution: A Simple Approach for Weakly-and Semi-Supervised Semantic Segmentation," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 7268-7277.
    [35] O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," in International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 18 Nov. 2015: Springer, pp. 234-241.
    [36] I. V. Tetko, D. J. Livingstone, and A. I. Luik, "Neural Network Studies. 1. Comparison of Overfitting and Overtraining," Journal of Chemical Information and Computer Sciences, vol. 35, no. 5, pp. 826-833, 1 Sep. 1995.
    [37] C. Shorten and T. M. Khoshgoftaar, "A Survey on Image Data Augmentation for Deep Learning," Journal of Big Data, vol. 6, no. 1, p. 60, 6 July 2019.
    [38] A. R. Smith, "Color Gamut Transform Pairs," Association for Computing Machinery (ACM) Special Interest Group in Graphics (SIGGRAPH) Computer Graphics, vol. 12, no. 3, pp. 12-19, 1978.
    [39] D. Tellez, G. Litjens, P. Bándi, W. Bulten, J.-M. Bokhorst, F. Ciompi, and J. van der Laak, "Quantifying the Effects of Data Augmentation and Stain Color Normalization in Convolutional Neural Networks for Computational Pathology," Medical Image Analysis, vol. 58, p. 101544, 1 Dec. 2019.
    [40] K. He, X. Zhang, S. Ren, and J. Sun, "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 37, no. 9, pp. 1904-1916, 2015.
    [41] X. Lu, Z. Lin, X. Shen, R. Mech, and J. Z. Wang, "Deep Multi-Patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation," in 2015 IEEE International Conference on Computer Vision (ICCV), 7-13 Dec. 2015, pp. 990-998.
    [42] L. Pan, Y. Dai, M. Liu, and F. Porikli, "Simultaneous Stereo Video Deblurring and Scene Flow Estimation," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July 2017, pp. 6987-6996.
    [43] S. Ramakrishnan, S. Pachori, A. Gangopadhyay, and S. Raman, "Deep Generative Filter for Motion Deblurring," in 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 22-29 Oct. 2017, pp. 2993-3000.
    [44] D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," arXiv preprint arXiv:1412.6980, 2014.
    [45] N. Qian, "On the Momentum Term in Gradient Descent Learning Algorithms," Neural networks, vol. 12, no. 1, pp. 145-151, 1999.
    [46] J. Duchi, E. Hazan, and Y. Singer, "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization," Journal of Machine Learning Research, vol. 12, no. 7, pp. 2121-2159, 2011.
    [47] G. Hinton, N. Srivastava, and K. Swersky, "Neural Networks for Machine Learning Lecture 6a Overview of Mini-Batch Gradient Descent," vol. 14, no. 8, pp. 26-31, 2012.
    [48] L. Bottou, "Large-Scale Machine Learning with Stochastic Gradient Descent," in Computer Statistics (COMPSTAT) 2010: Springer, 2010, pp. 177-186.
    [49] D. M. Allen, "Mean Square Error of Prediction as a Criterion for Selecting Variables," Technometrics, vol. 13, no. 3, pp. 469-475, 1971.
    [50] H. Zhang and V. M. Patel, "Densely Connected Pyramid Dehazing Network," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 18-23 June 2018, pp. 3194-3203.
    [51] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, "Automatic Differentiation in Pytorch," 2017.
    [52] Q. Huynh-Thu and M. Ghanbari, "Scope of Validity of PSNR in Image/Video Quality Assessment," Electronics letters, vol. 44, no. 13, pp. 800-801, 2008.
    [53] W. Zhou, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image Quality Assessment: from Error Visibility to Structural Similarity," IEEE Transactions on Image Processing (TIP), vol. 13, no. 4, pp. 600-612, 2004.
    [54] T. H. Kim and K. M. Lee, "Segmentation-Free Dynamic Scene Deblurring," in 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 23-28 June 2014, pp. 2766-2773.
    [55] Z. Fu, T. Ma, Y. Zheng, H. Ye, J. Yang, and L. He, "Edge-Aware Deep Image Deblurring," arXiv preprint arXiv:1907.02282, 2019.

    無法下載圖示 全文公開日期 2024/08/24 (校內網路)
    全文公開日期 本全文未授權公開 (校外網路)
    全文公開日期 本全文未授權公開 (國家圖書館：臺灣博碩士論文系統)
    QR CODE