簡易檢索 / 詳目顯示

回結果列表
研究生： 呂營程
Ying-Cheng Lu
論文名稱： 基於除模糊核預測之二階段單一影像除模糊網路
Two-stage Single Image Deblurring Network Based on Deblur Kernel Estimation
指導教授： 林昌鴻
Chang Hong Lin
口試委員: 阮聖彰
Shanq-Jang Ruan
吳晉賢
Chin-Hsien Wu
林淵翔
Yuan-Hsiang Lin
學位類別： 碩士
Master
系所名稱： 電資學院 - 電子工程系
Department of Electronic and Computer Engineering
論文出版年： 2021
畢業學年度： 109
語文別： 英文
論文頁數： 77
中文關鍵詞： 影像除模糊 、影像品質改善 、深度學習 、卷積神經網路 、聯合學習
外文關鍵詞： Image Deblurring, Image Quality Improvement, Deep Learning, Convolution Neural Network, Joint Learning
相關次數： 點閱：204  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 動態場景除模糊對於計算機視覺領域是一項具有挑戰的題目，其模糊成因是由於曝光期間相機晃動或物體移動所引起的。許多照片拍攝的瞬間是無法重現的，因此若照片中的資訊產生模糊，便無法還原其內容。隨著科技的發展，有大量的應用是藉由影像進行辨識、分析等。若輸入的影像由於模糊而降低其影像品質，會影響其性能。因此影像除模糊成為一項重要的技術，此技術不僅可以讓我們還原丟失的影像還可以幫助一些高階影像處理方法提升性能。近年來隨著深度學習在影像處理領域的成功，本論文提出一個除模糊的系統，利用兩階段的卷積神經網路(Convolutional Neural Network，簡稱CNN)以聯合學習的方式來達到去模糊結果。第一階段卷積神經網路預測像每個像素的除模糊核並先對輸入圖像進行預除模糊再由第二階段卷積神經網路直接預測清晰的影像。由於運動模糊通常是相機晃動或是物體移動所造成，其潛在像素資訊會散布在周圍的空間，除模糊核即是使用周圍的資訊來還原中心像素，這可以有效的去除較細小的模糊，但受限於核的大小，除模糊核對於較大的運動模糊效果並不佳，因此使用第二階段網路來補償除模糊核的有限視野(Receptive Field)。我們在模糊數據集上評估我們的方法。結果表明，與現有技術相比，我們的方法在定量和定性方面都能產生更好的結果。

    Image deblurring for dynamic scenes is a challenging computer vision problem. Motion blur is caused by camera shaking or object movement during the exposure time. Many photos cannot be reproduced at the moment they are taken, so if the motion blur occurs, its content cannot be restored. With the development of technology, several applications use images for recognition and analysis. If the input image is blurred, its performance will be affected. Image deblurring technology not only allows us to restore lost images but also helps some high-level image processing methods to improve performance. This thesis proposes a deblurring system that uses a two stage convolutional neural network (CNN) to achieve image deblurring with a joint learning strategy. The first stage network predicts the deblur kernel of each pixel and pre-deblurs the input image, and then the second stage network directly predicts clear images. Since latent pixels’ information are scattered in a motion blurred image, the deblur kernel is to use the surrounding information to restore the center pixel, which can effectively remove the small motion blur. However, the deblur kernel is not effective in large motion blur, so the second stage network is used to compensate for the limited receptive field of the first stage deblur kernel. We evaluate our method on benchmark blur datasets. Results show that our method can produce better results than state-of-the-art methods, both quantitatively and qualitatively.

    摘要 I ABSTRACT II 致謝 III LIST OF CONTENTS IV LIST OF FIGURES VII LIST OF TABLES IX CHAPTER 1 INTRODUCTIONS 1 1.1 Motivation 1 1.2 Contributions 3 1.3 Thesis Organization 4 CHAPTER 2 RELATED WORKS 5 2.1 Image Blur Model 5 2.2 Blur Kernel Estimation for Image Deblurring 6 2.2.1 Uniform Blur Kernel Estimation 6 2.2.2 Non-Uniform Blur Kernel Estimation 7 2.3 Kernel-Free for Image Deblurring 8 CHAPTER 3 PROPOSED METHOD 9 3.1 Data Augmentation 11 3.1.1 Random Crop 11 3.1.2 Geometric Self-ensemble 13 3.1.3 Saturation Adjustment 14 3.1.4 Hue Adjustment 15 3.2 Network Architecture 17 3.2.1 U-net architecture [28] 17 3.3 Pixel-wise Kernel Estimation Network 19 3.3.1 Pixel-wise Deblur Kernel 19 3.3.2 Dynamic Local Filtering [17, 18] 19 3.3.3 Architecture Detail 21 3.3.4 Residual Channel Attention Block [29] 23 3.3.5 Residual Dense Block [30] 25 3.4 Image Deblurring Network 27 3.4.1 Architecture Detail 28 3.5 Training Setting 30 3.5.1 Joint Learning 30 3.5.2 Initialization 31 3.5.3 Optimizer 32 3.5.4 Learning Rate Decay 33 3.6 Loss Function 34 3.6.1 Spatial Loss 35 3.6.2 Spectral Loss 35 3.6.3 SSIM Loss 36 3.6.4 Gradient Loss 36 CHAPTER 4 EXPERIMENTAL RESULTS 37 4.1 Experimental Environment 37 4.2 Blur Dataset 38 4.2.1 GOPRO Dataset [11] 38 4.3 Evaluation Methods 41 4.3.1 PSNR 41 4.3.2 SSIM [39] 41 4.3.3 MS-SSIM [42] 42 4.4 Performance Evaluation 43 4.4.1 GOPRO testing set [11] 43 4.4.2 Kohler Dataset [43] 47 4.4.3 Su Dataset [44] 51 4.4.4 Lai Dataset [45] 54 CHAPTER 5 CONCLUSIONS AND FUTURE WORKS 60 5.1 Conclusions 60 5.2 Future Works 61 REFERENCES 62

    [1] T. F. Chan and C.-K. Wong, "Total variation blind deconvolution," IEEE transactions on Image Processing, vol. 7, no. 3, pp. 370-375, 1998.
    [2] A. Goldstein and R. Fattal, "Blur-kernel estimation from spectral irregularities," in European Conference on Computer Vision, 2012: Springer, pp. 622-635.
    [3] S. Cho and S. Lee, "Fast motion deblurring," in ACM SIGGRAPH Asia 2009 papers, 2009, pp. 1-8.
    [4] Y. Bahat, N. Efrat, and M. Irani, "Non-uniform blind deblurring by reblurring," in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3286-3294.
    [5] J. Pan, Z. Hu, Z. Su, and M.-H. Yang, "Deblurring text images via L0-regularized intensity and gradient prior," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 2901-2908.
    [6] J. Pan, D. Sun, H. Pfister, and M.-H. Yang, "Blind image deblurring using dark channel prior," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1628-1636.
    [7] L. Xu and J. Jia, "Two-phase kernel estimation for robust motion deblurring," in European conference on computer vision, 2010: Springer, pp. 157-170.
    [8] L. Xu, S. Zheng, and J. Jia, "Unnatural l0 sparse representation for natural image deblurring," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 1107-1114.
    [9] C. J. Schuler, M. Hirsch, S. Harmeling, and B. Schölkopf, "Learning to deblur," IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 7, pp. 1439-1451, 2015.
    [10] J. Sun, W. Cao, Z. Xu, and J. Ponce, "Learning a convolutional neural network for non-uniform motion blur removal," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 769-777.
    [11] S. Nah, T. Hyun Kim, and K. Mu Lee, "Deep multi-scale convolutional neural network for dynamic scene deblurring," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 3883-3891.
    [12] X. Tao, H. Gao, X. Shen, J. Wang, and J. Jia, "Scale-recurrent network for deep image deblurring," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8174-8182.
    [13] H. Zhang, Y. Dai, H. Li, and P. Koniusz, "Deep stacked hierarchical multi-patch network for image deblurring," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 5978-5986.
    [14] H. Gao, X. Tao, X. Shen, and J. Jia, "Dynamic scene deblurring with parameter selective sharing and nested skip connections," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 3848-3856.
    [15] H. Sim and M. Kim, "A deep motion deblurring network based on per-pixel adaptive kernels with residual down-up and up-down modules," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2019.
    [16] S. Lim, J. Kim, and W. Kim, "Deep Spectral-Spatial Network for Single Image Deblurring," IEEE Signal Processing Letters, 2020.
    [17] Y. Jo, S. W. Oh, J. Kang, and S. J. Kim, "Deep video super-resolution network using dynamic upsampling filters without explicit motion compensation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3224-3232.
    [18] X. Jia, B. De Brabandere, T. Tuytelaars, and L. Van Gool, "Dynamic filter networks," in Advances in neural information processing systems, 2016, pp. 667-675.
    [19] D. Kundur and D. Hatzinakos, "Blind image deconvolution," IEEE signal processing magazine, vol. 13, no. 3, pp. 43-64, 1996.
    [20] R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, "Removing camera shake from a single photograph," in ACM SIGGRAPH 2006 Papers, 2006, pp. 787-794.
    [21] Q. Shan, J. Jia, and A. Agarwala, "High-quality motion deblurring from a single image," ACM transactions on graphics (tog), vol. 27, no. 3, pp. 1-10, 2008.
    [22] M. Hirsch, C. J. Schuler, S. Harmeling, and B. Schölkopf, "Fast removal of non-uniform camera shake," in 2011 International Conference on Computer Vision, 2011: IEEE, pp. 463-470.
    [23] A. Gupta, N. Joshi, C. L. Zitnick, M. Cohen, and B. Curless, "Single image deblurring using motion density functions," in European Conference on Computer Vision, 2010: Springer, pp. 171-184.
    [24] J. Pan, Z. Hu, Z. Su, H.-Y. Lee, and M.-H. Yang, "Soft-segmentation guided object motion deblurring," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 459-468.
    [25] T. H. Kim, B. Ahn, and K. M. Lee, "Dynamic scene deblurring," in Proceedings of the IEEE International Conference on Computer Vision, 2013, pp. 3160-3167.
    [26] T. H. Kim and K. M. Lee, "Segmentation-free dynamic scene deblurring," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 2766-2773.
    [27] R. Timofte, R. Rothe, and L. Van Gool, "Seven ways to improve example-based single image super resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1865-1873.
    [28] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention, 2015: Springer, pp. 234-241.
    [29] Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, "Image super-resolution using very deep residual channel attention networks," in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 286-301.
    [30] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2472-2481.
    [31] B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced deep residual networks for single image super-resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 136-144.
    [32] T. Tong, G. Li, X. Liu, and Q. Gao, "Image super-resolution using dense skip connections," in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 4799-4807.
    [33] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
    [34] X. Glorot and Y. Bengio, "Understanding the difficulty of training deep feedforward neural networks," in Proceedings of the thirteenth international conference on artificial intelligence and statistics, 2010, pp. 249-256.
    [35] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
    [36] N. Qian, "On the momentum term in gradient descent learning algorithms," Neural networks, vol. 12, no. 1, pp. 145-151, 1999.
    [37] J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization," Journal of machine learning research, vol. 12, no. 7, 2011.
    [38] G. Hinton, N. Srivastava, and K. Swersky, "Neural networks for machine learning lecture 6a overview of mini-batch gradient descent," Cited on, vol. 14, no. 8, 2012.
    [39] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE transactions on image processing, vol. 13, no. 4, pp. 600-612, 2004.
    [40] H. Zhang and V. M. Patel, "Densely connected pyramid dehazing network," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3194-3203.
    [41] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, "Automatic differentiation in pytorch," 2017.
    [42] Z. Wang, E. P. Simoncelli, and A. C. Bovik, "Multiscale structural similarity for image quality assessment," in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, 2003, vol. 2: Ieee, pp. 1398-1402.
    [43] R. Köhler, M. Hirsch, B. Mohler, B. Schölkopf, and S. Harmeling, "Recording and playback of camera shake: Benchmarking blind deconvolution with a real-world database," in European conference on computer vision, 2012: Springer, pp. 27-40.
    [44] S. Su, M. Delbracio, J. Wang, G. Sapiro, W. Heidrich, and O. Wang, "Deep video deblurring for hand-held cameras," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1279-1288.
    [45] W.-S. Lai, J.-B. Huang, Z. Hu, N. Ahuja, and M.-H. Yang, "A comparative study for single image blind deblurring," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1701-1709.

    無法下載圖示 全文公開日期 2026/01/26 (校內網路)
    全文公開日期 本全文未授權公開 (校外網路)
    全文公開日期 本全文未授權公開 (國家圖書館：臺灣博碩士論文系統)
    QR CODE