Basic Search / Detailed Display

Author: 莊承恩
Cheng-En Zhuang
Thesis Title: 基於深度學習之陰影去除:針對目前方法學缺失進行改善
Deep learning based Shadow Removal: Target to current methodology flaws
Advisor: 吳怡樂
Yi-Leh Wu
Committee: 楊竹星
林祝興
李正吉
謝仁偉
吳怡樂
Degree: 碩士
Master
Department: 電資學院 - 資訊工程系
Department of Computer Science and Information Engineering
Thesis Publication Year: 2022
Graduation Academic Year: 110
Language: 中文
Pages: 41
Keywords (in Chinese): 陰影去除深度學習照明模型
Keywords (in other languages): Shadow Removal, Deep Learning, Illumination Model
Reference times: Clicks: 256Downloads: 2
Share:
School Collection Retrieve National Library Collection Retrieve Error Report
  • 陰影去除(Shadow removal),是將具有陰影的圖片轉換為沒有陰影的圖片,目前是一項尚未成熟的技術,在去除陰影時需要考慮各種因素,包括環境、光照等等,利用傳統影像處理的方式複雜且繁瑣,效果卻不盡理想。近期隨著深度學習(Deep Learning)迅速發展,研究者提出不同的理論來處理此任務,成果獲得不少提升,但仍具部分問題需要解決。

    目前基於深度學習進行陰影去除論文中大致可列為三個問題,第一是陰影去除後出現顏色不一致(Color inconsistency),陰影區域在去除後難以恢復正確顏色,並與非陰影區域具明顯顏色差異,第二是在陰影去除後,陰影邊界明顯殘留於圖片中,第三是此領域資料集難以蒐集,導致訓練集資料匱乏,令模型無法完全適應各種場景。本論文針對上述三個問題分別提出解決方法,最終模型在評估陰影區域及非陰影區域的均方根誤差(Root mean square error, RMSE)及結構相似性(Structural similarity index, SSIM)皆獲得有效的提升。


    Shadow removal, which converts images with shadows to images without shadows is currently an immature technology. When removing shadows, we need to consider various factors including lighting, environment etc. The traditional image processing method is complex and cumbersome, but the result is not ideal. Recently, with the rapid development of deep learning, various papers have proposed different theories to deal with this task, and the results have been greatly improved, but there are still some problems to be solved.

    At present, there are three problems in the shadow removal paper based on deep learning. First, the problem is color inconsistency. The shadow area is difficult to restore the correct color and has a significant color difference from non-shadow areas after removal. Second, the shadow boundaries are clearly left in the image. Last, it is difficult to collect datasets in this field, resulting in a lack of training sets, making the model unable to adapt to various scenarios.

    This paper proposes solutions to the above three problems respectively. The final model can effectively improve both the root mean square error (RMAE) and the structural similarity index (SSIM) of the shaded and non-shaded regions.

    中文摘要 i Abstract iv 誌謝 v 圖目錄 viii 表目錄 ix 第一章 緒論 1 1.1研究動機與目的 1 1.2相關研究 2 第二章 系統架構與硬體規格 4 2.1系統架構 4 2.2硬體規格 4 第三章 深度學習介紹 5 3.1深度學習 5 3.2卷積神經網路 5 3.2.1卷積層(Convolution Layer) 5 3.2.2池化層(Pooling Layer) 6 3.2.3全連接層(Fully Connect Layer) 6 第四章 陰影去除研究介紹 8 4.1 基於深度學習利用照明模型實現陰影去除 8 4.1.1 理論 8 4.1.2 Shadow removal Framework 10 4.2 SynShadow 13 第五章 研究方法 17 5.1顏色不一致性 17 5.2陰影邊緣明顯問題 20 5.3資料集不足問題 21 5.4訓練流程 22 第六章 實驗結果 23 6.1資料集介紹 23 6.1.1 USR資料集(Unpaired Shadow Removal Dataset) 23 6.1.2 ISTD(Image Shadow Triplets dataset) 24 6.1.2 AISTD(Adjust Image Shadow Triplets dataset) 24 6.2評估指標 25 6.2.1 均方根誤差(Root Mean Square Error, RMSE) 25 6.2.2 結構相似性(Structural similarity index, SSIM) 25 6.3實驗結果 26 第七章 結論 29 參考文獻 30

    [1] H. Le and D. Samaras, “Shadow Removal via Shadow Image Decomposition,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Oct. 2019, pp. 8577–8586. doi: 10.1109/ICCV.2019.00867.
    [2] N. Inoue and T. Yamasaki, “Learning from Synthetic Shadows for Shadow Detection and Removal,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 11, pp. 4187–4197, Nov. 2021, doi: 10.1109/TCSVT.2020.3047977.
    [3] C. Xiao, R. She, D. Xiao, and K.-L. Ma, “Fast Shadow Removal Using Adaptive Multi-Scale Illumination Transfer: Shadow Removal by Multi-Scale Illumination Transfer,” Computer Graphics Forum, vol. 32, no. 8, pp. 207–218, Dec. 2013, doi: 10.1111/cgf.12198.
    [4] R. Guo, Q. Dai and Derek Hoiem, “Paired Regions for Shadow Detection and Removal,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, Issue 12, Dec. 2013, p. 2956-2967.
    [5] L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, Jul. 2017, pp. 2308–2316. doi: 10.1109/CVPR.2017.248.
    [6] I. J. Goodfellow et al., “Generative Adversarial Networks,” arXiv, Jun. 10, 2014. Accessed: Jun. 04, 2022. [Online]. Available: http://arxiv.org/abs/1406.2661
    [7] X. Hu, Y. Jiang, C.-W. Fu, and P.-A. Heng, “Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2019, pp. 2472–2481. doi: 10.1109/ICCV.2019.00256.
    [8] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks,” arXiv, Aug. 24, 2020. Accessed: Jun. 04, 2022. [Online]. Available: http://arxiv.org/abs/1703.10593
    [9] J. Wang, X. Li, L. Hui, and J. Yang, “Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal,” arXiv, Dec. 06, 2017. Accessed: Jun. 06, 2022. [Online]. Available: http://arxiv.org/abs/1712.02478
    [10] Y. Shor and D. Lischinski, “The Shadow Meets the Mask: Pyramid-Based Shadow Removal,” Computer Graphics Forum, vol. 27, no. 2, pp. 577-586, Apr. 2008, doi: 10.1111/j.1467-8659.2008.01155.x.
    [11] X. Cun, C.-M. Pun, and C. Shi, “Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN,” arXiv, Nov. 20, 2019. Accessed: Jun. 04, 2022.
    [Online]. Available: http://arxiv.org/abs/1911.08718
    [12] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated Residual Transformations for Deep Neural Networks,” arXiv, Apr. 10, 2017. Accessed: Jun. 04, 2022. [Online]. Available: http://arxiv.org/abs/1611.05431
    [13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” arXiv, Dec. 10, 2015. Accessed: Jun. 04, 2022. [Online]. Available: http://arxiv.org/abs/1512.03385
    [14] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” arXiv, May 18, 2015. Accessed: Jun. 05, 2022. [Online]. Available: http://arxiv.org/abs/1505.04597
    [15] H. Zhang et al., “ResNeSt: Split-Attention Networks,” arXiv, Dec. 30, 2020. Accessed: Jun. 05, 2022. [Online]. Available: http://arxiv.org/abs/2004.08955
    [16] X. Hu, C.-W. Fu, L. Zhu, J. Qin, and P.-A. Heng, “Direction-aware Spatial Context Features for Shadow Detection and Removal,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 11, pp. 2795–2808, Nov. 2020, doi: 10.1109/TPAMI.2019.2919616.
    [17] X. Qin, Z. Zhang, C. Huang, C. Gao, M. Dehghan, and M. Jagersand, “BASNet: Boundary-Aware Salient Object Detection,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, Jun. 2019, pp. 7471–7481. doi: 10.1109/CVPR.2019.00766.
    [18] L. Fu et al., “Auto-Exposure Fusion for Single-Image Shadow Removal,” arXiv, Apr. 12, 2021. Accessed: Jun. 06, 2022. [Online]. Available: http://arxiv.org/abs/2103.01255
    [19] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. on Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004, doi: 10.1109/TIP.2003.819861.
    [20] Y. Jin, A. Sharma, and R. T. Tan, “DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, Oct. 2021, pp. 5007–5016. doi: 10.1109/ICCV48922.2021.00498.
    [21] L. Zhu et al., “Bidirectional Feature Pyramid Network with Recurrent Attention Residual Modules for Shadow Detection,” in Computer Vision – ECCV 2018, vol. 11210, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds. Cham: Springer International Publishing, 2018, pp. 122-137. doi: 10.1007/978-3-030-01231-1_8.

    QR CODE