陰影去除(Shadow removal)，是將具有陰影的圖片轉換為沒有陰影的圖片，目前是一項尚未成熟的技術，在去除陰影時需要考慮各種因素，包括環境、光照等等，利用傳統影像處理的方式複雜且繁瑣，效果卻不盡理想。近期隨著深度學習(Deep Learning)迅速發展，研究者提出不同的理論來處理此任務，成果獲得不少提升，但仍具部分問題需要解決。

目前基於深度學習進行陰影去除論文中大致可列為三個問題，第一是陰影去除後出現顏色不一致(Color inconsistency)，陰影區域在去除後難以恢復正確顏色，並與非陰影區域具明顯顏色差異，第二是在陰影去除後，陰影邊界明顯殘留於圖片中，第三是此領域資料集難以蒐集，導致訓練集資料匱乏，令模型無法完全適應各種場景。本論文針對上述三個問題分別提出解決方法，最終模型在評估陰影區域及非陰影區域的均方根誤差(Root mean square error, RMSE)及結構相似性(Structural similarity index, SSIM)皆獲得有效的提升。

Shadow removal, which converts images with shadows to images without shadows is currently an immature technology. When removing shadows, we need to consider various factors including lighting, environment etc. The traditional image processing method is complex and cumbersome, but the result is not ideal. Recently, with the rapid development of deep learning, various papers have proposed different theories to deal with this task, and the results have been greatly improved, but there are still some problems to be solved.

At present, there are three problems in the shadow removal paper based on deep learning. First, the problem is color inconsistency. The shadow area is difficult to restore the correct color and has a significant color difference from non-shadow areas after removal. Second, the shadow boundaries are clearly left in the image. Last, it is difficult to collect datasets in this field, resulting in a lack of training sets, making the model unable to adapt to various scenarios.

This paper proposes solutions to the above three problems respectively. The final model can effectively improve both the root mean square error (RMAE) and the structural similarity index (SSIM) of the shaded and non-shaded regions.

[1] H. Le and D. Samaras, “Shadow Removal via Shadow Image Decomposition,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), Oct. 2019, pp. 8577–8586. doi: 10.1109/ICCV.2019.00867.
[2] N. Inoue and T. Yamasaki, “Learning from Synthetic Shadows for Shadow Detection and Removal,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 11, pp. 4187–4197, Nov. 2021, doi: 10.1109/TCSVT.2020.3047977.
[3] C. Xiao, R. She, D. Xiao, and K.-L. Ma, “Fast Shadow Removal Using Adaptive Multi-Scale Illumination Transfer: Shadow Removal by Multi-Scale Illumination Transfer,” Computer Graphics Forum, vol. 32, no. 8, pp. 207–218, Dec. 2013, doi: 10.1111/cgf.12198.
[4] R. Guo, Q. Dai and Derek Hoiem, “Paired Regions for Shadow Detection and Removal,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, Issue 12, Dec. 2013, p. 2956-2967.
[5] L. Qu, J. Tian, S. He, Y. Tang, and R. W. H. Lau, “DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, Jul. 2017, pp. 2308–2316. doi: 10.1109/CVPR.2017.248.
[6] I. J. Goodfellow et al., “Generative Adversarial Networks,” arXiv, Jun. 10, 2014. Accessed: Jun. 04, 2022. [Online]. Available: http://arxiv.org/abs/1406.2661
[7] X. Hu, Y. Jiang, C.-W. Fu, and P.-A. Heng, “Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2019, pp. 2472–2481. doi: 10.1109/ICCV.2019.00256.
[8] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks,” arXiv, Aug. 24, 2020. Accessed: Jun. 04, 2022. [Online]. Available: http://arxiv.org/abs/1703.10593
[9] J. Wang, X. Li, L. Hui, and J. Yang, “Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal,” arXiv, Dec. 06, 2017. Accessed: Jun. 06, 2022. [Online]. Available: http://arxiv.org/abs/1712.02478
[10] Y. Shor and D. Lischinski, “The Shadow Meets the Mask: Pyramid-Based Shadow Removal,” Computer Graphics Forum, vol. 27, no. 2, pp. 577-586, Apr. 2008, doi: 10.1111/j.1467-8659.2008.01155.x.
[11] X. Cun, C.-M. Pun, and C. Shi, “Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN,” arXiv, Nov. 20, 2019. Accessed: Jun. 04, 2022.
[Online]. Available: http://arxiv.org/abs/1911.08718
[12] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated Residual Transformations for Deep Neural Networks,” arXiv, Apr. 10, 2017. Accessed: Jun. 04, 2022. [Online]. Available: http://arxiv.org/abs/1611.05431
[13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” arXiv, Dec. 10, 2015. Accessed: Jun. 04, 2022. [Online]. Available: http://arxiv.org/abs/1512.03385
[14] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” arXiv, May 18, 2015. Accessed: Jun. 05, 2022. [Online]. Available: http://arxiv.org/abs/1505.04597
[15] H. Zhang et al., “ResNeSt: Split-Attention Networks,” arXiv, Dec. 30, 2020. Accessed: Jun. 05, 2022. [Online]. Available: http://arxiv.org/abs/2004.08955
[16] X. Hu, C.-W. Fu, L. Zhu, J. Qin, and P.-A. Heng, “Direction-aware Spatial Context Features for Shadow Detection and Removal,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 11, pp. 2795–2808, Nov. 2020, doi: 10.1109/TPAMI.2019.2919616.
[17] X. Qin, Z. Zhang, C. Huang, C. Gao, M. Dehghan, and M. Jagersand, “BASNet: Boundary-Aware Salient Object Detection,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, Jun. 2019, pp. 7471–7481. doi: 10.1109/CVPR.2019.00766.
[18] L. Fu et al., “Auto-Exposure Fusion for Single-Image Shadow Removal,” arXiv, Apr. 12, 2021. Accessed: Jun. 06, 2022. [Online]. Available: http://arxiv.org/abs/2103.01255
[19] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. on Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004, doi: 10.1109/TIP.2003.819861.
[20] Y. Jin, A. Sharma, and R. T. Tan, “DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, Oct. 2021, pp. 5007–5016. doi: 10.1109/ICCV48922.2021.00498.
[21] L. Zhu et al., “Bidirectional Feature Pyramid Network with Recurrent Attention Residual Modules for Shadow Detection,” in Computer Vision – ECCV 2018, vol. 11210, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds. Cham: Springer International Publishing, 2018, pp. 122-137. doi: 10.1007/978-3-030-01231-1_8.