在現今的社會裡面拍照、錄影成為了我們主要紀錄美好的回憶與重要資訊的主要方式，然而在人們對於相片與影片的品質越加要求的同時，人們發現圖像會在某些時候產生奇怪的波紋與突兀的顏色，人們開始研究後發現這是由於攝影設備中的感光元件畫素空間頻率與拍攝圖像條紋的空間頻率相近時對我們的視覺產生干擾，從而產生了這些奇怪的波紋與顏色，他們將其稱為摩爾紋。對於現今攝影設備越來越高級，但與之對應的影像品質卻被摩爾紋給破壞，這是不被接受的，所以人們開始研究如何有效地來去除這些照片影片上奇怪的波紋。在本論文中，首先我們引入了 ESDNet [14] 作為我們除摩爾紋模型的基本框架，接著我們將圖片轉換YUV 顏色空間 [1] 來訓練模型以保留較好的圖像色彩，並引入優良的特徵提取手段 [2] 來減少圖像中的摩爾紋，另外加入了特別的通道注意力機制 [3] 以保留更精細的圖像細節，最後改變了 U-net [4] 的架構，使不同階層的解碼器可以同時考慮到其他不同尺度的特徵，讓模型在學習時可以更加全面。從實驗結果得知，我們提出的方法在三個主流資料集上得到了比目前方法還要好的結果，在輸出圖片中也有效的去除摩爾紋，並同時保留圖像色彩與細節。

In today's society, taking photos and videos has become the main way for us to record good memories and important information. However, as people have more and more requirements for the quality of photos and videos. People find that images sometimes produce strange ripples and abrupt colors, people began to study and found that is because of the spatial frequency of the photosensitive element in the photographic device is similar to the spatial frequency of the captured image stripes, and it interferes with our vision to create these strange ripples and colors, which is called moiré. Today's photography equipments are getting more and more advanced, but the corresponding image quality is destroyed by moiré, which is not accepted, so people began to study how to effectively remove the strange ripples on these photos and video. In the thesis, first of all, we adopted the ESDNet [14] as the basic framework of our moiré removal model, and then we convert the image to YUV color space [1] to train the model to preserve better image color, and introduce an excellent feature extraction method [2] to reduce the moiré in the image, and also add a special channel attention mechanism [3] to retain finer image details. Finally, we changed the architecture of U-net [4], so that decoders of different levels can consider other features of different scales at the same time, so that the model can be more comprehensive in learning. The experimental results show that our proposed method has achieved better results than the state-of-the-art methods on the three mainstream datasets, and effectively removes moiré patterns in output images while preserving image colors and details.

REFERENCES

[1] E. Reinhard and T. Pouli, "Colour spaces for colour transfer," Computational Color Imaging: Third International Workshop, pp. 1-15, 2011.
[2] S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, and L. Shao, "Learning enriched features for fast image restoration and enhancement," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 2, pp. 1934-1948, 2022.
[3] S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, and L. Shao, "Multi-stage progressive image restoration," IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14821-14831, 2021.
[4] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234-241, 2015.
[5] N. Wiener, "Ideal low-pass filter." IRE Transactions on Information Theory, pp. 1-9, 1949.
[6] J. A. Fessler, "Spatial Resolution Properties of Penalized-Likelihood Image Reconstruction: Spatially Invariant Tomographs," IEEE Transactions on Image Processing, vol. 5, no. 9, pp. 1346-1358, 1996.
[7] P. J. Burt and E. H. Adelson, "A multiresolution spline with application to image mosaics," ACM Transactions on Graphics, vol. 2, no. 4, pp. 217-236, 1983.
[8] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[9] J. Wang, L. Chen, P. Chen, and X. Mao, "Moiré pattern removal with a generative adversarial network," International Conference on Graphics and Signal Processing, pp. 81-86, 2020.
[10] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," Advances in Neural Information Processing Systems, vol. 27, 2014.
[11] Z. Xu, S. Li, and W. Deng, "Learning temporal features using LSTM-CNN architecture for face anti-spoofing," Asian Conference on Pattern Recognition, pp. 141-145, 2015.
[12] Y. He, J. Sun, and X. Zhang, "Removing Moire Patterns by Guided Filter with Sparse Coding." IEEE Transactions on Image Processing, pp. 5024-5039, 2013.
[13] B. Zheng, S. Yuan, C. Yan, X. Tian, J. Zhang, Y. Sun, L. Liu, A. Leonardis, and G. Slabaugh, "Learning frequency domain priors for image demoiréing," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 7705-7717, 2021.
[14] X. Yu, P. Dai, W. Li, L. Ma, J. Shen, J. Li, and X. Qi, "Towards efficient and scale-robust ultra-high-definition image demoiréing," European Conference on Computer Vision, pp. 646-662, 2022.
[15] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," IEEE Conference on Computer Vision and Pattern Recognition, pp. 2472-2481, 2018.
[16] F. Yu and V. Koltun, "Multi-scale context aggregation by dilated convolutions," International Conference on Learning Representations, 2015.
[17] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, no. 6088, pp. 533-536, 1986.
[18] A. Thevenot, "Understand and Visualize Color Spaces to Improve Your Machine Learning and Deep Learning Models," Towards Data Science, 2019.
[19] A. Krizhevsky and G. Hinton, "Learning multiple layers of features from tiny images," 2009.
[20] K. Saarinen, "Comparison of decimation and interpolation methods in case of multiple repeated RGB-YCbCr colour image format conversions," IEEE International Symposium on Circuits and Systems, vol. 3, pp. 269-272, 1994.
[21] Y. Hua, B. Wu, and G. Wu, "A color image fragile watermarking algorithm based on DWT-DCT," IEEE Chinese Control and Decision Conference, pp. 2840-2845, 2010.
[22] H.-K. Tang and Z.-Q. Feng, "Hand's skin detection based on ellipse clustering," IEEE International Symposium on Computer Science and Computational Technology, vol. 2, pp. 758-761, 2008.
[23] T. Chai and R. R. Draxler, "Root mean square error (RMSE) or mean absolute error (MAE) – Arguments against avoiding RMSE in the literature," Geoscientific Model Development, vol. 7, no. 3, pp. 1247-1250, 2014.
[24] J. Johnson, A. Alahi, and F.-F. Li, "Perceptual losses for real-time style transfer and super-resolution," European Conference on Computer Vision, pp. 694-711, 2016.
[25] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," International Conference on Learning Representations, 2015.
[26] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and F.-F. Li, "Imagenet: A large-scale hierarchical image database," IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
[27] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
[28] I. Loshchilov and F. Hutter, "Sgdr: Stochastic gradient descent with warm restarts," International Conference on Learning Representations, 2017.
[29] S. Yuan, R. Timofte, G. Slabaugh, A. Leonardis, B. Zheng, X. Ye, X. Tian, Y. Chen, X. Cheng, Z. Fu, "Aim 2019 challenge on image demoireing: Methods and results," IEEE/CVF International Conference on Computer Vision Workshop, pp. 3534-3545, 2019.
[30] B. He, C. Wang, B. Shi, and L.-Y. Duan, "FHDe^2Net: Full High Definition Demoireing Network." European Conference on Computer Vision, pp. 713–729, 2020.
[31] A. Hore and D. Ziou, "Image quality metrics: PSNR vs. SSIM," IEEE International Conference on Pattern Recognition, pp. 2366-2369, 2010.
[32] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.
[33] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, "The unreasonable effectiveness of deep features as a perceptual metric," IEEE Conference on Computer Vision and Pattern Recognition, pp. 586-595, 2018.
[34] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in Neural Information Processing Systems, vol. 25, 2012.
[35] J. Hahn, J. Hausman, and G. Kuersteiner, "Estimation with weak instruments: Accuracy of higher‐order bias and MSE approximations," The Econometrics Journal, vol. 7, no .1, pp. 272-306, 2004.
[36] Y. Sun, Y. Yu, and W. Wang, "Moiré photo restoration using multiresolution convolutional neural networks," IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 4160-4172, 2018.
[37] X. Cheng, Z. Fu, and J. Yang, "Multi-scale dynamic feature encoding network for image demoiréing," IEEE/CVF International Conference on Computer Vision Workshop, pp. 3486-3493, 2019.
[38] L. Liu, J. Liu, S. Yuan, G. Slabaugh, A. Leonardis, W. Zhou, Q. Tian, "Wavelet-based dual-branch network for image demoiréing," European Conference on Computer Vision, pp. 86-102, 2020.
[39] B. He, C. Wang, B. Shi, and L.-Y. Duan, "Mop moire patterns using mopnet," IEEE/CVF International Conference on Computer Vision, pp. 2424-2432, 2019.
[40] B. Zheng, S. Yuan, G. Slabaugh, and A. Leonardis, "Image demoireing with learnable bandpass filters," IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3636-3645, 2020.