隨著資訊科技的發展，真實的影像可能遭受到有心人士的惡意竄改，進而造成嚴重的社會問題。傳統的商業影像編輯軟體，經由人工操縱的方式可以將影像中特定的目標進行複製搬移、拼接等等，從而產生出偽造的影像，但是近年來，出現了新型的且危險的偽造攻擊手法，如生成對抗網路(GAN)，這種偽造技術不需要人工手動地編輯圖片，只要將大量的訓練資料準備好，電腦就可以開始學習如何對影像進行偽造攻擊，並生成出非常逼真的偽造影像。
在本論文中，我們嘗試自行建構一個用於繁體漢字偽造偵測的大規模影像資料集，並將該偽造漢字影像資料集取名為ForgeryCC (Forgery Chinese Characters Dataset)，其中總共蒐集了超過4,000個偽造繁體漢字影像，並包含了兩種經常被使用的漢字書寫方式，分別是手寫風格與書法風格，而且這些偽造的繁體漢字影像是經由多個生成對抗網路模型所生成的，以提升該資料集的品質和多樣性。同時，我們對偽造漢字影像進行量化分析，並且提出一個基於光譜並結合深度學習的偽造偵測系統，用於判斷漢字影像的真偽。
最後，我們使用了多個具有代表性的神經網路對由GAN產生的漢字影像進行綜合的研究，實驗結果表明我們採用密集連接的卷積神經網絡(DenseNet)可以得到最好的性能，在手寫漢字上可以達到99.0%的準確度，而在書法漢字上可以達到95.0%的準確度。另外，藉由使用離散傅立葉轉換所得的光譜作為模型輸入，不只可以大幅地提升模型的偵測性能，還可以有效地泛化到未曾見過的GAN模型上，進而使偽造漢字影像偵測模型能夠擁有更好的泛化能力。

With the advancement of information technology, real images may be subjected to malicious tampering by individuals with bad intentions, leading to serious social issues. Traditional commercial image editing software allows for manipulation of images through manual operations, enabling specific objects within the image to be duplicated, moved, or spliced, thereby creating tampered images. However, in recent years, a new and dangerous form of forgery attack has emerged, known as Generative Adversarial Networks (GANs). This type of forgery technique does not require manual image editing; instead, by preparing a large amount of training data, computers can learn how to conduct forgery attacks on images and generate highly realistic forged images.
In this thesis, we attempt to construct a large-scale image dataset for detecting forged traditional Chinese characters, which we name ForgeryCC (Forgery Chinese Characters Dataset). The dataset consists of over 4,000 forged images of traditional Chinese characters, containing two commonly used styles of Chinese character writing: handwriting style and calligraphy style. These forged traditional Chinese character images are generated using multiple GAN models to enhance the quality and diversity of the dataset. Besides, we perform quantitative analysis on forged Chinese character images and propose a forgery detection system based on spectrum combined with deep learning to determine the authenticity of Chinese character images.
Finally, we use several representative neural networks to conduct comprehensive research on Chinese character images generated by GAN. The experimental results show that we can get the best performance by using a densely connected convolutional neural network (DenseNet). 99.0% accuracy can be achieved on handwriting Chinese characters, and 95.0% accuracy can be achieved on calligraphy Chinese characters. In addition, by using the spectrum obtained by discrete Fourier transform as model input, not only can greatly improve the detection performance of the model, but also can be effectively generalized to GAN models that have not been seen before, so that the forged Chinese character image detection model can have better generalization capability.

[1] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2017, pp. 2223-2232.
[2] J. Kim, M. Kim, H. Kang, and K. Lee, “U-GAT-IT: Unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation,” arXiv preprint arXiv:1907.10830, 2019.
[3] H. Tang, H. Liu, D. Xu, P. H. S. Torr, and N. Sebe, “AttentionGAN: Unpaired image-to-image translation using attention-guided generative adversarial networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 4, pp. 1972-1987, 2023.
[4] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434, 2016.
[5] Y. Mirsky and W. Lee, “The creation and detection of deepfakes: A survey,” ACM Computing Surveys, vol. 54, no. 1 pp. 1-41, 2021.
[6] Y. Tian, “Zi2zi: Master Chinese calligraphy with conditional adversarial networks,” [Online] Available: https://github.com/kaonashi-tyc/zi2zi. [Accessed Feb. 04, 2023].
[7] B. Chang, Q. Zhang, S. Pan, and L. Meng, “Generating handwritten Chinese characters using CycleGAN,” in Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, Nevada, 2018, pp. 199-207.
[8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” arXiv preprint arXiv:1406.2661, 2014.
[9] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 2017, pp. 1125-1134.
[10] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 2017, pp. 4700-4708.
[11] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, 2016, pp. 770-778.
[12] C.-L. Liu, F. Yin, D.-H. Wang and Q.-F. Wang, “CASIA online and offline Chinese handwriting databases,” in Proceedings of the IEEE International Conference on Document Analysis and Recognition, Beijing, China, 2011, pp. 37-41.
[13] F. Marra, D. Gragnaniello, L. Verdoliva, and G. Poggi, “Do GANs leave artificial fingerprints?” in Proceedings of the IEEE Conference on Multimedia Information Processing and Retrieval, San Jose, California, 2019, pp. 506-511.
[14] N. Yu, L. Davis, and M. Fritz, “Attributing fake images to GANs: Learning and analyzing GAN fingerprints,” in Proceedings of the IEEE International Conference on Computer Vision, Seoul, South Korea, 2019, pp. 7556-7566.
[15] X. Zhang, S. Karaman, and S.-F. Chang, “Detecting and simulating artifacts in GAN fake images,” in Proceedings of the IEEE International Workshop on Information Forensics and Security, Delft, Netherlands, 2019, pp. 1-6.
[16] C.-S. Fahn and T.-C. Wu, “A deep-neural-network-based approach to detecting forgery images generated from various generative adversarial networks,” in Proceedings of the IEEE International Conference on Machine Learning and Cybernetics, Toyama, Japan, 2022, pp. 115-123.
[17] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, 2016, pp. 2818-2826.
[18] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 2017, pp. 1251-1258.
[19] F. Marra, D. Gragnaniello, D. Cozzolino, and L. Verdoliva, “Detection of GAN-generated fake images over social networks,” in Proceedings of the IEEE Conference on Multimedia Information Processing and Retrieval, Miami, Florida, 2018, pp. 384-389.
[20] L. Verdoliva, “Media forensics and deepfakes: an overview,” IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 5, pp. 910-932, 2020.
[21] D. Gragnaniello, D. Cozzolino, F. Marra, G. Poggi, and L. Verdoliva, “Are GAN generated images easy to detect? A critical analysis of the state-of-the-art,” in Proceedings of the IEEE International Conference on Multimedia and Expo, Shenzhen, China, 2021, pp. 1-6.
[22] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “CycleGAN and pix2pix in Pytorch,” [Online] Available: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix. [Accessed Feb. 06, 2023].
[23] J. Kim, M. Kim, H. Kang, and K. Lee, “U-GAT-IT — official Pytorch implementation,” [Online] Available: https://github.com/znxlwm/UGATIT-pytorch. [Accessed Feb. 06, 2023].
[24] H. Tang, H. Liu, D. Xu, P. H. S. Torr, and N. Sebe, “AttentionGAN for unpaired image-to-image translation,” [Online] Available: https://github.com/Ha0Tang/AttentionGAN. [Accessed Feb. 06, 2023].
[25] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training GANs,” arXiv preprint arXiv:1606.03498, 2016.
[26] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “GANs trained by a two time-scale update rule converge to a local Nash equilibrium.” arXiv preprint arXiv: 1706.08500, 2017.
[27] M. Bińkowski, D. J. Sutherland, M. Arbel, and A. Gretton, “Demystifying MMD GANs,” arXiv preprint arXiv: 1801.01401, 2018.
[28] S. Barratt and R. Sharma. “A note on the inception score,” arXiv preprint arXiv:1801.01973, 2018.
[29] A. Krizhevsky, I. Sutskever, and G. E. Hinton. “ImageNet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84-90, 2017.