風格轉換是透過解析風格圖片，像是藝術作品或繪畫，將其創作 元素生成到指定的目標圖上，使目標圖片擁有與風格圖片相似的風格。 L.A.Gatys 最早提出以卷積神經網路做風格轉換之方法，然而，此方 法每次風格轉換都需重新訓練生成圖片，故基於風格庫之風格轉換 (StyleBank)方法被提出，在原方法前加入轉換網路，訓練網路參數， 供之後風格轉換重複使用，而這轉換網路使用風格濾波將風格另外加 入，成功地分離出風格特徵和內容，並具有彈性和擴充性，在風格轉 換時可以動態選擇欲轉換之風格的風格濾波。
本論文針對基於風格庫之風格轉換模型進行改進和分析，一是發 現在目標圖片淺色區會出現棋盤效應，會影響生成的品質，改用最近 鄰插值縮放後卷積取代原先的反卷機，成功消除棋盤效應。二是透過 視覺化卷積濾波器後，發現有高度相似性，故透過減少濾波器個數來 減少網路複雜性，減少了 12.1%的訓練時間和 42.8%的轉換時間。最 後，對風格濾波做高斯混合分群，獲得風格主要元素的均值，重新生 成圖片，達到風格抽象化。

Style transfer is a technique for extracting stylish elements from a style image, such as art or painting, and reproduce these styles on a specified target image. L.A. Gays proposed a convolutional network that may transfer the styles of the style image onto the target image effectively, but it requires a lot of computations for style transfer of each image. To deal with this issue, StyleBank model was proposed to facilitate the process of style transfer, in which the style features could be separated from the content and incorporated into the style filters. This model is flexible and scalable because the style filters may be selected or extended dynamically.
In this study the StyleBank model is investigated, improved, and analyzed. First, it was found the StyleBank model is apt to produce checkerboard effect on light-color background, which degrades the quality of the synthesized image. An approach of nearest-neighbor interpolation was proposed in the deconvolution to eliminate the checkerboard effect and improve the image quality. In addition, through the visualization of the convolutional filters, it could be found some filters are highly similar and might be redundant. A pruning scheme is therefore proposed to reduce the network complexity and improve the speed, which can reduce the learning time and synthesis time by about 12.1% and 42.8%, respectively. Furthermore, a clustering algorithm based on Gaussian mixture models was conducted on the style filters so as to obtain the main averaged stylish filters. Based on the analysis, the target image may be decomposed into different views, and resynthesized into new style image with style
abstraction.

[1] https://www.cw.com.tw/article/article.action?id=5075522
[2] krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks," In Advances in neural information processing systems, pages 1097-1105, 2012
[3] https://mropengate.blogspot.com/2017/02/deep-learning-role-of-activation.html
[4] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," ArXiv e-prints, Sep. 2014
[5] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller, "Striving for Simplicity: The All Convolutional Net," ArXiv e-prints, Apr. 2015
[6] Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pascal Vincent, "Visualizing High-Layer Features of a Deep Network"
[7] https://prisma-ai.com
[8] Amir Semmo, Daniel Limberger, Jan Eric Kyprianidis, and Jürgen Döllner, "Image stylization by interactive oil paint filtering," Computers & Graphics 55, 157 – 171, 2015
[9] YiChang Shih, Sylvain Paris, Connelly Barnes, William T. Freeman, and Fredo Durand, "Style transfer for headshot portraits. ACM Trans," Graph. 33, 4, Article 148, 2014
[10] L. A. Gatys, A. S. Ecker, and M. Bethge, "Texture Synthesis using Convolutional Neural Networks," in Advances in Neural Information Processing Systems, pp. 262-270, 2015
[11] L. A. Gatys, A. S. Ecker, and M. Bethge, "A Neural Algorithm of Artistic Style," ArXiv e-prints, Aug. 2015
[12] Justin Johnson, Alexandre Alahi, Li Fei-Fei, "Perceptual Losses for Real-Time Style Transfer and Super-Resolution ," ArXiv e-prints, Mar. 2016
[13] 郭宗憲。結合局部相關損失之區域導向圖片風格轉換研究。碩士論文，臺灣 科 技 大 學 資 訊 管 理 研 究 所 ， 2018 。 https://ndltd.ncl.edu.tw/cgi- bin/gs32/gsweb.cgi/ccd=44cmY2/record?r1=1&h1=0。
51
[14] Dongdong Chen, Lu Yuan, Jling Liao, Nenghai Yu, Gang Hua, "StyleBank: An Explicit Representation for Neural Image Style Transfer," ArXiv e-prints, Mar. 2017
[15] https://distill.pub/2016/deconv-checkerboard/
[16] https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation
[17] Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, Jan Kautz, "Pruning Convolutional Neural Networks for Resource Efficient Inference," ArXiv e-prints, Jun. 2017