本論文提出一套3D模擬彩妝的系統流程，使用3D掃描的方式獲得人臉未上妝的3D模型，但由於現今使用的掃描儀器影像解析度不高，導致3D模擬彩妝影像模糊，效果不佳。因此在彩妝模擬前，先搭配高階數位相機獲得高解析度影像進行修正，獲得色彩資訊較為正確的未上妝臉部模型。
本論文所提出的3D彩妝模擬方式是先擷取3D掃描模型的正面影像，將3D模型正投影成2D影像，使用一個臉部外觀模型找出此影像與上妝參考影像的特徵點，進行相應位置的彩妝上色模擬。在研究過程中發現，當上妝前後的影像膚色色彩差距過大時，會使模擬結果看起來不自然。因此提出一種新的平均臉優化方法，使彩妝的模擬不是只侷限在五官的區域內，可以使整張模擬影像更為自然。最後再將模型的2D影像重新的貼回原來的3D模型與網格資訊結合，達到3D彩妝模型的效果。

This study presents a 3D facial cosmetic simulation system to predict the color appearance of a 3D facial model after makeup. A low-resolution 3D scanner is used to scan a human face without makeup. As the resolution of the color texture is low, a high-resolution image taken by a high-end digital camera is used to raise the resolution of the color texture.
The proposed method first extracts a 2D front view image from the 3D face model and then uses a face appearance model to detect landmarks of both the no-makeup front-view image and a reference makeup image. Afterward, the color information of the reference image is assigned to the corresponding facial regions of the front-view image using image fusion technologies. However, the resulted 2D makeup simulation image doesn’t have a natural looking. To solve this problem, an optimal image fusion method based on average facial color is designed to make the predicted simulation look better. The high-resolution 2D simulation facial image finally remap to the 3D model to reach the 3D facial cosmetic simulation.

[1] Szeliski, R. (2010). Computer vision: algorithms and applications. Springer Science & Business Media.
[2] Huang, L. L., Shimizu, A., Hagihara, Y., & Kobatake, H. (2003). Face detection from cluttered images using a polynomial neural network. Neurocomputing, 51, 197-211.
[3] Dalal, N., & Triggs, B. (2005, June). Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) (Vol. 1, pp. 886-893). IEEE.
[4] Smola, A., & Vapnik, V. (1997). Support vector regression machines. Advances in neural information processing systems, 9, 155-161.
[5] Cootes, T. F., Edwards, G. J., & Taylor, C. J. (1998, June). Active appearance models. In European conference on computer vision (pp. 484-498). Springer Berlin Heidelberg.
[6] Asthana, A., Zafeiriou, S., Cheng, S., & Pantic, M. (2013). Robust discriminative response map fitting with constrained local models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3444-3451).
[7] King, D. E. (2009). Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10(Jul), 1755-1758.
[8] Murray, J., & Van Ryper, W. (2005). Wavefront OBJ File Format Summary.
[9] Bourke, P. (2009). Ply-polygon file format. Dostupné z www: http://paulbourke. net/dataformats/ply.
[10] Lavelle, J. P., Schuet, S. R., & Schuet, D. J. (2004, February). High-speed 3D scanner with real-time 3D processing. In Photonics Technologies for Robotics, Automation, and Manufacturing (pp. 179-188). International Society for Optics and Photonics.
[11] Blais, F., Picard, M., & Godin, G. (2004). Accurate 3D acquisition of freely moving objects. In Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004. (pp. 422-429).
[12] Tong, W. S., Tang, C. K., Brown, M. S., & Xu, Y. Q. (2007, October). Example-based cosmetic transfer. In Computer Graphics and Applications, 2007. PG'07. 15th Pacific Conference on (pp. 211-218). IEEE.
[13] Guo, D., & Sim, T. (2009, June). Digital face makeup by example. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 73-79). IEEE.
[14] 周靜茹,「彩妝膚色之顯示再現及白光LED之膚色喜好度研究」, 碩士論文, 國立臺灣科技大學, 台北 (2013)
[15] Huang, C. G., Huang, T. S., Lin, W. C., & Chuang, J. H. (2013). Physically based cosmetic rendering. Computer Animation and Virtual Worlds, 24(3-4), 275-283.
[16] Howse, J., Puttemans, S., Hua, Q., & Sinha, U. (2015). OpenCV 3 Blueprints. Packt Publishing Ltd.
[17] Hung-Shing Chen, Shih-Han Chen, Yen-Hsiang Chao, Ronnier Luo and Pei-Li Sun, “Applying Image-based Color Palette for Achieving High Image Quality for Displays.” Color Research and Application, Volume 39, Issue 2, pp. 154-168. 2014
[18] Beier, T., & Neely, S. (1992, July). Feature-based image metamorphosis. InACM SIGGRAPH Computer Graphics (Vol. 26, No. 2, pp. 35-42). ACM.
[19] Yang, M. H., Kriegman, D. J., & Ahuja, N. (2002). Detecting faces in images: A survey. IEEE Transactions on pattern analysis and machine intelligence, 24(1), 34-58.
[20] Abate, A. F., Nappi, M., Riccio, D., & Sabatino, G. (2007). 2D and 3D face recognition: A survey. Pattern Recognition Letters, 28(14), 1885-1906.
[21] Hsu, R. L., Abdel-Mottaleb, M., & Jain, A. K. (2002). Face detection in color images. IEEE transactions on pattern analysis and machine intelligence, 24(5), 696-706.
[22] Nars, F. (2004) “Makeup your Mind”, PowerHouse Books, Hardcover Edition.
[23] Prince, S. J. (2012). Computer vision: models, learning, and inference. Cambridge University Press.
[24] 黃日鋒、詹文鑫、陳鴻興、胡國瑞、徐道義、孫沛立、羅梅君 編著, 陳鴻興 編審, “顯示色彩工程學”, 全華圖書股份有限公司, 台北, 2011。
[25] 吳欣穎,「不同環境照度下液晶示螢幕膚色再現之研究」, 碩士論文, 世新大學, 台北 (2006)
[26] 許友嘉,「膚色相關色空間下影像相依性膚色修正方法」, 碩士論文, 國立立臺灣科技大學, 台北 (2010)
[27] 林新強,「博物館典藏繪畫之色彩品質評價」, 碩士論文, 國立臺灣科技大學, 台北 (2014)
[28] 盧霖,「開發臉部影像特徵點為中心之膚色對應演算法」, 碩士論文, 國立臺灣科技大學, 台北 (2015)
[29] 大田登 著,陳鴻興、陳詩涵 編譯,色彩工程學 與應用,第2版,台 ,全華科技圖書股份有限公司 (2007)