本論文目標為設計一部仿真人臉部表情動作之智慧型機器頭顱，配合樂譜辨識與語音合成的功能，使機器頭顱能展現出看譜唱歌的功能，也能使用CCD捕捉人臉影像並複製表情動作。首先先參考人類臉部的運動與肌肉組織，定義出機器頭顱外表皮膚內部的控制點來控制機器人臉部的表情，之後利用機器頭顱眼睛中的攝影機擷取畫面中的音樂簡譜，並將簡譜中的歌曲資訊文字辨識後輸出至語音合成軟體，合成出接近人類發出的中文歌聲，最後配合嘴型控制軟體，使機器頭顱唱歌時具有與真人相似的嘴型動作。在人臉偵測定位部分為使用膚色資訊與臉上幾何特徵來選擇出候選區域，並使用倒傳遞類神經網路作人臉與非人臉之分類，最後再利用訓練好的倒傳遞類神經網路來判別人臉表情的情緒。該機器頭顱後來整合雙足人形機器人成為仿真人臉之仿真人機器人(Android)，並實際在機器人劇場中演出娛樂節目。

This dissertation aims to devise a robotic head with human-like face and skin (face robot) that can read randomly composed simplified musical notation and sing out the corresponding contents of the song. The face robot can use CCD cameras to recognize the human faces and mimic their facial expressions. The face robot is composed of an artificial facial skin that can express a number of facial expressions via motions driven by internal servo motors. Two cameras, each installed inside an eyeball of the face, provide vision capability for reading the simplified music notation. Computer vision techniques are subsequently used to interpret the simplified musical notation and lyrics of the song. Voice synthesis techniques are then implemented to enable the robot to sing out the song. The mouth patterns of the face robot will automatically change to match the lyrics of the song. Experimental show that the face robot can successfully read and sing the song a high percentage of the time. Human skin information and geometric features are used to select face candidates, and a three-layered back-propagation neural network is used to classify faces and non-faces from face candidates. Finally, a back-propagation neural network after training is used to interpret facial expressions. The android head is then integrated with a bipedal robot to form an android that later performed a number of shows on a robot theater.

[1]B.C. Yoseph, “Actuation of biologically inspired intelligent robotics using articial muscles,” International journal of industrial robot, vol. 30, no.4, pp. 331-337, 2003.
[2]F. Hara, “Artificial emotion of face robot through learning in communicative interactions with human,” RO-MAN04 plenary lecture, Kurashiki Prve. pp. 7-15, 2004.
[3]Y. Bar-Cohen, “Electric flex,” IEEE Spectrum, vol. 41, no. 6, pp. 29–33, 2004.
[4]K. Kahler, J. Haber and H.P. Seidel, “Geometry-based muscle modeling for facial animation,” In proc. graphics interface, pp. 37–46, 2001.
[5]C. Breazeal, and J. Velasquez, “Toward teaching a robot `infant' using emotive communication acts,” Proceedings of 1998 simulation of adaptive behavior, workshop on socially situated intelligence, pp. 25-40, 1998.
[6]K. Berns and J. Hirth, “Control of facial expressions of the humanoid robot head ROMAN,” Proceedings of the 2006 IEEE/RSJ international conference on intelligent robots and systems, pp. 3119-3124, 2006.
[7]H. Kim, G. York and G. Burton, “Design of an anthropomorphic robot head for studying autonomous development and learning,” Proceedings of the 2004 IEEE international conference on robotics and automation, pp. 3506-3511, 2004.
[8]W. Weiguo, M. Qingmei and W. Yu, “Development of the humanoid head portrait robot system with flexible face and expression,” Proceedings of the 2004 IEEE international conference on robotics and biomimetics, pp. 757-762, 2004.
[9]F. Hara and K. Endo, “Dynamic control of lip-configuration of a mouth robot for Japanese vowels,” Journal of robotic and autonomous systems, vol. 31, pp. 161-169, 2000.
[10]A. Ogiso, S. Kurokawa and M. Yamanaka, “Expression of emotion in robots using a flow of artificial consciousness,” Proceedings of 2005 IEEE symposium on computational intelligence in robotic and automation, pp. 421-426, 2005.
[11]J. Lou and T. Tan, “3-D model-based vehicle tracking,” IEEE transactions on image processing, vol. 14, no.10, pp. 1561-1569, 2005.
[12]D.J. Bullock and J.S. Zelek, “Real-time tracking for visual interface applications in cluttered and occluding situations,”　Image and vision computing, pp.1083–1091, 2004.
[13]F. Smeraldi and O.C. Saccadic, “Search with gabor features applied to eye detection and real-time head tracking,” Image and vision computing, pp. 323–329, 2000.
[14]N.R. Howe, “Silhouette lookup for monocular 3D pose tracking,” Image and vision computing, pp. 331–341, 2007.
[15]M. Fujita and H. Kimo, “Development of an quadruped robot for robot entertainment,” Autonomous robots, vol. 5, pp. 7-18, 1998.
[16]M. Fujita, K. Sabe, Y. Kumki, T. Ishida and T. Doi, “SDR-4X II: A small humanoid as an entertainer in home environment,” International symposium of robotics research, Siena, Italy, Oct. 2003.
[17]Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki and K. Fujimura, “The intelligent ASIMO: system overview and integration,” IEEE/RSJ international conference on intelligent robots and system, vol. 3, pp.2478-2483, 2002.
[18]Y. Hosoda, S. Egawa, J. Tamamoto, K. Yamamoto, R. Nakamura and M. Togami, “Basic design of human-symbiotic robot EMIEW,” IEEE/RSJ international conference on intelligent robots and systems, pp. 5079-5084, 2006.
[19]R. Hiura, K. Onishi, H. Okazaki and S. Asano, “Development and demonstrate of receptionist robot "Wakamaru",” Nippon kikai gakkai robotikusu, vol.2006, pp.1A1-E02, 2006.
[20]J. Oh, D. Hanson, W. Kim, I. Han, J. Kim and I. Park, “Design of android type humanoid robot Albert HUBO,” IEEE/RSJ International conference on intelligent robots and systems, Beijin, China, 2006.
[21]J. Hirth, N. Schmitz and K. Berns, “Emotional architecture for the humanoid robot head roman,” IEEE International conference on robotics and automation, Rome, Italy, April 11-13, pp. 2150–2155, 2007.
[22]D. Hanson, “Expanding the aesthetics possibilities for humanlike robots,” IEEE humanoid robotics conference, special session on the Uncanny Valley, Tskuba, Japan, 2005.
[23]M. Blow, K. Dautenhahn, A. Appleby, C. L. Nehaniv and D. Lee, “The art of designing robot faces - dimensions for human-robot interaction,” Proceedings of HRI06, pp. 331-332, 2006.
[24]H.S. Lee, J.W. Park and M.J. Chung, “A linear affect–expression space model and control points for mascot-type facial robots,” IEEE transactions on robotics, vol. 23, no. 5, pp. 863-873, 2007.
[25]Successful Development of a Robot with Appearance and Performance Similar to Humans, Available: http://www.aist.go.jp/ aist_e/latest_research/2009/20090513/20090513.html, 2009.
[26]Toyota Partner Robots, Available: http://www.toyota.co.jp/en/special /robot/, 2008
[27]ASL at ETH Zurich Robot Theater Project, Available: http:// asl.epfl.ch/research/projects/RobotsTheatre/index.html, 2009.
[28]W. R. Johannes, Y. Chihiro and L. D. Elke, Color atlas of anatomy: a photografic study of the human body, Williams & Wilkins, Fourth Edition, 1999.
[29]簡志忠，光學式瞳位追蹤器之人機介面系統研製，逢甲大學自動控制工程研究所碩士論文，1997。
[30]張志承，植基於電腦視覺之樂譜辨識系統，國立台灣科技大學自動化及控制研究所碩士學位論文，2008。
[31]廖皇量，國語歌聲合成信號品質改進之研究，國立台灣科技大學資訊工程系碩士學位論文，2005。
[32]S. W. Foo, Y. Lian and L. Dong, “Recognition of visual speech elements using adaptively boosted hidden markov models,” IEEE transactions on circuits and systems for video technology, vol. 14, no. 5, pp. 693-705, 2004.
[33]B. Tiddeman and D. Perrett, “Prototyping and transforming visemes for animated speech,” Proceedings of computer animation, pp. 248-251, 2002.
[34]Ministry of Education, Available: http://www.edu.tw, 2007.
[35]林明德，由複雜背景影像中快速進行特定人物辨識之方法，國立台灣科技大學機械工程系碩士學位論文，2004。
[36]邱培文，人臉表情之視覺辨識技術，國立台灣科技大學機械工程系碩士學位論文，2006。
[37]蘇芳生，人臉表情辨識系統，國立中正大學通訊工程研究所碩士論文，2004。
[38]吳明衛，自動化臉部表情分析系統，國立成功大學資訊工程系碩士論文，2003。
[39]Y. Ogura, K. Shimomura, H. Kondo, A. Morishima, T. Okubo, S. Momoki, H. Lim and A. Takanishi, “Human-like walking with knee stretched, heel-contactand toe-off motion by a humanoid robot,” Proceedings of the 2006 IEEE/RSJ international conference on intelligent robots and systems, pp. 3976, 2006.
[40]M. Vukobratovic, B. Borovac, D. Surla and D. Stokic, Biped Locomotion: Dynamics, Stability, Control and Application, vol.7 of Scientific Fundamentals of Robotics, 1990.