本論文提出以具有伺服雲台的RGB-D視覺系統實現人形機器人即時視覺模仿人類的三維運動之任務。當人站立於視覺系統前方平行的位置，表演一連串的三維動作，經由視覺系統偵測人體骨架，獲得七個關節點(即頭部、雙手掌、雙手肘、雙腳掌)的三維座標。由於本研究未設計人形機器人的動態平衡機制，而將其動作分為上半身(即手部)與下半身(即腳部)分別處理。藉由分析頭部及雙腳掌三個座標，設定適當的特徵向量，以易於區分腳部的十一類動作，並以支持向量機(SVM)演算法分類此腳部動作。而手部動作則以混合式學習的類神經網路演算法，將手部動作分為八個子工作區域進行建模。首先，收集雙手掌及手肘座標及其相對應的八個馬達角度，接著建立此反運動學(IK)的類神經網路模式。由於RGB-D視覺系統位於人形機器人的頭部，在即時的模仿時會造成捕捉人類的三維動作的困難，例如，影像歪斜、局部失落、運動所造成的模糊，故本論文將設計具有伺服雲台的RGB-D視覺系統，以確保其俯仰及滾轉方位能隨機器人動作即時修正，以獲得適當影像利於視覺模仿任務之執行。最後，結合手部動作與腳部動作，完成中型人形機器人對人類的三維運動之即時模仿的任務。並以實驗驗證所建議方法之可行性及有效性。

Humanoid robot with servo-cradle-head RGB-D system for the on-line visual imitation of 3D human motion is developed. The human in the face of humanoid robot (HR) performs 3D motion, which is captured by the RGB-D system to obtain seven joints (i.e., head, two hands, two elbows, and two feet). Since the dynamic balance of the HR is not considered, the proposed on-line visual imitation is divided into two parts, lower body (LB) and upper body (UB). Eleven stable motions of LB with the developed feature vector based on the 3-D coordinates of head, left and right feet are classified by the proposed modified multi-class support vector machine (MMSVM). The imitation of UB is based on the inverse kinematics (IK) of two pairs of (hand, elbow). To enhance one-to-one mapping and to reduce the modeling complexity of IK, two arms of UB are partitioned into eight sub-work spaces, and each one is approximated by a pre-trained hybrid learning model. Combining the classified motion of LB with the operated IK motion of UB accomplishes the task of imitating the 3-D motions of a human. Since the RGB-D system located on the head of HR, the image capturing for 3D motion, e.g., the askew of visual window, partial loss target image, and the blurring image caused by the motion of HR, will be difficult. Hence, the HR with servo-cradle-head is employed to guarantee the pitching and rolling directions to maintain the upright visual window for obtaining the correct image of human motion. Finally, the corresponding experiments are presented to confirm the effectiveness and practicality of the proposed method.

[1] Y. Kuniyoshi, M. Inaba, and H. Inoue, "Learning by watching: extracting reusable task knowledge from visual observation of human performance," IEEE Trans. Robotics and Autom., vol. 10, no. 6, pp. 799-822, Dec. 1994.
[2] A. Alissandrakis, C. L. Nehaniv, and K. Dautenhahn, "Correspondence mapping induced state and action metrics for robotic imitation," IEEE Trans. Syst., Man, & Cybern., Pt. B, vol. 37, no. 2, pp. 299-307, Apr. 2007.
[3] S. Calinon, F. Guenter, and A. Billard, "On learning, representing, and generalizing a task in a humanoid robot," IEEE Trans. Syst. Man & Cybern., Pt. B, vol. 37, no. 2, pp. 286-298, Apr. 2007.
[4] A. Ude, A. Gams, T. Asfour, and J. Morimoto, "Task- specific generalization of discrete and periodic dynamic movement primitives," IEEE Trans. Robotics, vol. 26, no. 5, pp. 800-815, Oct. 2010.
[5] S. M. Khansari-Zadeh and A. Billard, "Learning stable nonlinear dynamical systems with Gaussian mixture models," IEEE Trans. Robotics, vol. 27, no. 5, pp. 943-957, Oct. 2011.
[6] A. Thobbi and W. Sheng, "Imitation learning of hand gestures and its evaluation for humanoid robots," IEEE Int. Conf. on Information and Automation (ICIA), Harbin, China, pp. 60-65, 2010.
[7] H. Y. Liu, W. J. Wang, R. J. Wang, C. W. Tung, P. J. Wang, and I. P. Chang, "Image recognition and force measurement application in the humanoid robot imitation," IEEE Trans. Instru. and Meas., vol. 61, no. 1, pp. 149-161, Jan. 2012.
[8] J. W. Hsieh, Y. T. Hsu, H. Y. M. Liao, and C. C. Chen, "Video-based human movement analysis and its application to surveillance systems," IEEE Trans. Multimedia, vol. 10, no. 3, pp. 372-384, Apr. 2008.
[9] K. Takahashi, T. Sakaguchi, and J. Ohya, "Remarks on a real-time 3D human body posture estimation method using trinocular images," Proceedings of the 15th Int. Conf. on Pattern Recognition, Barcelona, Spain, pp. 693-697, 2000.
[10] C. F. Juang, C. M. Chang, J. R. Wu, and D. Lee, "Computer vision-based human body segmentation and posture estimation," IEEE Trans. Syst., Man & Cybern., Pt . A, vol. 39, no. 1, pp. 119-133, Jan. 2009.
[11] V. V. Nguyen and J. H. Lee, “Full-body imitation of human motions with Kinect and heterogeneous kinematic structure of humanoid robot,” IEEE/SICE International Symposium on System Integration, pp. 93-98, Kyushu University, Fukuoka, Japan, December 16-18, 2012.
[12] L. W. Chuang, C.Y. Lin, and A. Cangelosi, “Learning of composite actions and visual categories via grounded linguistic instructions: humanoid robot simulations,” IEEE IJCNN2012, Brisbane Australia, Jun.10~15, 2012.
[13] X. Zhang, C. Li, W. Hu, X. Tong, S. Maybank, and Y. Zhang, “Human pose estimation and tracking via parsing a tree structure based human model,” IEEE Trans. Syst. Man & Cybern.: Syst., vol. 44, no. 5, pp. 580-592, May 2014.
[14] C. L. Hwang and B. L. Chen, "The extraction of key-posture frame of 3D motion of a human,” IEEE CIVEMSA2013, pp. 60-65, July 15-17, 2013, Milan, Italy.
[15] Y. Kim and H. Ling, “Human activity classification based on micro-Doppler signatures using a support vector machine,” IEEE Trans. Geoscience & Remote Sensing, vol. 47, no. 5, pp. 1328-1337, May 2009.
[16] M. Varewyck and J. P. Martens, “A practical approach to model selection for support vector machines with a Gaussian kernel,” IEEE Trans. Syst. Man & Cybern., Pt. B, vol. 41, no. 2, pp. 330-340, Apr. 2011.
[17] Q. Ye, Z. Han, J. Jiao, and J. Liu, “Human detection in images via piecewise linear support vector machines,” IEEE Trans. Image Processing, vol. 22, no. 2, pp. 778-786, Feb. 2013.
[18] R. Tong, D. Xie, and M. Tang, “Upper body human detection and segmentation in low contrast video,” IEEE Trans. Circu. & Syst. for Video Technol., vol. 23, no. 9, pp. 1502- 1512, Sep. 2013
[19] B. Daya, S. Khawandi, and P. Chauvet, “Neural network system for inverse kinematics problem in 3 DOF robotics,” IEEE Conf. on Bio-Inspired Computing: Theories and Applications, pp.1150-1155, 2010.
[20] B. Botond, N. T. Duy, C. Lehel, S. Bernhard, and P. Jan, “Learning inverse kinematics with structured prediction,” IEEE Conf. on Intelligent Robots and Systems, San Francisco, USA, pp. 698-704, September 25-30, 2011.
[21] G. S. Huang, C. K. Tung, H. C. Lin, and S. H. Hsiao, “Inverse kinematics analysis trajectory planning for a robot arm,” IEEE Conf. on 8th Asian Control Conference (ASCC), Kaohsiung, Taiwan, pp. 965-970, May 15-18, 2011.
[22] E. Lazarevska, “A neuro-fuzzy model of the inverse kinematics of a 4 DOF robotic arm,” IEEE 14th Int. Conf. on Modeling and Simulation, pp. 306-311, 2012.
[23] C. L. Hwang and J. Y. Huang, “Neural-network-based 3-D localization and inverse kinematics for target grasping of a humanoid robot by an active stereo vision system,” IEEE IJCNN2012, Brisbane Australia, pp. 1-8, Jun. 10~15, 2012.
[24] S. Haykin, Neural Networks and Learn Machines, Pearson Education Inc., 3rd Ed., 2009.
[25] C. Chang and C. Lin, LIBSVM: A Library for Support Vector Machines, 2001. [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm.
[26] http://kheresy.wordpress.com/2012/01/19/asus-xtion-pro-live/