本論文提出以立體視覺系統(Stereo Vision System, SVS)實現人形機器人即時視覺模仿(Online Visual Imitation)人的三維運動之任務。當人站立於立體視覺系統前方平行的位置，表演一連串的三維動作，經由立體視覺鏡頭偵測人體骨架，得到關節點的三維座標資訊，並且記錄七個關節點之所有運動軌跡的座標資訊，包括頭部、雙手掌、雙手肘、雙腳掌。由於實驗所使用之人形機器人沒有動態平衡的機制，我們將動作分為上半身(手部)與下半身(腳部)兩個部份分別處理。藉由分析頭部及雙腳掌三個特徵點的座標資訊，設計適當的特徵向量，並使用支持向量機(Support Vector Machine, SVM)的演算法來分類腳部動作。手部動作則以混合式學習(Hybrid Learning, HL)的類神經網路演算法，將手部動作分為八個子工作區域進行建模，運用雙手掌及雙手肘的座標資訊，得到手部動作的位置與角度之間的關係，達到推算反運動學(Inverse Kinematic, IK)的目的。結合手部動作與腳部動作，完成人形機器人對人的三維運動之即時模仿的任務，並以實驗結果證明提出之方法的有效性及可靠性。

In this paper, the on-line visual imitation of an HR for different humans with 3-D motions is developed by the proposed visual imitation principle. First, the sequence of 3-D motion of a human is captured by a stereo vision system (SVS), which skeleton algorithm can capture and estimate 3-D coordinates of fifteen joints, i.e., head, neck, torso, left shoulder, left elbow, left hand, right shoulder, right elbow, right hand, left hip, left knee, left foot, right hip, right knee, and right foot. Because the dynamic balance of the imitator (i.e., humanoid robot (HR)) is not considered, the proposed on-line visual imitation is partitioned into upper body (UB) and lower body (LB). Based on the 3-D coordinates of head, left and right feet, eleven stable motions with developed feature vectors for the LB are classified by the proposed multi-class support vector machine (MSVM). The inverse kinematics (IK) of two pairs of (hand, elbow) of the UB are respectively approximated by eight pre-trained hybrid learning models for eight sub-work spaces of two arms. The comparisons between hybrid learning model based (HLMB) IK and ordinary IK are also discussed. Combining the classified motions of LB and the IKs for two pairs of (hand, elbow) of UB with interpolation imitates the 3-D motions of different humans. Finally, the corresponding experiments confirm the effectiveness and robustness of the proposed method.

[1] Y. Kuniyoshi, M. Inaba and H. Inoue, "Learning by watching: extracting reusable task knowledge from visual observation of human performance," IEEE Trans. Robotics and Autom., vol. 10, no. 6, pp. 799-822, Dec. 1994.
[2] A. Alissandrakis, C. L. Nehaniv and K. Dautenhahn, "Correspondence mapping induced state and action metrics for robotic imitation," IEEE Trans. Syst., Man, & Cybern., Pt. B, vol. 37, no. 2, pp. 299-307, Apr. 2007.
[3] S. Calinon, F. Guenter and A. Billard, "On learning, representing, and generalizing a task in a humanoid robot," IEEE Trans. Syst. Man & Cybern., Pt. B, vol. 37, no. 2, pp. 286-298, Apr. 2007.
[4] A. Ude, A. Gams, T. Asfour and J. Morimoto, "Task- specific generalization of discrete and periodic dynamic movement primitives," IEEE Trans. Robotics, vol. 26, no. 5, pp. 800-815, Oct. 2010.
[5] S. M. Khansari-Zadeh and A. Billard, "Learning stable nonlinear dynamical systems with Gaussian mixture models," IEEE Trans. Robotics, vol. 27, no. 5, pp. 943-957, Oct. 2011.
[6] A. Thobbi and W. Sheng, "Imitation learning of hand gestures and its evaluation for humanoid robots," IEEE Int. Conf. on Information and Automation (ICIA), Harbin, China, pp. 60-65, 2010.
[7] H. Y. Liu, W. J. Wang, R. J. Wang, C. W. Tung, P. J. Wang and I. P. Chang, "Image recognition and force measurement application in the humanoid robot imitation," IEEE Trans. Instru. and Meas., vol. 61, no. 1, pp. 149-161, Jan. 2012.
[8] J. W. Hsieh, Y. T. Hsu, H. Y. M. Liao and C. C. Chen, "Video-based human movement analysis and its application to surveillance systems," IEEE Trans. Multimedia, vol. 10, no. 3, pp. 372-384, Apr. 2008.
[9] K. Takahashi, T. Sakaguchi and J. Ohya, "Remarks on a real-time 3D human body posture estimation method using trinocular images," Proceedings of the 15th Int. Conf. on Pattern Recognition, Barcelona, Spain, pp. 693-697, 2000.
[10] C. F. Juang, C. M. Chang, J. R. Wu and D. Lee, "Computer vision-based human body segmentation and posture estimation," IEEE Trans. Syst., Man & Cybern., Pt . A, vol. 39, no. 1, pp. 119-133, Jan. 2009.
[11] V. V. Nguyen and J. H. Lee, “Full-body imitation of human motions with Kinect and heterogeneous kinematic structure of humanoid robot,” EEE/SICE International Symposium on System Integration, pp. 93-98, Kyushu University, Fukuoka, Japan, December 16-18, 2012.
[12] L. W. Chuang, C.Y. Lin and A. Cangelosi, “Learning of composite actions and visual categories via grounded linguistic instructions: humanoid robot simulations,” IEEE IJCNN2012, Brisbane Australia, Jun.10~15, 2012.
[13] X. Zhang, C. Li, W. Hu, X. Tong, S. Maybank and Y. Zhang, “Human pose estimation and tracking via parsing a tree structure based human model,” IEEE Trans. Syst. Man & Cybern., Syst., 2013 (earlier access).
[14] C. L. Hwang and B. L. Chen, "The extraction of key-posture frame of 3D motion of a human,” IEEE CIVEMSA2013, pp. 60-65, July 15-17, 2013, Milan, Italy.
[15] Y. Kim and H. Ling, “Human activity classification based on micro-Doppler signatures using a support vector machine,” IEEE Trans. Geoscience & Remote Sensing, vol. 47, no. 5, pp. 1328-1337, May 2009.
[16] M. Varewyck and J. P. Martens, “A practical approach to model selection for support vector machines with a Gaussian kernel,” IEEE Trans. Syst. Man & Cybern., Pt. B, vol. 41, no. 2, pp. 330-340, Apr. 2011.
[17] Q. Ye, Z. Han, J. Jiao and J. Liu, “Human detection in images via piecewise linear support vector machines,” IEEE Trans. Image Processing, vol. 22, no. 2, pp. 778-786, Feb. 2013.
[18] R. Tong, D. Xie, and M. Tang, “Upper body human detection and segmentation in low contrast video,” IEEE Trans. Circu. & Syst. for Video Technol., vol. 23, no. 9, pp. 1502- 1512, Sep. 2013
[19] B. Daya, S. Khawandi and P. Chauvet, “Neural network system for inverse kinematics problem in 3 DOF robotics,” IEEE Conf. on Bio-Inspired Computing: Theories and Applications, pp.1150-1155, 2010.
[20] B. Botond, N. T. Duy, C. Lehel, S. Bernhard and P. Jan, “Learning inverse kinematics with structured prediction,” IEEE Conf. on Intelligent Robots and Systems, San Francisco, USA, pp. 698-704, September 25-30, 2011.
[21] G. S. Huang, C. K. Tung, H. C. Lin and S. H. Hsiao, “Inverse kinematics analysis trajectory planning for a robot arm,” IEEE Conf. on 8th Asian Control Conference (ASCC), Kaohsiung, Taiwan, pp. 965-970, May 15-18, 2011.
[22] E. Lazarevska, “A neuro-fuzzy model of the inverse kinematics of a 4 DOF robotic arm,” IEEE 14th Int. Conf. on Modeling and Simulation, pp. 306-311, 2012.
[23] C. L. Hwang and J. Y. Huang, “Neural-network-based 3-D localization and inverse kinematics for target grasping of a humanoid robot by an active stereo vision system,” IEEE IJCNN2012, Brisbane Australia, pp. 1-8, Jun. 10~15, 2012.
[24] S. Haykin, Neural Networks and Learn Machines, Pearson Education Inc., 3rd Ed., 2009.
[25] C. Chang and C. Lin, LIBSVM: A Library for Support Vector Machines, 2001. [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm.
[26] http://kheresy.wordpress.com/2012/01/19/asus-xtion-pro-live/