本研究整合雙臂機器人和ZED雙目相機，使機器人左右臂可利用影像系統辨識物件並達成協作目的，可規劃為居家型服務機器人之基本功能。整體平台的控制核心包含可程式化邏輯陣列 (FPGA) 和高效能運算電腦，雙臂系統中將每顆馬達的控制系統模組化，如此可以快速除錯，且更易於觀察控制時各軸所產生的振盪現象。
本研究中所用之影像辨識系統，是由高效能運算電腦搭配雙目立體相機 (ZED Stereo Camera) 的SDK，以OpenCV的函式庫進行影像校正、HSV色彩空間轉換、形態學處理、影像輪廓搜尋和三維座標轉換，判斷出該杯子的位置，再經過影像資訊的計算後，回傳該杯子的三維坐標夾取點到FPGA的運動控制系統，操控機器手臂移動至正確位置並順利夾取物體。
其中高效能運算電腦和FPGA間的通訊是通過非同步收發傳輸器 (UART) 回傳機械手臂點到點間的路徑規劃，是使用FPGA內之模糊滑動控制器 (FSMC) 作運動控制，使各馬達到達的位置軌跡能追蹤依操作目標所規劃之軌跡，以順利夾取杯耳進行雙臂的合作。

In this research, a ZED Binocular Stereo Camera is installed near a dual-arm robot as the eyes of the robot system for image recognition. This integrating system can be planned as a home-type service robot whose left and right arm can collaborate with each other. The control kernel of the overall system includes Field Programmable Gate Array, FPGA, and a high-performance computer. Each motor component and controller of the dual-arm movement system are modularized. Hence it is easy to debug the program and design control law, and easy to observe the control performance and oscillation.
In this research, high-performance computer is employed to drive ZED SDK with OpenCV library to execute the image correction, HSV color space conversion, three-dimensional coordinate transformation, and image contour search for determining the position of the cup. After a series of calculation on image information, the computer can send the three-dimensional gripping point back to FPGA motion control system. Then, the FPGA control system can operate robot to the correct position and grip the cup.
Among them, the asynchronous transceiver (UART) is used to execute at communication between the computer and FPGA system. The dual-arm robot point-to-point path planning and motion control is executed by software codes within FPGA. Fuzzy Sliding-mode Controller (FSMC) is adopted to monitor each motor’s motion control for achieving precise path tracking and minimizing the terminal point steady state error. The experimental results show that the dual-arm robot can successfully pick up the specified cup and cooperate with each other.

【1】 A. Karami, L. Jeanpierre and A. Mouaddib, "Human-robot collaboration for a shared mission," 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Osaka, Japan, 2010, pp. 155-156.
【2】 R. Rosenberg-Kima, Y. Koren, M. Yachini and G. Gordon, "Human-Robot-Collaboration (HRC): Social Robots as Teaching Assistants for Training Activities in Small Groups," 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Daegu, Korea (South), 2019, pp. 522-523.
【3】 U. B. Himmelsbach, T. M. Wendt and M. Lai, "Towards Safe Speed and Separation Monitoring in Human-Robot Collaboration with 3D-Time-of-Flight Cameras," 2018 Second IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA , 2018, pp. 197-200.
【4】 S. H. Han, J. W. Choi, K. Son, M. C. Lee, J. M. Lee and M. H. Lee, "A study on feature-based visual servoing control of eight axes-dual arm robot by utilizing redundant feature," Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164), Seoul, Korea (South) , 2001, pp. 2622-2627 vol.3.
【5】 J. Liang et al., "Dual quaternion based kinematic control for Yumi dual arm robot," 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, Korea (South), 2017, pp. 114-118.
【6】 D. I. Park, C. Park, J. Park, D. Kim, K. Park and T. Choi, "Joint module with passive compliance for dual arm rescue robot," 2014 11th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Kuala Lumpur, Malaysia, 2014, pp. 238-239.
【7】 C. Li, F. Dai, F. Li, S. Wang, X. Gao and K. Li, "Development of a dual-mode mobile robot system for practical applications," 2011 IEEE International Conference on Mechatronics and Automation, Beijing, China, 2011, pp. 1485-1490.
【8】 A. Zahavi, S. N. Haeri, D. C. Liyanage and M. Tamre, "A Dual-Arm Robot for Collaborative Vision-Based Object Classification," 2020 17th Biennial Baltic Electronics Conference (BEC), Tallinn, Estonia, 2020, pp. 1-5.
【9】 S. Zhao, B. Liu, Y. Ren and J. Han, "Color tracking vision system for the autonomous robot," 2009 9th International Conference on Electronic Measurement & Instruments, Beijing, China, 2009, pp. 3-182-3-185.
【10】 S. Fang, X. Huang, H. Chen and N. Xi, "Dual-arm robot assembly system for 3C product based on vision guidance," 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), Qingdao, China, 2016, pp. 807-812.
【11】 J. Lee, S. Kang, K. Kim, J. Kim and J. B. Kim, "A development of easily trainable vision system for the multi-purpose dual arm robots," 2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, Korea (South), 2013, pp. 609-614.
【12】 D. -K. Jeong, H. -S. Kang, D. -E. Kim and J. -M. Lee, "Mask-RCNN based object segmentation and distance measurement for Robot grasping," 2019 19th International Conference on Control, Automation and Systems (ICCAS), Jeju, Korea (South), 2019, pp. 671-674.
【13】 D. Liu, J. Cao and X. Lei, "Slabstone Installation Skill Acquisition for Dual-Arm Robot based on Reinforcement Learning," 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dali, China, 2019, pp. 1298-1305.
【14】 T. T. Hoang, P. M. Duong, N. T. T. Van , D. A. Viet and T. Q. Vinh, “Multi-sensor perceptual system for mobile robot and sensor fusion-based localization,” International Conference on Control Automation and Information Sciences (ICCAIS),Ho Chi Minh City, Vietnam, 2012.
【15】 ZED Stereo Labs, “ZED Camera and SDK Overview”, Revise
Oct 2019
【16】 ALTERA, “DE2-115 User Manual,” ALTERA Corporation, 2010
【17】 簡劭宇, “以雙輪倒單擺為移動平台之雙臂機器人,” 碩士論文, 國立臺灣科技大學機械工程系, 2021
【18】 Avago HCTL-2032, HCTL-2032-SC, HCTL-2032-SCT, HCTL-2022
Quadrature Decoder/ Counter Interface ICs, Retrieved December, 2020
【19】 IG-36 PGM 01 & 02 Type DC Motor Data sheet, Shayang Ye, Taiwan, October, 2012
【20】 IG-22 GM 01 & 02 Type DC Motor Data sheet, Shayang Ye, Taiwan, October, 2012
【21】 Maxon A-max DC Motor Data sheet, May, 2018
【22】 HEDM-55xx/560x & HEDS-55xx/56xx Quick Assembly Two and Three Channel Optical Encoders Data sheet, November, 2020
【23】 Slim Incremental 50-mm-dia. Rotary Encoder E6C2-C Data Sheet, OMRON Corporation, October, 2015
【24】 SN65175, SN75175 Quadruple Differential Line Receivers Datasheet, Revised November, 2006
【25】 M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and Control, United States: John Wiley & Sons, 2006.
【26】 黃泗堯, “移動式雙臂機器人之視覺導引裝配應用,” 碩士論文, 國立臺灣科技大學機械工程系, 2019
【27】 施慶隆、李文猶著，機電整合控制第三版：多軸運動設計與應用，全華圖書股份有限公司，2015
【28】 P. Hillman, J. Hannah and D. Renshaw, "Alpha channel estimation in high resolution images and image sequences," Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR, Kauai, HI, USA, 2001, pp. I-I.
【29】 Retrieved August 7, 2021, from https://en.wikipedia.org/wiki/RGBA
【30】 Retrieved August 7, 2021, from https://en.wikipedia.org/wiki/HSL_and_HSV
【31】 Retrieved August 7, 2021, from https://zh.wikipedia.org/wiki/ Hue
【32】 由 Samus_ - 自己的作品, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1302915
【33】 Taskin, Cem, et al. Image Processing on Android Devices With Qpencv. Fundamental Sciences and Applications, 2017, 73.
【34】 GOYAL, Megha. Morphological image processing. IJCST, India, 2011, 2.4.
【35】 S. Suzuki and K. Abe, “Topological Structural Analysis of Digitized Binary Images by Border Following”, computer vision, graphics , and image processing 30, pp. 32-46, 1985
【36】 G. Bradski and A. Kaehler, 于仕琪譯 , 劉瑞禎譯 “Learning OpenCV 中文版 ”, pp. 259-263
【37】 Douglas Peucker_algorithmGervasi O., Caprini L., Maccherani G. (2013) Virtual Exhibitions on the Web: From a 2D Map to the Virtual World. In: Murgante B. et al. (eds) Computational Science and Its Applications – ICCSA 2013. ICCSA 2013. Lecture Notes in Computer Science, vol 7971. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39637-3_56
【38】 S. Benhimane and E. Malis, “Homography based 2D Visual Tracking and
Servoing,” International Journal of Robotics R esearch, pp.661 676 , 2007
【39】 Retrieved August, 15, 2021, from https://blog.csdn.net/weixin_42505877/article/details/106100721?utm_medium=distribute.pc_relevant.none-task-blog-baidujs_title4&spm=1001.2101.3001.4242
【40】 Y. Fang and X. Chen, "Design and Simulation of UART Serial Communication Module Based on VHDL," 2011 3rd International Workshop on Intelligent Systems and Applications, Wuhan, China, 2011, pp. 1-4.
【41】 由 IngenieroLoco - 自己的作品, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=53165575
【42】 R. Palm, “Sliding mode fuzzy control”, IEEE Int. Conference Fuzzy System, San Diego, CA, pp. 519-526, Mar 1992
【43】 L. X. Wang, “Stable adaptive fuzzy control of nonlinear systems”, IEEE Trans. Fuzzy System, vol.1, pp.146-155, 1993.
【44】 R. Palm, “Robust control by fuzzy sliding mode,” Automatica, Vol. 30, No. 9, pp. 1429-1437, 1994
【45】 S.C. Lin, and Y.Y. Chen, “Design of adaptive fuzzy sliding mode control for nonlinear system control,” in Proc. 3rd IEEE Conf. Fuzzy Syst., IEEE World congress Computat. Intell., Orlando, FL, Vol.1, pp.35-39, Jun 1994
【46】 S.Y. Yi and M.J. Chung, “Robustness of Fuzzy Logic Control for an Uncertain Dynamic System”, IEEE Transactions on Fuzzy Systems, Vol.6, No.2, pp. 216-225, May 1998
【47】 巫憲欣, “以系統晶片發展具機器視覺之機械手臂運動控制,” 碩士論文, 國立臺灣科技大學機械工程系, 2006