在眾多機器人使用的感測器中，視覺感測器為能取得最豐富資訊的一種感測器。結合機器視覺與運動控制以達到機器人全自主任務的技術一般通稱為「影像伺服」(Visual Servoing)技術，其基本原理為以當前的位置誤差或影像特徵誤差不斷更新控制命令，使機器人能順利地從初始位置到達目標位置。對於移動型機器人來說，由於其姿態在行進過程中會不斷改變，在複雜場景的情況受到許多不可預期干擾影響在實現上有其困難度，雖然已有部分學者提出進階的相機參數估測演算法以便求得適應性控制律，但其效能與強健性目前仍然有不少應用上的限制。近年來由於人工智慧技術有著突破性的應用進展，且在諸多研究領域獲得不錯的實證效果，因此機器人學者亦開始嘗試將機器學習相關方法與影像伺服相結合，期望藉此改善影像伺服系統的運動控制效能與強健性。本研究為利用基於卷積神經網路學習並求得當前影像與目標影像間的特徵關係，以估測彼此間相對的平移與旋轉關係後，再利用移動型機器人的運動學求得控制律，接著不斷地透過彼此的影像特徵進行估測，使移動型機器人順利地從初始位置移動到目標位置。本研究首先於虛擬場景中驗證此系統的可行性，之後再於真實場景驗證移動型機器人影像伺服控制效能。除此之外，本研究亦設計諸多實驗包括增加影像干擾、給予外部干擾(推力)和置於未知場景進行實驗。實驗結果證實本研究所發展的影像伺服控制系統能夠適用於未知的複雜場景中且具有良好的強健性，未來將可作為發展服務型機器人與工業型無人搬運車等機器人實務應用的良好技術基礎。

Vision sensors are commonly adopted in robotic systems because they can acquire most ample environmental information among the available robotic sensors. The technique that integrates machine vision and motion control for robots are referred to visual servoing, which is basically a method that applies Cartesian position error or image feature errors to update the motion control commands and autonomously regulate the robot from the initial position to the target position. For mobile robots that are used in complex scenes, since the pose is varying during the entire motion process, the performance and robustness of many advanced visual servoing algorithms for adaptive parameter estimation and adjustment are still limited in practical applications. However, because the rapid development of artificial intelligence has made a tremendous breakthrough and has shown many successful achievements in diversified research fields, recently several robotic scholars have attempted to develop visual servoing methods from the perspective of machine learning. In this thesis, a convolutional neural network is proposed to learn the relationship between the current image and the target image to estimate the relative translation and rotation between these two images. The control law of mobile robot is then derived using kinematics. The image features are continuously estimated by the applied neural network and are used to make the mobile robot gradually move from the initial position to the target position. Before actual implementation the study first justifies the feasibility of the developed neural network based visual servoing system by performing simulation in the constructed virtual scenes. The experimental results demonstrate the great robustness of the developed visual servoing system even adding significant image noises, perturbing the robot during movement and implementing in unknown complex scenes. This system may be applied to the development of practical service robots or automated guided vehicles for industries in the future.

[1]自動化生產線機械手臂, Available from:
https://www.7car.tw/articles/read/36565
[2]倉儲搬運機器人, Available from:
http://technews.tw/2014/10/15/quiet-logistics-warehouse/
[3]農業機器人, Available from:
https://www.zhinengl.com/2017/07/agricultural-robots/
[4]醫療機器人與科考機器人, Available from:
https://kknews.cc/zh-tw/tech/n5rek48.html
[5]iRobot Roomba 980, Available from:
https://www.momoshop.com.tw/goods/GoodsDetail.jsp?
i_code=4654281&memid=6000013245&cid=apuad&oid=1&osm=league
[6]Boston Dynamics Atlas, Available from:
https://www.bostondynamics.com/atlas
[7]Acrobotics: a revolutionary movement, Available from:
https://cosmosmagazine.com/technology/acrobotics-a-revolutionary-movement
[8]S. Hutchinson, G. D. Hager, and P. I. Corke, “A tutorial on visual servo
control,” IEEE Transactions on Robotics and Automation, vol. 12, no. 5, pp.
651-670, Oct. 1996.
[9]K. Hashimoto, “A Review on Vision-based Control of Robot Manipulators,”
Advanced Robotics, vol.17, no. 10, pp. 969-991, 2003.
[10]R. Y. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D
Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,” IEEE
Journal on Robotics and Automation, vol. 3, no. 4, pp. 323-344, Aug. 1987.
[11]Z. Zhang, “A Flexible New Technique for Camera Calibration,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11,
pp. 1330-1334, Nov. 2000.
[12]L. E. Weiss, A. C. Sanderson, and C. P. Neuman, “Dynamic Sensor-Based Control
of Robots with Visual Feedback,” IEEE Journal on Robotics and Automation,
vol. 3, no. 5, pp. 404-417, Oct. 1987.
[13]K. Ahlin et al., “Autonomous Leaf Picking Using Deep Learning and Visual-
Servoing,” IFAC-PapersOnLine, vol. 49, no. 16, pp.177-183, 2016.
[14]A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with
Deep Convolutional Neural Networks,” Advances in neural information
processing systems, pp. 1097–1105, 2012.
[15]H. Cheng et al., “Deep learning for manipulator visual positioning,” 2018
IEEE 8th Annual International Conference on CYBER Technology in Automation,
Control, and Intelligent Systems (CYBER), Tianjin, China, 2018, pp. 373-378.
[16]S. Ren et al., “Faster R-CNN: Towards Real-Time Object Detection with Region
Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.
[17]T. Lampe and M. Riedmiller, “Acquiring Visual Servoing Reaching and Grasping
Skills using Neural Reinforcement Learning,” The 2013 International Joint
Conference on Neural Networks (IJCNN), Dallas, TX, USA, 2013, pp. 1-8.
[18]F. Zhang et al., “Towards Vision-Based Deep Reinforcement Learning for
Robotic Motion Control,” Australasian Conference on Robotics and Automation
(ACRA), 2015.
[19]K. Hwang et al., “Image Base Visual Servoing base on Reinforcement Learning
for Robot Arms,” 2017 56th Annual Conference of the Society of Instrument and
Control Engineers of Japan (SICE), Kanazawa, Japan, 2017, pp. 566-569.
[20]H. Shi et al., “Decoupled Visual Servoing With Fuzzy Q-Learning,” IEEE
Transactions on Industrial Informatics, vol. 14, no. 1, pp. 241-252, Jan.
2018.
[21]H. Shi et al., “Adaptive Image-based Visual Servoing with Reinforcement
Learning for Wheeled Mobile Robots,” 2018 IEEE International Conference on
Mechatronics and Automation (ICMA), Changchun, China, 2018, pp. 954-959.
[22]C. J. C. H. Watkins and P. Dayan, “Q-Learning,” Machine Learning, vol. 8, no.
3–4, pp. 279–292, 1992.
[23]C. Sampedro et al., “Image-Based Visual Servoing Controller for Multirotor
Aerial Robots Using Deep Reinforcement Learning,” 2018 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 2018, pp.
979-986.
[24]T. P. Lillicrap et al., “Continuous control with deep reinforcement
learning,” International Conference on Learning Representations (ICLR), 2016.
[25]Q. Bateux et al., “Training Deep Neural Networks for Visual Servoing,” 2018
IEEE International Conference on Robotics and Automation (ICRA), Brisbane,
QLD, Australia, 2018, pp. 1-8.
[26]A. Saxena et al., “Exploring Convolutional Networks for End-to-End Visual
Servoing,” 2017 IEEE International Conference on Robotics and Automation
(ICRA), Singapore, 2017, pp. 3817-3823.
[27]A. Dosovitskiy et al., “FlowNet: Learning Optical Flow with Convolutional
Networks,” 2015 IEEE International Conference on Computer Vision (ICCV),
Santiago, Chile, 2015, pp. 2758-2766.
[28]B. Glocker et al., “Real-Time RGB-D Camera Relocalization,” 2013 IEEE
International Symposium on Mixed and Augmented Reality (ISMAR), Adelaide, SA,
Australia, 2013, pp. 173-179.
[29]游凱安, “針對室內個人化影像伺服操作物件之深度學習分類器開發,” 國立臺灣科技大學電機工程
學系研究所, 2017.
[30]賴彥均, “應用Q-learning於搭載機械手臂自走車系統之基於影像視覺伺服研究,” 國立成功大學
電機工程學系研究所, 2017.
[31]李佳霖, “以加強式學習實現適應性視覺伺服於機器手臂控制,” 國立中山大學電機工程學系研究
所, 2017.
[32]Caffe深度學習框架, Available from:
https://caffe.berkeleyvision.org/
[33]NVIDIA 1080Ti, Available from:
https://www.nvidia.com/en-us/geforce/products/10series/geforce-gtx-1080-ti/
[34]Microsoft LifeCam Cinema網路攝影機, Available from:
https://www.microsoft.com/zh-tw/
[35]NVIDIA 950M, Available from:
https://www.geforce.com.tw/hardware/notebook-gpus/geforce-gtx-
950m/specifications
[36]Arduino Due控制板, Available from:
https://store.arduino.cc/usa/due
[37]Faulhabr馬達, Available from:
https://www.faulhaber.com/en/home/
[38]BTS7960馬達驅動模組, Available from:
https://www.openimpulse.com/blog/
[39]Avago HCTL-2032晶片, Available from:
https://docs.broadcom.com/docs/AV02-0096EN
[40]雙向邏輯電平轉換晶片(Logic Level Converter), Available from:
https://www.taiwaniot.com.tw/
[41]D. H. Hubel and T. N. Wiesel, “Receptive fields, binocular interaction and
functional architecture in the cat's visual cortex,” J. Physiol., pp. 106-
154, Jan. 1962.
[42]K. Fukushima, S. Miyake, and T. Ito, “Neocognitron: A Neural Network Model
for a Mechanism of Visual Pattern Recognition,” IEEE Transactions on Systems,
Man, and Cybernetics, vol. SMC-13, no. 5, pp. 826-834, Sep.-Oct. 1983.
[43]D. E. Rumelhart, G. E. Hintion, and R. J. Williams, “Learning representations
by back-propagating errors,” Nature, pp. 533-536, 1986.
[44]Y. Lecun et al., “Gradient-Based Learning Applied to Document Recognition,”
Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
[45]天下雜誌, Available from:
https://www.cw.com.tw/article/article.action?id=5093587
[46]M. D. Zeiler et al., “Deconvolutional Networks,” 2010 IEEE Computer Society
Conference on Computer Vision and Pattern Recognition, San Francisco, USA,
2010, pp. 2528-2535.
[47]Y. Li et al., “Fully Convolutional Instance-Aware Semantic Segmentation,”
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Honolulu, USA, 2017, pp. 4438-4446.
[48]D. Eigen, C. Puhrsch, and R. Fergus, “Depth Map Prediction from a Single
Image using a Multi-Scale Deep Network,” Neural Information Processing
Systems (NIPS), 2014.
[49]Y. Ganin and V. Lempitsky, “ -Fields: Neural Network Nearest Neighbor Fields
for Image Transforms,” 12th Asian Conference on Computer Vision(ACCV),
Singapore, 2014, pp.536-551.
[50]A. Kendall, M. Grimes and R. Cipolla, “PoseNet: A Convolutional Network for
Real-Time 6-DOF Camera Relocalization,” 2015 IEEE International Conference on
Computer Vision (ICCV), Santiago, Chile, 2015, pp. 2938-2946.
[51]不同學習率影響, Available from:
https://medium.com/@chih.sheng.huang821
[52]主動驅動輪, Available from:
https://slideplayer.com/slide/5072146/
[53]輪子種類, Available from:
https://www.dis.uniroma1.it/~deluca/rob1_en/16_MobileRobotsKinematics.pdf/
[54]N. E. N. Rodriguez et al., Advanced Mechanics in Robotic System. 1st ed., New
York, NY, USA: Springer, 2011, pp. 41-57.
[55]全向輪, Available from:
https://en.wikipedia.org/wiki/Omni_wheel
[56]A. Kelly, “A Vector Algebra Formulation of Kinematics of Wheeled Mobile
Robots,” Robotics Institute, Carnegie Mellon University, CMU-RI-TR-10-33,
Aug. 2010.
[57]D. Garcia-Sillas et al., “Kinematics modeling and simulation of an autonomous
omni-directional mobile robot,” Ingeniería E Investigación, vol. 35, no. 2,
pp. 74-79, Aug. 2015.
[58]P. Corke, Robotics, Vision and Control. 1st ed., vol. 73, New York, NY, USA:
Springer, 2011, pp. 12-131.
[59]Mobile Robotics, Available from:
https://dokumen.tips/documents/ee565mobile-robotics-lecture-2-
aknfilesotherteachingmobile-roboticsee565.html
[60]A. Astolfi, “Exponential Stabilization of a Wheeled Mobile Robot Via
Discontinuous Control,” Journal of Dynamic Systems, Measurement, and Control,
vol. 121, no. 1, pp.121-126, 1999.
[61]A. Astolfi, “On the stabilization of nonholonomic systems,” Proceedings of
1994 33rd IEEE Conference on Decision and Control, Lake Buena Vista, FL, USA,
1994, pp. 3481-3486.
[62]G. Casalino et al., “Closed-loop steering for unicycle-like vehicles: A
simple lyapunov like approach,” Fourth IFAC Symposium on Robot Control,
Capri, Italy, Sep. 19-21, 1994, pp. 335-342.
[63]R. W. Brockett, “Asymptotic stability and feedback stabilization,”
Differential Geometry control theory, pp. 181-191, 1983.
[64]E. Ilg et al., “FlowNet 2.0: Evolution of Optical Flow Estimation with Deep
Networks,” 2017 IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), Honolulu, HI, USA, 2017, pp. 1647-1655.