棧板箱型物件全自動上下載系統｜國立臺灣科技大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	周宣宏 XUAN-HONG ZHOU
論文名稱：	棧板箱型物件全自動上下載系統 Pallet Box Autonomous Relocation System
指導教授：	林其禹 Chyi-Yeu Lin
口試委員:	林其禹 Chyi-Yeu Lin 邱士軒 Shih-Hsuan Chiu 郭重顯 Chung-Hsien Kuo 林柏廷 Po Ting Lin
學位類別：	碩士 Master
系所名稱：	工程學院 - 機械工程系 Department of Mechanical Engineering
論文出版年：	2017
畢業學年度：	105
語文別：	中文
論文頁數：	75
中文關鍵詞：	棧板、箱體搬運、雙眼立體視覺、三維點雲、線偵測、ORB特徵、機械手臂
外文關鍵詞：	box relocation, points cloud, ORB feature
相關次數：	點閱：221 下載：2
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

上一筆

本研究所發展的棧板箱型物件全自動上下載系統為基於大型機械手臂結合電腦視覺模組之智慧化技術整合，定義主要針對棧板上堆疊之產品箱物全自動上下載至另一棧板或指定地面。箱型物件可由使用者或由系統自動偵測。系統具有兩種視覺模組，一為使用Kinect v2，二為使用雙眼立體視覺，分別應用在不同工作空間及精度需求。本系統的視覺偵測技術為結合2D影像視覺及3D點雲演算法，針對箱體表面具有圖案之箱型物體，先使用2D影像獲取ORB(Oriented FAST and rotated BRIEF)特徵點，接著為了強化特徵比對結果並算出箱體位置，採用隨機取樣篩選演算法(Random sample consensus, RANSAC)來估測平面轉換矩陣(Homography matrix)，以準確的標出箱體位置。，針對表面無圖案的箱型物件，本研究使用2D影像線偵測(Line detection)去除箱型物件點雲邊界雜點，再使用3D點雲叢聚法(Clustering)及3D點雲具方向性邊界盒(Oriented bounding box, OBB)精準定位箱體位置，以取得棧板及箱型物件空間位置及大小資訊。接續再考慮機械手臂穩健之抓取及擺放姿態因素下，系統自動計算和規劃出機械手臂抓取和推疊箱物路徑。本系統提供三種機器手臂抓取和堆疊棧板箱體方式:一為複製原棧板堆疊方式、二為採用使用者設定之堆疊方式、和三為系統執行最佳化推疊。本系統經過實測，證實強健性和準確度，具備高度產業應用價值。

This study develops an autonomous pallet boxes relocation system that can fully automatically download boxes on the pallet and then relocate the boxes to designated locations. The system is based on the motion capability of a large robot arm and the computer vision modules that enable autonomous operation. The system comprises two sets of vision modules, one is Kinect v2 and the other is the stereo camera system, to serve in different workspace and requirement. The vision detection techniques developed in this system combine 2D image processing codes, and the points cloud algorithm. For detecting boxes with pattern surfaces, we use ORB(Oriented FAST and rotated BRIEF) feature detection and match template image features to the detected image so as to determine the box position and rotation. In order to determine the correct box position, we use RANSAC(Random sample consensus) and Homography matrix. For boxes without pattern features, we combine the 2D line detection to find the edge line of the box and remove the points on the edge line. After that, 3D clustering and OBB(Oriented bounding box, OBB) are implemented to obtain the precious positions of boxes and pallets. Then system will automatically calculate the path of the robot arm to pick the boxes and download them. In this system, there are three path planning options: first is to duplicate the stacking formation of the boxes on the original pallet to a new pallet, second is to allow the user to select the stacking pattern on a specific floor area, and third is to allow the user to define the size and the position of the target floor area, and the system will perform the optimized stacking pattern to save the space and time. The experiments have proven the system effective and precise, and with high commercial values.

目錄
摘要    I
Abstract    II
目錄    IV
表目錄    VII
圖目錄    VII
第一章    緒論....………………………………………………………………...………….1
1.1.前言………………………………………………………………………………..1
1.2.研究目的與動機…………………………………………………………………..2
1.3.文獻回顧…………………………………………………………………………..3
1.4.本文架構…………………………………………………………………………..6
第二章    基礎理論…………………………………………………………………………7
2.1.相機校正…………………………………………………………………………..7
2.1.1.相機成像原理-Pinhole camera model……………………………………..7
2.1.2.外部參數（Extrinsic parameters）………………………………………..9
2.1.3.內部參數（Intrinsic parameters）……………………....………..….…..10
2.1.4.形變參數（Distortion coefficients）………………………………….…12
2.2.立體視覺原理……………………………………………………………..……..14
2.3.相機與機械手臂……………………………………………………………..…..15
2.4.2D影像演算法……...…………...………………………………………………16
2.4.1.ORB（Oriented FAST and Rotated BRIEF）………..…………………..16
2.4.2.Homographymeters…………………………………………………...…..19
2.4.3.RANSAC………………………………………………...…………….….21
2.4.4.顏色偵測……………………………………………………………….…23
2.4.5.Canny邊界偵測………………………………………………………..…25
2.4.6.minAreaRect………………………………………………………………27
2.4.7.霍夫轉換(Hough Transform)……………………..………………………28
2.4.9.1.隨機霍夫轉換(Probabilistic Hough Transform)…....……………..28

2.4.8.形態學………………………………………….…………………………29
2.4.9.Kd-tree鄰近點搜尋………………………………………………………30
2.4.9.1.Kd-tree原理………………………………………….……………31
2.4.10.二維箱體最佳化排列…………………………..……….………………33
2.5.3D點雲演算法……...…………...………………………………………………35
2.5.1.減採樣(Downsampling)………………………………………………..…35
2.5.2.叢聚法(Clustering)………………………………………..………………36
2.5.3.OBB(Orient bounding box)…………………………………………….…38
第三章    實驗器材與設置………………………………………………………………..39
3.1.攝影機(相機)……...…………...……………………………………………...…40
3.2.Kinect v2……..……...…………...………………………………………………41
3.3.HIWIN關節式機器手臂-RA620………………………………………..………42
3.4.紙箱吸盤夾爪……..……..………………………………………………………43
3.5.紙箱與棧板(表面無圖案)……...……………………………………………..…45
第四章    實驗方法與結果………………………………………………………………..46
4.1.表面有圖案之箱體偵測...…………...…..……………….…………………...…46
4.2.無表面圖案之箱體偵測...………...…..……………….……………………...…49
4.2.1.尋找上平面……………………………...……………………………..…49
4.2.2.偵測上平面箱體輪廓線……………………..………………………..….51
4.2.3.過濾邊界線上的點雲……………..…...…………………………..…..…52
4.2.4.取得箱體位置及大小……………..…...…………………………..…..…52
4.3.路徑規劃...…………...…………………..……………….…………………...…54
4.3.1.複製原始堆疊方式……………………...……………………………..…54
4.3.2.使用者設定之堆疊方式……………………..………………………..….63
4.3.3. 使用者設定之區域自動最佳化推疊方式…..……………………....….65
4.4.實驗結果數據...…………...…………………..……………….………………...67
4.4.1.具表面圖案之箱體偵測數據……….....…………..…………………..…67
4.4.2.無表面圖案之箱體偵測數據……………...……….………………....….68
第五章    結論與未來展望………………………………………………………………..69
5.1.結論...…………...…..…………………………………….…………………...…69
5.2.未來展望...…………...…..……………….…………………………………...…71
第六章    參考文獻………………………………………………………………………..73
 
表目錄
表1 多種特徵比較表…………………………………………………………………….18
表2 HIWIN關節式機器手臂-RA620規格表…………………………………………….42
表3 具表面圖案之箱體偵測時間數據表……………………….………………………67
表4 具圖案之箱體實驗誤差圖表……………………………………………………….67
表5 表面無圖案之箱體偵測時間數據表……………………………………………….68
表6 無圖案之箱體實驗誤差圖表……………………………………………………….68

圖目錄
圖1 Pinhole camera model…………………………………..…….……………………7
圖2 座標轉換示意圖………………………………………...……………………………8
圖3 世界座標轉換為相機座標…………………….………………………………..……9
圖4 投影至虛擬成像平面…………..……………….…………………………………..10
圖5 Skew夾角………………………………………………….………………………...12
圖6 Radial distortion type……………………………………...…………………..12
圖7 切像形變發生情況與影像圖…………………….…………………………………13
圖8 立體影像處理的模型……………………….………………………………………14
圖9 攝影機跟世界座標關係之上視圖……………….…………………………………14
圖10 相機與手臂的轉換關係3D示意圖………………….……………………………15
圖11 ORB演算法流程圖…………………………………………………………………16
圖12 ORB特徵檢測………………………………………………………………………18
圖13 RANSAC示意圖……………………………………………..………………………21
圖14 用RANSAC做homography estimation的結果……………………….…………23
圖15 實際偵測棧板之二值化圖像…………………………...…………………………24
圖16 Sobel面罩…………………………………………………………………....……25
圖17 Canny邊界偵測……………………………………………………………....……26
圖18 實際箱體Canny邊界偵測………...………………………………………………26
圖19 minAeraRect函式所找出的四個角點示意圖………...………………………….27
圖20 使用minAeraRect得到棧板位置及轉矩………...………………………………27
圖21 霍夫轉換參數空間………...………………………………………………………28
圖22 霍夫線檢測箱體………...…………………………………………………………29
圖23 形態學實驗圖………...……...…………………………………………………….30
圖24 箱形物體表面邊界盒………...……………………………………………………30
圖25 最近鄰近點搜尋結果………...……………………………………………………30
圖26 二維數據點區群………...…………………………………………………………31
圖27 以x1作為切割點分開空間………...……………………………………………..31
圖28 以y1作為分割點分割空間………...……………………………………………..32
圖29 Kd-tree 結構圖………...…………………………………………………………32
圖30 二維箱體最佳化排列說明圖1...………………………………………………….33
圖31 二維箱體最佳化排列樹狀圖1...………………………………………………….33
圖32 二維箱體最佳化排列說明圖2...………………………………………………….34
圖33 二維箱體最佳化排列樹狀圖2...………………………………………………….34
圖34 箱體最佳化排列實測1………...………………………………………………….34
圖35 箱體最佳化排列實測2………...………………………………………………….34
圖36 立體像素網格………...……………………………………………………………35
圖37 叢聚法…………………………………………………………………………...…36
圖38 叢聚法搜尋上平面群點………...…………………………………………………37
圖39 叢聚法搜尋每一箱體表面群點………...…………………………………………37
圖40 OBB檢測每一箱體表面群點………...…………………………………………….38
圖41 視覺模組架設前視圖………...……………………………………………………39
圖42 Kinect v2模擬架設圖………...………………………………………………….39
圖43 Basler aca2500-14gc………...…………………………………………………..40
圖44 Basler aca2500-14gc機身規格………...……………………………………….40
圖45 Kinect v2………...………………….…………………………………………….41
圖46 HIWIN關節式機器手臂-RA620………...………………………………………….42
圖47 (左)真空吸盤、夾爪………...……………………………………………………43
圖48 紙箱吸盤夾爪………...………………………...………………………………….43
圖49 吸盤規格圖………...………………………………………………………………44
圖50 實際吸盤夾爪直裝配圖………...…………………………………………………44
圖51 棧板上視圖………...………………………………………………………………45
圖52 紙箱推疊於棧板上………...………………………………………………………45
圖53 ORB匹配結果圖………...………………………………………………………….46
圖54 ORB匹配數據及時間………...…………………………………………………….47
圖55 系統已找出此層所有箱體結果圖…...…...……………………………………….47
圖56 表面具圖案箱體實驗流程圖...……………………………………………………48
圖57 表面具圖案箱體實驗結果圖...……………………………………………………48
圖58 偵測畫面………...…………………………………………………………………49
圖59 叢聚法找出上平面………...………………………………………………………50
圖60 上平面之邊界盒………...…………………………………………………………50
圖61 上平面之影像………...……………………………………………………………50
圖62 邊線檢測………...…………………………………………………………………51
圖63 形態學增強邊線檢測………...……………………………………………………51
圖64 刪除邊線點之上平面點雲……………………..………………………………….52
圖65 每一箱體表面點雲………...………………………………………………………53
圖66 OBB檢測每一箱體表面群點………...…………………………………………….53
圖67 基於點為中心之轉換矩陣………...………………………………………………54
圖68 系統實驗架構…………………………………………………….………………..55
圖69 原始箱物堆疊方式………………………………………………………………...55
圖70 第一階段箱體偵測結果…………………………………………………………...56
圖71 第一階段紙箱模擬放至第二棧板………………………………………………...56
圖72 第一階段機械手臂抓取過程-1…………………………………………………...57
圖73 第一階段機械手臂抓取過程-2…………………………………………………...57
圖74 第一階段擺放結果………………………………………………………………...58
圖75 第二階段箱體偵測結果…………………………………………………………...58
圖76 第二階段紙箱模擬放至第二棧板………………………………………………...59
圖77 第二階段機械手臂抓取過程-1…………………………………………………...59
圖78 第二階段機械手臂抓取過程-2…………………………………………………...60
圖79 第二階段擺放結果………………………………………………………………...60
圖80 第三階段箱體偵測結果…………………………………………………………...61
圖81 第三階段紙箱模擬放至第二棧板………………………………………………...61
圖82 第三階段機械手臂抓取過程-1…………………………………………………...62
圖83 第三階段機械手臂抓取過程-2…………………………………………………...62
圖84 第三階段擺放結果………………………………………………………………...63
圖85 使用者自定義之擺放型式………………………………………………………...63
圖86 使用者自定義模式擺放過程-1……………………………………………….......64
圖87 使用者自定義模式擺放過程-2…………………………………………………...64
圖88 使用者定義之擺放結果…………………………………………………………...65
圖89 最佳化2D箱體擺放……………………………………………………………….65
圖90 自動最佳化推疊方式手臂夾取過程-1…………………………………………...66
圖91 自動最佳化推疊方式手臂夾取過程-2…………………………………………...66

                                

[1] Xu, Y., Liu, Y., Hao, L., and Cheng, H., 2016, "Design of palletizing algorithm based on palletizing robot workstation," Proc. Real-time Computing and Robotics (RCAR), IEEE International Conference on, IEEE, pp. 609-613.
[2] Mahalik, N. P., 2009, "Processing and packaging automation systems: a review," Sensing and Instrumentation for Food Quality and Safety, 3(1), pp. 12-25.
[3] Bloss, R., 2006, "JTM use Motoman to stack pails of lubricants," Industrial Robot: An International Journal, 33(1), pp. 24-26.
[4] Hemmingson, E., 1998, "Palletizing robots for the consumer goods industry," Industrial Robot: An International Journal, 25(6), pp. 384-388.
[5] Bloss, R., 2010, "Palletizing candy orders and never squeezing the chocolates," Assembly Automation, 30(1), pp. 32-35.
[6] Han, J., Shao, L., Xu, D., and Shotton, J., 2013, "Enhanced computer vision with microsoft kinect sensor: A review," IEEE transactions on cybernetics, 43(5), pp. 1318-1334.
[7] Zhang, Z., 2012, "Microsoft kinect sensor and its effect," IEEE multimedia, 19(2), pp. 4-10.
[8] Rusu, R. B., 2010, "Semantic 3d object maps for everyday manipulation in human living environments," KI-Künstliche Intelligenz, 24(4), pp. 345-348.
[9] Rusu, R. B., Marton, Z. C., Blodow, N., Dolha, M., and Beetz, M., 2008, "Towards 3D point cloud based object maps for household environments," Robotics and Autonomous Systems, 56(11), pp. 927-941.
[10] Rusu, R. B., Blodow, N., and Beetz, M., "Fast point feature histograms (FPFH) for 3D registration," Proc. Robotics and Automation, 2009. ICRA'09. IEEE International Conference on, IEEE, pp. 3212-3217.
[11] Brantmark, H., and Hemmingson, E., 2001, "FlexPicker with PickMaster revolutionizes picking operations," Industrial robot: An international journal, 28(5), pp. 414-420.
[12] Heikkila, J., and Silven, O., "A four-step camera calibration procedure with implicit image correction," Proc. Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on, IEEE, pp. 1106-1112.
[13] Zhang, Z., 2000, "A flexible new technique for camera calibration," IEEE Transactions on pattern analysis and machine intelligence, 22(11), pp. 1330-1334.
[14] Fu, K., Gonzales, R., and Lee, C., 1987, "Robotics: Control, Sensing, Vision, and Intelligence. McGrawHill," Inc., Singapore.
[15] Rublee, E., Rabaud, V., Konolige, K., and Bradski, G., "ORB: An efficient alternative to SIFT or SURF," Proc. Computer Vision (ICCV), 2011 IEEE international conference on, IEEE, pp. 2564-2571.
[16] Benhimane, S., and Malis, E., 2007, "Homography-based 2d visual tracking and servoing," The International Journal of Robotics Research, 26(7), pp. 661-676.
[17] Fischler, M. A., and Bolles, R. C., 1981, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography," Communications of the ACM, 24(6), pp. 381-395.
[18] Canny, J., 1986, "A computational approach to edge detection," IEEE Transactions on pattern analysis and machine intelligence(6), pp. 679-698.
[19] Matas, J., Galambos, C., and Kittler, J., 2000, "Robust detection of lines using the progressive probabilistic hough transform," Computer Vision and Image Understanding, 78(1), pp. 119-137.
[20] Bentley, J. L., 1975, "Multidimensional binary search trees used for associative searching," Communications of the ACM, 18(9), pp. 509-517.
[21] Lodi, A., Martello, S., and Vigo, D., 2002, "Recent advances on two-dimensional bin packing problems," Discrete Applied Mathematics, 123(1), pp. 379-396.
[22] Rusu, R. B., 2010, "Semantic 3d object maps for everyday manipulation in human living environments," KI-Künstliche Intelligenz, 24(4), pp. 345-348.

簡易檢索 / 詳目顯示

相關論文