虛擬實境技術可使使用者藉由配戴頭戴式顯示器體驗虛擬環境中身歷其境的快感，但必需額外搭配定位裝置追蹤使用者的位置，具有一定的空間限制；再者，使用者因配戴顯示器無法看到外界實際情況容易受到傷害如撞到障礙物等問題。擴增實境技術則是利用影像間位置及角度的改變使虛擬物件隨環境產生變化，常用於實現手機應用程式互動功能。然而目前的擴增實境技術大多僅將虛擬物件放入真實影像中，使用者無法融入虛擬環境外且需較為空曠的空間才能執行。本研究的目的為創造一個完全基於真實環境所構建的虛擬環境，透過計算攝影機影像的位置及角度建立環境，並將環境中的物件加以區分後使用虛擬物件取代創造新的環境。這樣的作法可讓使用者享受體驗虛擬環境外，同時也解除了環境限制的問題。本文利用即時定位與地圖構建建立所處環境，並結合圖像分割進行分群，接著將分群結果進行形狀擬合用以找出合適的虛擬物件進行取代。實驗結果證實了本系統的可行性。

Virtual reality and augmented reality have become more and more popular in many application fields recently. The technology of virtual reality allows the user to wear a head-mounted display to have the feelings of experiencing various virtual environments. However, to track the user’s position this technology also needs to be equipped with extra positioning devices and thus has certain workspace constraints. Another issue is that the users are vulnerable to the physical damages such as obstacle collision due to the view occlusion introduced by the wearable devices. The augmented reality, on the other hand, applies the position and orientation changes between image frames to make the virtual objects vary with environments. The most common application of augmented reality can be found in the interactive functions of intelligent mobile phones. One major drawback for the current augmented reality technology is that the virtual objects are only added to the background of the real images; it is difficult for users to be integrated with the virtual environments and a spacious area is still needed to implement the whole system. This research aims to develop a virtual environment entirely based on the real environmental scenarios. The environment is first established by calculating the image poses and the objects in the established environment are then divided into separate pieces and replaced with some virtual objects. This allows the users to enjoy the virtual environment experiences without considering environmental restrictions. In the current study, the environment is built by using the technique of real-time simultaneous localization and mapping, and an image segmentation method is developed to implement object grouping. The clustering results are shape-fitted to match suitable virtual objects to be replaced. Finally, the performance of the experimental results justifies the feasibility of the developed system.

[1] https://zh.wikipedia.org/wiki/%E8%99%9A%E6%8B%9F%E7%8E%B0%E5%
AE%9E
[2] https://mic.iii.org.tw/aisp/FreeS.aspx?id=3076
[3] https://www.oculus.com/
[4] https://www.vive.com/tw/
[5] https://asia.playstation.com/cht-tw/psvr/
[6] https://vr.google.com/cardboard/
[7] https://www.samsung.com/global/galaxy/gear-vr/
[8] http://gank.fanpiece.com/leiphone/HTC-vive-Oculus-rift-PSVR%E4%B8%89
%E5%A4%A7%E9%A0%AD%E9%A1%AF%E9%83%BD%E6%98%AF%E5%A6%82%E4%BD%95%E5%81%9A%E5%AE%9A%E4%BD%8D%E7%9A%84-c1229753.html
[9] https://www.oculus.com/blog/the-oculus-rift-oculus-touch-and-vr-games-at-e3/
[10] P. Milgram, and F. Kishino, “A Taxonomy of Mixed Reality Visual Displays”, IEICE Trans. Information Systems, vol. E77-D, no. 12, 1994, pp. 1321-1329.
[11] https://www.digi-capital.com/news/2018/04/ar-vr-startups-raise-record-36billion- in-last-12-months-as-market-transition-accelerates/#.WzyMQlUzaUk
[12] https://www.cnbc.com/video/2018/05/29/the-future-of-augmented-reality.html
[13] https://www.microsoft.com/en-us/hololens
[14] https://developers.google.com/ar/discover/
[15] https://developer.apple.com/arkit/
[16] L. Yang, R. B. Noelle, and M. Markus, “Augmented Reality Powers a Cognitive Prosthesis for the Blind ”, doi: https://doi.org/10.1101/321265 , 22 May 2018
[17] G. Evans, J. Miller, M. I. Pena, A. MacAllister, and E. Winter, “Evaluating the Microsoft HoloLens through an augmented reality assembly application,” in Proceedings of SPIE Defense, 7 May 2017, Anaheim, California, United States.
[18] N. Cui, P. Kharel, and V. Gruev, “Augmented reality with Microsoft HoloLens holograms for near infrared fluorescence based image guided surgery”, in Proceedings of SPIE BiOS, 8 February 2017, San Francisco, California, United States.
[19] A. Davison, I. Reid, N. Molton, and O. Stasse, “MonoSLAM: Real-time single camera SLAM”, Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 29(6):1052-1067, 2007.
[20] G. Klein and D. Murray, “Parallel tracking and mapping for small AR workspaces”, In International Symposium on Mixed and Augmented Reality (ISMAR), 13-16 Nov. 2007, Nara, Japan, pp. 225-234.
[21] R. Mur-Artal, J. Montiel, and J. Tardos, “ORB-SLAM: a versatile and accurate monocular SLAM system”, IEEE Transactions on Robotics, 31(5):1147–1163, 2015.
[22] L. Valgaerts, A. Bruhn, M. Mainberger, and J. Weickert, “Dense versus sparse approaches for estimating the fundamental matrix”, International Journal of Computer Vision (IJCV), 96(2):212–234, 2012.
[23] R. Ranftl, V. Vineet, Q. Chen, and V. Koltun, “Dense monocular depth estimation in complex dynamic scenes”, In International Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June 2016, Las Vegas, NV, USA.
[24] R. Newcombe, S. Lovegrove, and A. Davison, “DTAM: Dense tracking and mapping in real-time”, In International Conference on Computer Vision (ICCV), 6-13 Nov. 2011, Barcelona, Spain, pp. 2320-2327.
[25] J. St¨uhmer, S. Gumhold, and D. Cremers, “Real-time dense geometry from a handheld camera”, In Pattern Recognition (DAGM), 2010, pp. 11-20.
[26] J. Engel, T. Sch¨ops, and D. Cremers, “LSD-SLAM: Largescale direct monocular SLAM”, in Proceedings of European Conference on Computer Vision (ECCV), Sep. 2014, Zurich, Switzerland, pp. 834-849.
[27] J. Engel, V. Usenko and D. Cremers, “A Photometrically Calibrated Benchmark For Monocular Visual Odometry”, arXiv:1607.02555, July 2016.
[28] J. Engel, V. Koltun and D. Cremers, “Direct Sparse Odometry”, arXiv:1607.02565, July 2016.
[29] D. G. Lowe, ”Object recognition from local scale-invariant features,” in Proceedings of the Seventh IEEE International Conference on Computer Vision, 20-27 Sept. 1999, Kerkyra, Greece, pp. 1150-1157.
[30] H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool,”SURF: Speeded Up Robust Features”, Computer Vision and Image Understanding (CVIU), 110(3): 346–359, 2008.
[31] R. B. Rusu, N. Blodow ,and M. Beetz, “Fast Point Feature Histograms (FPFH) for 3D Registration”, IEEE International Conference on Robotics and Automation, 12-17 May 2009, Kobe, Japan, pp. 3212-3217.
[32] M. A. Fischler, and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography”, Communications of the ACM, 24(6): 381-395, June 1981.
[33] B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon,” Bundle Adjustment —A Modern Synthesis”, International Workshop on Vision Algorithms (IWVA): Vision Algorithms: Theory and Practice, pp. 298-372, 2000.
[34] T. Rabbani, F. A. Van Den Heuvel, and G. Vosselman, “Segmentation of point clouds using smoothness constraint”, Int. Arch. Photogramm. Remote Sens. Spatial Inform Sci., 36(5): 248-253, 2006.
[35] D. Holz, and S. Behnke, “Fast Range Image Segmentation and Smoothing Using Approximate Surface Reconstruction and Region Growing”, In Proceedings of the International Conference on Intelligent Autonomous Systems (IAS), pp. 61–73, 2013.
[36] A. Habib, and Y. J. Lin, “Multi-Class Simultaneous Adaptive Segmentation and Quality Control of Point Cloud Data”, Remote Sens., 8(2), 2016.
[37] D. Belton, and D. Lichti, “Classification and segmentation of terrestrial laser scanner point clouds using local variance information”, Int. Arch. Photogramm. Remote Sens., Volume XXXVI, Part 5, pp. 44-49, 2006.
[38] G. Vosselman, B. G. Gorte, G. Sithole, and T. Rabbani, “Recognising structure in laser scanner point clouds”, Int. Arch. Photogramm. Remote Sens. Spatial Inform. Sci., 46(8): 33-38, 2004.
[39] D. H. Ballard”, “Generalizing the hough transform to detect arbitrary shapes. Pattern recognition”, Pattern Recognition, 13(2): 111–122, 1981.
[40] H. Song, and H. Y. Feng, “A global clustering approach to point cloud simplification with a specified data reduction ratio”, Computer-Aided Design, 40 (3): 281–292, March 2008.
[41] B. Q. Shi, J. Liang, and Q. Liu, “Adaptive simplification of point cloud using k-means clustering”, Computer-Aided Design, 43(8): 910–922, August 2011.
[42] U. von Luxburg, “A tutorial on spectral clustering”, Statistics and Computing, 17(4): 395-416, 2007.
[43] P. F. Felzenszwalb, D. P. Huttenlocher, “Efficient graph-based image segmentation”, International Journal of Computer Vision, 59(2): 167-181, September 2004.
[44] K. Duncan, S. Sarkar, R. Alqasemi, and R. Dubey, “Multi-scale Superquadric Fitting for Efficient Shape and Pose Recovery of Unknown Objects”, in IEEE International Conference on Information and Automation (ICRA), 6-10 May 2013, Karlsruhe, Germany, pp. 4238-4243.
[45] https://www.techbang.com/posts/2936-get-to-know-how-it-works-kinect
[46] https://zoomicon.wordpress.com/2015/07/28/kinect-for-xbox-360-and-kinect-for-windows-kfw-v1-specs/
[47] Shangjie Shi, Wei Zhou, and Shuang Liu, “A high efficient 3D SLAM algorithm based on PCA”, in IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), 19-22 June 2016, Chengdu, China, pp. 109-114.
[48] https://ccjou.wordpress.com/2013/04/15/%E4%B8%BB%E6%88%90%E5%88%86%E5%88%86%E6%9E%90/
[49] https://www.slideshare.net/MostafaGMMostafa/neural-networks-principal-component-analysis-pca
[50] https://en.wikipedia.org/wiki/Voxel
[51] http://pointclouds.org/documentation/tutorials/normal_estimation.php
[52] E. Wahl, U. Hillenbrand, and G. Hirzinger, “Surflet-pair-relation histograms: a statistical 3D-shape representation for rapid classification”, in Proceedings of International Conference on 3-D Digital Imaging and Modeling, 6-10 Oct. 2003, Banff, Alta., Canada, pp. 474-481.
[53] https://www.slideserve.com/siusan/ransac
[54] A. Angeli, S. Doncieux, and J. A. Meyer, “Real-Time Visual Loop-Closure Detection”, in IEEE International Conference on Robotics and Automation, 19-23 May 2008, Pasadena, CA, USA, pp. 1842-1847.
[55] J. More, “The Levenberg-Marquardt algorithm: implementation and
theory”, in Conference on numerical analysis, edited by G.A. Watson, vol. 630, pp. 105–116. : Springer Berlin Heidelberg, 1978.
[56] R. B. Rusu, N. Blodow, Z. C. Marton and M. Beetz, “Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in domestic environments”, in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 10-15 Oct. 2009, St. Louis, MO, USA, pp. 1-6.
[57] Y. Xu, S. Tuttas and U. Stilla, “Segmentation of 3d outdoor scenes using hierarchical clustering structure and perceptual grouping laws”, in IAPR Workshop on Pattern Recogniton in Remote Sensing (PRRS), 4 Dec. 2016, Cancun, Mexico, pp. 1–6.
[58] Y. Xu, L. Hoegner, S. Tuttas and U. Stilla, “VOXEL- AND GRAPH-BASED POINT CLOUD SEGMENTATION OF 3D SCENES USING PERCEPTUAL GROUPING LAWS”, in ISPRS Ann. Photogramm. Remote Sens. Spatial Inf., 30 May 2017, Hannover, Germany, pp. 43-50.
[59] J. Papon, A. Abramov, M. Schoeler and F. Worgotter, “Voxel Cloud Connectivity Segmentation - Supervoxels for Point Clouds”, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 23-28 June 2013, Portland, OR, USA.
[60] R. Kaucic, R. Hartley and N. Dano, “Plane-based Projective Reconstruction”, in Proceedings of IEEE International Conference on Computer Vision (ICCV), 7-14 July 2001. Vancouver, BC, Canada.
[61] C. S. Rothwell, A. Zisserman, D. A. Forsyth, and J. L. Mundy, “Planar object recognition using projective shape representation”, Int. J. Comput. Vision, 16 (1): 57-99, September 1995.
[62] J. Biosca, and J. Lerma. “Unsupervised robust planar segmentation of terrestrial laser scanner point clouds based on fuzzy clustering methods”, ISPRS Journal of Photogrammetry and Remote Sensing, 63(1): 84-98, January 2008.
[63] P. F. U. Gotardo, O. R. P. Bellon, and L. Silva. “Range image segmentation by surface extraction using an improved robust estimator”, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 18-20 June 2003, Madison, WI, USA.
[64] D. Borrmann, J. Elseberg, K. Lingemann and A. Nchter, “The 3d hough transform for plane detection in point clouds- a review and a new accumulator design”, 3D Research, June 2011.
[65] L. Li, F. Yang, H. Zhu, D. Li, Y. Li, L. Tang, “An improved RANSAC for 3D point cloud plane segmentation based on normal distribution transformation cells”, Remote Sens., 9(5): 433, 2017.
[66] J. E. Deschaud, F. Goulette, “A fast and accurate plane detection algorithm for large noisy point clouds using filtered normals and voxel growing”, in Proceedings of the International Symposium on 3D Data Processing, Visualization and Transmission, May 2010, Paris, France.
[67] S. Alexandrov, “Geometrical Segmentation of Point Cloud Data by Spectral Analysis”, Sankt Augustin Germany: Hochschule Bonn-Rhein-Sieg, Septemper 2014.
[68] NP-hardness, https://baike.baidu.com/item/NP-hard
[69] 論P, NP, NP-hard, NP-complete問題,
http://bluelove1968.pixnet.net/blog/post/222283186-%E8%AB%96p,np,np-hard,np-complete%E5%95%8F%E9%A1%8C
[70] https://www.ycc.idv.tw/algorithm-complexity-theory.html
[71] https://people.cs.nctu.edu.tw/~chengchc/Course_2011_Fall_DS/DS_10_NPC.pdf
[72] J. Shi, J. Malik, “Normalized Cuts and Image Segmentation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8): 888-905, August 2000.
[73] Rayleigh quotient, https://en.wikipedia.org/wiki/Rayleigh_quotient
[74] U. von Luxburg, “A tutorial on spectral clustering”, Stat. Comput. 17(4): 395-416, 2007.
[75] A. Collet, D. Berenson, S. S. Srinivasa, and D. Ferguson, “Object Recognition and Full Pose Registration from a Single Image for Robotic Manipulation”, in IEEE International Conference on Robotics and Automation, 12-17 May 2009, Kobe, Japan, pp. 48-55.
[76] A. Collet, and S. S. Srinivasa, “Efficient Multi-View Object Recognition and Full Pose Estimation”, in International Conference on Robotics and Automation,
3-7 May 2010, Anchorage, AK, USA, pp. 2050-2055.
[77] K. Duncan, S. Sarkar, R. Alqasemi, and R. Dubey, “Multi-scale Superquadric Fitting for Efficient Shape and Pose Recovery of Unknown Objects”, in IEEE International Conference on Information and Automation (ICRA), 6-10 May 2013, Karlsruhe, Germany, pp. 4238-4243.
[78] A. Jaklic, A. Leonardis, and F. Solina, Segmentation and Recovery of Superquadrics, Computational Imaging and Vision, 2000.
[79] F. Solina, and R. Bajcsy, “Recovery of parametric models from range images: the case for superquadrics with global deformations”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(2): 131-147, Feb 1990.
[80] M. Strand, Z. Xue, M. Zoellner, and R. Dillmann, “Using superquadrics for the approximation of objects and its application to grasping”, in Proceedings of the IEEE International Conference on Information and Automation, 20-23 June, Harbin, China, pp. 48-53.
[81] P. Drews Jr, P. Núñez, R. Rocha, M. Campos, and J. Dias, “Novelty Detection and 3D Shape Retrieval using Superquadrics and Multi-Scale Sampling for Autonomous Mobile Robots”, in Proceedings of the IEEE International Conference on Robotics and Automation, 3-8 May 2010, Anchorage, Alaska, USA.
[82] A. Leonardis, A. Jaklic, and F. Solina, “Superquadrics for Segmenting and Modeling Range Data”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(11), pp. 1289-1295, 1997.
[83] D. Katsoulas, C. C. Bastidas, and D. Kosmopoulos, “Superquadric Segmentation in Range Images via Fusion of Region and Boundary Information”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(5): 781-795, May 2008.
[84] M. Strand, and R. Dillmann, “Segmentation and Approximation of Objects in Pointclouds using Superquadrics”, in International Conference on Information and Automation, Zhuhai, Macau, China, 22-24 June 2009, pp. 887-892.
[85] K. Moustakas, D. Tzovaras, and M. G. Strintzis, “SQ-Map: Efficient Layered Collision Detection and Haptic Rendering”, IEEE Transactions on Visualization and Computer Graphics, 13(1): 80-93, Jan.-Feb. 2007.
[86] A. H. Barr, “Superquadrics and angle-preserving transformations”, IEEE
Computer Graphics and Applications, 1(1): 11-23, January 1981.
[87] https://zh.wikipedia.org/wiki/%E8%B6%85%E6%A9%A2%E5%9C%93
[88] http://www.pointclouds.org/gsoc/
[89] https://zh.wikipedia.org/wiki/%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95
[90] https://zh.wikipedia.org/wiki/%E7%89%9B%E9%A1%BF%E6%B3%95
[91] https://baike.baidu.com/item/%E9%AB%98%E6%96%AF%E7%89%9B%E9%A1%BF%E6%B3%95
[92] https://blog.csdn.net/CSDNMicrosoftCSDN/article/details/38033427
[93] https://zh.wikipedia.org/wiki/OpenCV
[94] https://en.wikipedia.org/wiki/Point_Cloud_Library
[95] http://eigen.tuxfamily.org/index.php?title=Main_Page
[96] https://www.cs.ubc.ca/research/flann/
[97] https://www.threadingbuildingblocks.org/documentation
[98] https://www.591.com.tw/rent-detail-1805283.html?version=old
[99] I. Oikonomidis, N. Kyriazis, and A. Argyros, “Efficient model-based 3D tracking of hand articulations using Kinect”, in Proceedings of the British Machine Vision Conference, Jesse Hoey, Stephen McKenna and Emanuele Trucco, pp. 101.1-101.11. BMVA Press, 29 August-2 September 2011.
[100] Q. Chen, S. Xiao, W. Yichen, T. Xiaoou, and S. Jian, “Realtime and Robust Hand Tracking from Depth”, IEEE Conference on Computer Vision and Pattern Recognition, 23-28 June 2014, Columbus, OH, USA.