近年來，資訊產業的蓬勃發展帶動了虛擬實境(Virtual Reality)、擴增實境(Augmented Reality)在生活上的應用。其中擴增實境更用於智慧窗、智慧櫥窗等新應用。利用透明顯示器進行的擴增實境，可呈現動態或靜態、文字或圖像的資訊，為螢幕後方的目標物件加值。但目前加值的資訊多為二維，當附加材質圖於物件或是基於使用者視角轉動模型時，會有缺乏立體感的情況發生，是擴增實境互動應用上的缺陷。
為了改善此一問題，本論文提出兩套實驗流程應用於透明平面顯示器的擴增實境互動，對於最終呈現的物件提供附有立體感的貼圖加值。實驗一開發一套3D模型優化系統：對於後方物件進行3D建模，並分別透過二維與三維影像處理技術對物件進行優化，將欲呈現的物件進行雜訊去除及破損填補，得到較佳的3D建模結果。實驗二開發一套模型姿態辨識系統：首先對物件進行精確的3D掃描，透過對當前擺放物件進行物件辨識及姿態估計，接著將預先掃描的物件利用估計出的矩陣轉動至當前姿態並呈現於顯示器上，與使用者進行擴增實境互動應用。兩者的實驗結果皆能有效地提供精確3D物件於顯示器進行3D資訊的疊加融合。相較於2D資訊能呈現更好的立體互動效果。

In recent years, the rapid growing of information industries drive the development of virtual reality (VR) and augmented reality (AR). AR technology can be used in new applications such as smart windows and smart showcase which use transparent displays to add information to behind target objects. However, most of these attached information is in 2D. There is a lack of stereoscopic feeling when material maps are attached to an object or model rotates based on the user’s perspective.
To improve the 3D feeling, we proposed two pipelines to enhance the quality of 3D model for covering the target objects for user-perspective AR interaction. First pipeline uses a rear depth camera in real-time to do 2D and 3D filtering to remove noise and artifacts and combine multiple views of the depth images to make 3D model more complete. Second pipeline applies a precise scanned 3D model to correct position by means of 3D image recognition and pose estimation. Both pipelines show good results and make the visual experience more natural and comfortable when interact with a transparent display.

[1] M. H. Lin, Y. P. Pi, H. S. Chen, P. L. Sun, T. H. Lin, "Digital Cosmetic Coloring System for 3D Facial Images. ", The 8th International Conf. on 3D Systems and Applicationsz(3DSA), 2016.
[2] A. Daniel, "Medical Telementoring using an Augmented Reality Transparent Display. ", Medical telementoring using an augmented reality transparent display, Surgery, 159(6), pp. 1646-1653, 2016.
[3] H. Otmar, "HoloDesk: Direct 3D Interactions with a Situated See-Through Display. ", Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2421-2430, 2012.
[4] O. Schreer, P. Kauff, "3D Video-Communication Algorithms, Concepts and Real-time Systems in Human Centred Communication. ", Wiley, 2005.
[5] A. Kaehler, G. Bradski, "Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library. ", O'Reilly Media, 691-766, 2017.
[6] 馬然,王永芳,安平,沈禮權, "完全掌握；最強3D視訊處理技術書. " , 佳魁數位, 第二章, 55-62頁, 2018.
[7] 蕭毓庭, "光學穿透式頭戴顯示器影像品質優化與自動化前景影像分割. " , 國立台灣科技大學, 2018.
[8] L. Benjamin, K. Hartmann, "Depth Camera Technology Comparison and Performance Evaluation.", ICPRAM (2), pp. 438-444, 2012.
[9] M. Loghman, K. Chung, "Depth Map Estimation using Modified Census Transform and Semi-Global Matching. ", 2014 International SoC Design Conference (ISOCC), pp. 3-6, 2014.
[10] S. Cousins, "Point Cloud Library (pcl). ", 2011 IEEE international conference on robotics and automation, pp. 1-4, 2011.
[11] C. Zhang, Z. Zhang, "Calibration between Depth and Color Sensors for Commodity Depth Cameras. ", Computer vision and machine learning with RGB-D sensors , pp. 47-64, 2014.
[12] Z. Zhang, "A Flexible New Technique for Camera Calibration. ", IEEE Transactions on pattern analysis and machine intelligence, vol. 22, no. 11, pp. 1330-1334, 2000.

[13] Intel, "Intel RealSense D400 Series Calibration Tools. ", Support Home, Driver & Software, 2019.
[14] Q. Chen, "Realtime and Robust Hand Tracking from Depth. ", Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1106-1113, 2014.
[15] D. Min, J. Lu, "Depth Video Enhancement based on Weighted Mode Filtering. ", IEEE Transactions on Image Processing , 21(3), pp. 1176-1190, 2012.
[16] F. Tombari, S. Mattoccia, "Classification and Evaluation of Cost Aggregation Methods for Stereo Correspondence. ", 2008 IEEE Conference on Computer Vision and Pattern Recognition , pp. 1-8, 2008.
[17] H. Kaiming, J. Sun, X. Tang, "Guided Image Filtering. ", IEEE transactions on pattern analysis and machine intelligence, pp. 1397-1409, 2013.
[18] S. Y. Kim, D. Lee, "Depth Map Denoising and Object Boundary Refinement using Joint Bilateral Filtering. ", The 4th International Conf. on 3D Systems and Applicationsz(3DSA), 2012.
[19] H. Dirk, "Registration with the Point Cloud Library: A Modular Framework for Sligning in 3-D. ", IEEE Robotics & Automation Magazine, 22(4), pp. 110-124, 2015.
[20] Lowe, D. G., "Distinctive Image Features from Scale-Invariant Keypoints. ", International Journal of Computer Vision, 60(2), pp. 91-110, 2004.
[21] Harris, M. Stephens, "A Combined Coner and Edge Detector. ", Alvey vision conference, Vol. 15, pp.10-5244, 1988.
[22] C. Peng, J. Krumm, "Object Recognition with Color Cooccurrence Histograms. ", Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 498-504, 1999.
[23] D. Fernández, R. Arroyo, M. Sotelo, "Vehicle Logo Recognition in Traffic Images using HOG Features and SVM. ", 16th International IEEE Conference on Intelligent Transportation Systems, pp. 2229-2234, 2013.
[24] S. Ravi, C. V. Jawahar, "Word Image Retrieval using Bag of Visual Words. ", 2012 10th IAPR International Workshop on Document Analysis Systems, pp. 297-301, 2012.
[25] J. Armand, E. Grave, "Bag of Tricks for Efficient Text Classification. ", arXiv preprint arXiv:1607.01759, 2016.

[26] L. Omer, Y. Goldberg, "Dependency-Based Word Embeddings. ", Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Vol. 2, 2014.
[27] R. B. Rusu, N. Blodow, "Aligning Point Cloud Views using Persistent Feature Histograms. ", 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3384-3391, 2008.
[28] F. Tombari, "Unique Signatures of Histograms for Local Surface Description. ", European Conference on Computer Vision, pp. 356-369, 2010.
[29] T. Federico, Samuele Salti, Luigi Di Stefano, "A Combined Texture-Shape Descriptor for Enhanced 3D Feature Matching. ", 2011 18th IEEE International Conference on Image Processing, pp. 809-812, 2011.
[30] R. B. Rusu, G. Bradski, "Fast 3D Recognition and Pose using the Viewpoint Feature Histogram. ", 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems , Taipei, Taiwan, pp. 2155-2162, 2010.
[31] A. Aldoma, N. Blodow, "CAD Model Recognition and 6 DOF Pose Estimation using 3D Cues. ", ICCV 3D Representation and Recognition, pp. 585-592, 2011.
[32] A. Aldoma, F. Tombari, "OUR-CVFH: Oriented, Unique and Repeatable Clustered Viewpoint Feature Histogram for Object Recognition and 6 DOF Pose Estimation. ", Joint DAGM (German Association for Pattern Recognition) and OAGM Symposium, pp. 113-122, 2012.
[33] A. Aldoma, Z. Aitor, " Point Cloud Library: Three-Dimensional Object Recognition and 6 DoF Pose Estimation. ", IEEE Robotics & Automation Magazine, 19(3), 2012.
[34] H. Dirk, "Registration with the Point Cloud Library: A Modular Framework for Aligning in 3-D. ", IEEE Robotics & Automation Magazine, 22(4), 2015
[35] M. S. Ahmed, "3D Shape Retrieval using Bag of Word Approaches. ", 2019 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), pp. 1-7, 2019.
[36] Y. M. Chen, P. L. Sun, C. J. Li, H. Y. Chen, "A Simulation Platform to Convert Viewing Geometry of a Transparent Display for AR Application. ", Internation Display Workshoop (IDW'18), Japan, Nagoya, 2018.
[37] H. Linjia, S. Nooshabadi, "Massive Parallelization of Approximate Nearest Neighbor Search on KD-tree for High-Dimensional Image Descriptor Matching. ", Journal of Visual Communication and Image Representation, pp. 106-115, 2017.

[38] H. Jian, J. Kang, N. Qi, "On Vocabulary Size in Bag-of-Visual-Words Representation. ", Pacific-Rim Conference on Multimedia, pp. 414-424, 2010.