操作者在遠端遙控機器人時，必須依賴各種感測器(sensor)所提供的回饋資訊來掌握遠端，其中最為重要的即為影像資訊。遠端攝影機所能提供的視訊回饋，會直接影響到使用者的操控。為了讓操作者在有限的傳輸頻寬下仍能看到清晰、廣視野、低變形(distortion)、且即時的影像，本論文參考多解析度影像壓縮的研究，提出一個可應用於遠端遙控機器人之影像回饋系統架構，並以實作驗證其可行性，試圖解決因無線網路中頻寬不足造成系統操作性降低的問題。本實驗系統使用Point Grey Research的Ladybug2全景（panorama）攝影機作為實驗用攝影機，使遠端操作者能獲得高解析度的全景影像，並使用OpenGL等硬體加速機制，進行影像處理，對影像進行校正以及拼接，最終將影像以模擬人類視覺的Gaze-contingent display呈現，在符合人類視覺感官的狀況下，進行多解析度壓縮。本研究最終實作出一系統，結合適當的顯示裝置能提供最高180度、隨時任意指定視野之低變形影像，利用平行處理將影像處理時間縮短至110ms，且在1028x768解析度下可達到10FPS的流暢度。
另外我們透過低通濾波器，產生多層次符合人類視覺的多解析度影像，當操作者注視正確的視覺中心時，仍能清楚的辨識影像中心細節，感受不到多解析度影像周圍的失真，以達到影像壓縮，節省60~70%頻寬，在無線網路低頻寬下能提供更佳解析度的影像與視野。相信這些壓縮過的全景影像可以適當的幫助操作者重建機器人所在的環境，並做出適當的操作。

Among all sensory data of a remote controlled robot system, visual feedback provides the most vital information on constructing spatial presence to human operators. In this thesis, we utilize several techniques of remote control robot’s visual systems and image compression on gaze-contingent displays to develop an approach to improve user’s experiment and to reduce wireless network bandwidth. Furthermore, a panorama visual system scheme on remote controlled robot is also implemented.

There are a lot of hazards in a visual feedback system: video delay, narrow field of view, distortions and insufficient resolution of image. The proposed system utilizes Point Grey Research Ladybug2, the panorama camera, as visual feedback device, to provide wide field of view and resolution of streaming image. To compensate time delay, we use OpenGL functions to correct and to stitch images in hardware-accelerated processes. This system also integrates multiresolution compression with gaze-contingent displays to provide high resolution, wide field view and less distortion image in real time. The experimental system is able to provides up to 180 degree wide field of view, low distortions and no mechanical delay. Besides, low-pass filter is used to generate multiresolution images to match human visual acuity. When the user focuses on the foveal region, he or she can still recognize the detail of the compressed image within the range of the foveal region, and would not be aware that the resolution of surrounding image regions is decreased. These compressed images may help controller to reconstruct remote environment and to response correct operations.

[1] NASA’s Spirit and Opportunity <http://marsrovers.jpl.nasa.gov/home/>.
[2] 鍾慶彥, “廣視野視訊回饋應用於遠端機器人遙控系統之生成策略” ,碩士論文, 國立台灣科技大學, 2005.
[3] 施景元, “應用於遠端遙控機器人之廣視野影像回饋系統” ,碩士論文, 國立台灣科技大學, 2006.
[4] M. Sugimoto, G. Kagotani, H. Nii, N. Shiroma, M. Inami, F. Matsuno, “Time Follower's Vision : A Teleoperation Interface with Past Images,” IEEE Computer Graphics and Applications, Vol. 25, No. 1, pp. 54-63, 2005.
[5] G. Klančar, M. Kristan, R. Karba, “Wide-angle camera distortions and non-uniform illumination in mobile robot tracking,” Robotics and Autonomous Systems, Vol. 46, No. 2, pp. 125-133, 2004.
[6] Takashi Nishiyama, Hiroshi Hoshino, Kenshi Suzuki, Ryoji Nakajima, Kazuya Sawada and Susumu Tachi, “Development of Surrounded Audio-Visual Display System for Humanoid Robot Control,” International Conference on Artificial Reality and Telexistence（ICAT）, Vol.9, pp. 40-47, 1999.
[7] Susumu Tachi, Kiyoshi Komoriya, Kazuya sawada, Takashi Nishiyama, Toshiyuki Itoko, Masami Kobayashi, Kozo Inove, “Telexistence Cockpit for Humanoid Robot Control,” Advanced Robotics, Vol. 17, No. 3, pp. 199-217, 2003.
[8] 元宏, 岩田洋夫, “旋回式高解像度像”, 日本第6 回大論文集, Vol.7, No. 1, 2001.
[9] D. Kimber , J. Foote , S. Lertsithichai, “FlyAbout: spatially indexed panoramic video,” Proceedings of the ninth ACM international conference on Multimedia, Vol.9, pp. 339-347, 2001.
[10] Google 20% project – sometimes eight screens are better than one <http://google-latlong.blogspot.com/2009/12/sometimes-eight-screens-are-better-than.html>.
[11] B. Ricks, C. W. Nielsen, M. A. Goodrich, “Ecological Displays for Robot interaction : A new Perspective,” International Conference on Intelligent Robots and Systems （IROS）, Vol. 3, 2855-2860, 2004.
[12] T. Wertheim, ber die indirekte Sehschfe, Zeitschrift fr Psychologie und Physiologie der Sinnesorgane, Vol. 7, pp. 172-178, 1894.
[13] Hahn, P. J. and Mathews, V. J. “Perceptually lossless image compression.” In Proceedings of the Data Compression Industry Workshop, pp.442 , 1997.
[14] G. W. Mcconkie, and K. Rayner, “The span of the effective stimulus during a fixation in reading,” presented at the Annual Meeting of the American Educational Research Assn, Vol. 17, 578-586. 1975.
[15] P. Baudisch, D. Decarlo, A. T. Duchowski, and W. S. Geisler, “Focusing on the essential: Considering attention in display design”, Communications of the ACM, Vol. 46, No. 3, pp. 60-66, 2003.
[16] T. Ienaga, K. Matsunaga, K. Shidoji, M. Otsuru, S. Araki and Y. Matsuki, “An effect of large overlapped area of stereo pairs at the working point on a spatial multi-resolution stereoscopic video system,” Proceedings of IEEE Virtual Reality 2005, No. 6, pp. 277-278, 2005.
[17] T. Naemura, K. Sugita, T. Takano, and H. Harashima, “Multi-Resolution Stereoscopic Immersive Communication Using a Set of Four Cameras,” Proceedings of SPIE Stereoscopic Displays and Applications XI, Vol. 3957, pp. 271-282, 2000.
[18] A. Bernardino, and J. Santos-Victor, “Binocular Visual Tracking: Integration of Perception and Control,” IEEE Transactions on Robotics and Automation, Vol. 15, No. 6, 1999.
[19] G. Sandini, and V. Tagliasco, “A anthropomorphic retina-like structure for scene analysis,” Computer, Graphics and Image Processing, Vol. 14, pp. 365-372, 1980.
[20] E. Schwartz, “Anatomical and Physiological Correlates of Visual Computation from Striate to Infero-Temporal Cortex,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-14, No. 2, pp. 257-271, 1984.
[21] P. Kortum, and W. Geisler, “Implementation of a foveated image coding system for image bandwidth reduction,” Proceedings of SPIE: In Human Vision and Electronic Imaging, Vol. 2657, pp. 350-360, 1996.
[22] W. S. Geisler, and J. S. Perry, “Real-time simulation of arbitrary visual fields,” ACM Symposium on Eye Tracking Research & Applications, pp. 83-87, 2002.
[23] W. S. Geisler, and J. S.Perry, “Variable-resolution displays for visual communication and simulation,” The Society for Information Display, Vol. 30, pp. 420-423, 1999.
[24] E. C. Chang, S. Mallat, and C. Yap, “Wavelet Foveation,” Applied and Computational Harmonic Analysis, Vol. 9, No. 3, pp. 312–335, 2000.
[25] A. T. Duchowski, “Acuity-Matching resolution degradation through wavelet coefficient scaling.” IEEE Transactions Image Process, Vol. 9, No. 8, pp. 1437–1440, 2000.
[26] A. T. Duchowski, “Hardware-Accelerated real-time simulation of arbitrary visual fields,” In Proceedings of the 2004 Symposium on Eye Tracking Research and Applications, pp. 59, 2004.
[27] A. T. Duchowski, A.O ̈ltekin, “Foveated Gaze-Contingent Displays for Peripheral LOD Management, 3D Visualization, and Stereo Imaging” ACM Transactions on Multimedia Computing, Communications and Applications, Vol. 3, Issue. 4, No. 6, 2007.
[28] L. Williams, “Pyramidal Parametrics” ACM SIGGRAPH Computer Graphics, Vol. 17, Issue 3, 1983.
[29] OpenGL Specification < http://www.opengl.org/documentation/specs/>.
[30] D. Shreiner, The Khronos OpenGL ARB Working Group “OpenGL Programming Guide: The Official Guide to Learning OpenGL”, Addison-Wesley Professional; 7 edition, July 31, 2009, ISBN-10: 0321552628.
[31] Light House 3D <http://www.lighthouse3d.com/opengl/>.
[32] OpenGL Shading Language < http://www.opengl.org/documentation/glsl/>
[33] Point Grey Research, “Overview of the Ladybug image stitching process” Technical Application Note TAN2008010, 2008.
[34] W. S. Geisler, J. S. Perry and J. Najemnik, “Visual search: The role of peripheral information”, Journal of Vision, Vol. 6, 858–873, 2006.
[35] ActivMedia Robotics, “PIONEER P3-DX,” <http://www.activrobots.com/ROBOTS/p2dx.html>.
[36] Point Grey Research, “Ladybug JPEG image quality and buffer size settings” Technical Application Note TAN2008012, 2008.
[37] R. S. Wright, “The OpenGL SuperBible: Comprehensive Tutorial and Reference” Addison-Wesley Professional; 4 edition, June 28, 2007, ISBN-10: 0321498828.
[38] R. J. Rost Paperback, “OpenGL Shading Language” Addison-Wesley Professional, 3 edition (July 30, 2009), ISBN-10: 0321637631.
[39] Point Grey Research Knowledge Base <http://www.ptgrey.com/support/kb/>.
[40] FreeImage Project<http://freeimage.sourceforge.net/>.
[41] NeHe OpenGL game developer website<http://nehe.gamedev.net/>.
[42] Ladybug2 SDK<http://www.ptgrey.com/products/ladybugSDK/>.