仿真人鋼琴譜辨識與演奏機器人製作｜國立臺灣科技大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	林永偉 Yung-Wei Lin
論文名稱：	仿真人鋼琴譜辨識與演奏機器人製作 Humanoid Recognizing Piano Scores and Playing Robots Implementation
指導教授：	范欽雄 Chin-Shyurng Fahn
口試委員:	洪一平 none 王聖智 none 王榮華 none 林彥君 none
學位類別：	碩士 Master
系所名稱：	電資學院 - 資訊工程系 Department of Computer Science and Information Engineering
論文出版年：	2010
畢業學年度：	98
語文別：	英文
論文頁數：	79
中文關鍵詞：	光學樂譜辨識、鋼琴彈奏機器人、樣板比對、五線譜偵測、和弦彈奏、避碰機制
外文關鍵詞：	OMR, piano playing robot, template matching, staff detection, chord playing, collision avoidance
相關次數：	點閱：192 下載：2
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

上一筆

本篇論文的目的在於完成一部具有即時辨識鋼琴譜能力且能獨立彈奏的智慧型機器人系統，有鑒於近年來世界各國逐漸重視且開始大力推動機器人產業的發展，在過去，機器人相關設備常應用於工業製造上，對於一般人日常生活的影響十分有限，為了讓人們可以多加暸解智慧型機器人的應用對於增進生活品質的影響，我們設計了這一套仿真人鋼琴譜辨識與彈奏系統，期待藉由此套系統的發展，可以吸引更多人投入機器人相關領域的研究與開發。

此套系統，大體上由兩個子系統所組成，分別為視覺系統與彈奏系統，在視覺系統的部分，我們選擇一般用於視訊監控的PTZ攝影機，雖然這種攝影機不能像數位相機一樣擁有高解析度，但卻可以大角度轉動、傾斜與鏡頭自動對焦、高倍率放大，因而我們可以精準地運用這些功能去模仿人們閱讀鋼琴譜的習慣，以小節做為畫面擷取的單位，將取得的樂譜影像透過影像處理、樂譜符號的偵測與辨識技術得到演奏的資料，並將這些資料利用目前最為廣泛使用的MIDI格式儲存起來，方便後續的其他應用。根據實驗的結果，我們設計的樂譜辨識系統的辨識率達82%。

在彈奏系統部分，是由手指系統與手臂系統所組成，在手指系統部分，我們利用電磁閥通電及運作的原理，模擬人們按壓鋼琴鍵的動作，為了改進彈奏的流暢度盡量減少手臂移動的機率，我們在一個手臂上共配置了16個電磁閥，可以在不移動的情況下彈奏一個八度內的所有琴鍵，並且設計了一個簡單的訊號控制模組，可以接收電腦端送出的演奏訊號，在解碼後驅動相對應的手指執行壓放動作，另外，因為微處理器同一個時間只可以處理一道指令，為了可以彈奏和弦，我們設計了一個編碼方式使手指控制系統同時間可以驅動多個電磁閥來完成合奏的要求；在手臂系統的部分，為了簡化系統的複雜度，並且盡可能的模仿人們彈奏鋼琴的動作，我們利用兩組線性馬達，透過左右的滑移達到按壓不同音域的要求，此外，因為兩組線性馬達的移動範圍有重疊，為了避免彼此碰撞，我們設計了一個避碰機制，使手臂系統可以順利的在各音域之間滑移。根據實驗結果，我們所設計的彈奏系統可以無誤地同時演奏主副旋律並且具備彈奏和旋的能力，另外，避碰機制也發揮了作用，彈奏系統在實際運作中，可以順利的演奏不同音域的音而沒有碰撞發生。經由測試，彈奏系統允許最快的演奏速度為tempo 100。

The purpose of this thesis is to accomplish a playing piano robot which can real-time recognize printed piano score; and then play piano by itself. Recently, many countries in the world gradually focus on the advance industry of intelligent robots. In the past, the application of a robot system is usually in the manufactory, but few in daily life. In order to more understand the effect on improving the quality of life, we implement this playing piano robot and hope that it can attract more people to extend the application of such robot systems.

The playing piano robot is composed of two subsystems: a vision system and a two-hand playing system. On the vision system, we equip a PTZ camera commonly used in video surveillance. Although this kind of cameras does not capture high resolution images like a digital camera, it can pan, tilt, automatically focus, and zoom. Therefore, we may precisely manipulate these functionalities to simulate the human behavior of reading printed piano scores. We use a measure as the unit of image capturing and get its data by image processing and analysis, including the detection and recognition of music notes. Finally, for other applications, we store the music data by well-known data format called “MIDI”. According to the experimental result, our recognition rate of OMR system is 82%.

The two-hand playing system consists of a finger system and an arm system for each hand. Of the finger system, in order to simulate people pressing a piano key, we employ sixteen tubular solenoids which can work when they are electrifying and don’t need to move arm when it presses any keys in the octave. Besides, we design a simple signal control module, which can receive the signal from the host computer and drive the right tubular solenoid after decoded. We also propose a coding method for chord playing because the microprocessor can run just one command at the same time.

We reduce the complexity of the arm system to simulate that people play piano. The arm system is driven by two linear motors which can move different gamut by sliding from left to right and vice versa. In addition, because the sliding ranges of these two linear motors are overlapped, we design a collision avoidance mechanism to ensure their motions are safe. Based on the experimental result, the two-hand playing system can play theme and accompaniment simultaneously and have the ability of playing chord. Beside, the collision avoidance mechanism is useful. There are not any collisions during system operation. The safe speed of playing must be smaller than tempo 100.

致謝                                           v
中文摘要                                                   vi
Abstract                                          viii
List of tables                                                   xvi
Chapter 1Introduction                                           1
1 Overview                                           1
2 Background and motivation                                  2
3 Related work                                                    3
3.1 Some works about detection of staff                         3
3.2 Some works about recognition of music notation                5
3.3 Some works about system of playing piano                7
4 Thesis organization                                           10
Chapter 2 System Description                                          11
1 Hardware system                                           11
2 Vision system                                                    14
2.1 Specification of the camera                                  14
2.2 Specification of the video grabber                         15
2.3 Instruction and Controlling of the camera                16
3 Arm controlling system                                           17
3.1 Specification of the linear motor                         17
3.2 Specification of the AC motor driver                         18
3.3 Instruction and Controlling of the AC motor driver       19
4 Finger controlling system                                   20
4.1 Specification of the tubular solenoid                          20
4.2 Structure of the finger controller                          21
4.3 Instruction and Controlling of the finger controller        22
Chapter 3Two-hand Playing System                                   24
1 Piano keys positioning and the way of playing                 24
2 Collision Avoidance for Arm                                   25
3 Playing and Encoding of the Chord                          26
Chapter 4 Optical Music Recognition                                   31
1 Overview of the OMR system                                   31
2 Image processing methods used for detection                 33
2.1 Color space transformation                                   33
2.2 Binaryzation                                                     35
3 The measure-based OMR module                                   37
3.1 Detection of the printed piano scores                          37
3.2 Sub-image capturing order planning                          41
3.3 Room in position calculation of the camera                 42
4 The detection of music note                                   44
4.1 The computation of Staffspace’s width                          44
4.2 The deletion of staves                                   46
4.3 Connected component labeling used for music note segmentation     49
5 The recognition of music note                                   50
5.1 The decision of voice length                                   50
5.2 The decision of pitch                                            52
6 Storage of the result by MIDI format                          54
Chapter 5 Experimental Results and Discussions                 55
1 The installation of the experimental equipment                 55
2 The result of OMR                                            56
3 The operating result of two-hand playing system                 60
4 The storage result of using MIDI format                          61
Chapter 6Conclusions and Future Works                          62
1 Conclusions                                                     62
2 Future works                                                     63
References                                                      65

                                

[1]http://store.irobot.com/shop/index.jsp?categoryId=2804605
[2]K. Chida, I. Okuma, S. Isoda, Y. Saisu, K. Wakamatsu, K. Nishikawa, J. Solis, H. Takanobu and A. Takanishi, “Development of a New Anthropomorphic Flutist Robot WF-4,” in Proceedings of the IEEE International Conference on Robotics & Automation, New Orleans, LA, pp. 152-157, 2004.
[3]Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, and K. Fujimura, “The intelligent ASIMO system overview and integration,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and System, Lausanne, Switzerland, vol. 3, pp. 2478-2483, 2002.
[4]K. Yoshihiro, “Toyota's violin-playing robot,” International Journal of Industrial Robot, vol. 35, no. 6, pp. 504-506, 2008.
[5]http://www.ucube.com.tw/main.php
[6]H. Miyao and M. Okamoto, “Stave extraction for printed music scores using DP matching,” Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 8, no. 2, pp. 208-215, 2004.
[7]A. Rebelo, A. Capela, J. Costa, C. Guedes, E. Carrapatoso and J. Cardoso, “A shortest path approach for staff line detection,” Third International Conference on Automated Production of Cross Media Content for Multi-channel Distribution, pp. 79-85, 2007.
[8]K. J. Lu, “Humanoid Reading Printed Piano Scores Techniques for Robots,” Master Thesis, Department of Computer Science, National Taiwan University of Science and Technology, Taipei, Taiwan, 2009.
[9]R. O. Duda and P. E. Hart, “Use of the Hough transformation to detect lines and curves in pictures,” Communications of the Association for Computing Machinery, vol. 15, no. 1, pp. 11-15, 1972.
[10]K. T. Reed and J. R. Parker, “Automatic computer recognition of printed music,” in Proceedings of the International Conference on Pattern Recognition, Vienna, Austria, pp. 803-807, 1996.
[11]F. Kimura and M. Shridhar, “Handwritten numerical recognition based on multiple algorithms,” Pattern Recognition, vol. 24, no. 10, pp. 969-983, 1991.
[12]R. J. Randriamahefa, J.P. Cocquerez, C. Fluhr, F. Pépin, and S. Philipp, “Printed music recognition,” in Proceedings of International Conference on Document Analysis and Recognition, Tsukuba Science City, Japan, pp. 898-901, 1993.
[13]http://www.vexforum.com/gallery/showimage.php?i=993&noignore=0
&postid=683.
[14]http://www.mip.sdu.dk/~legobot/am36/tanzania/playbot.html.
[15] http://www.logosfoundation.org/instrum_gwr/playerpiano.html.
[16] D. Zhang, J. Lei, B. Li, D. Lau and C. Cameron, “Design and analysis of a piano playing robot,” in Proceedings of the 2009 IEEE International Conference on Information and Automation, Zhuhai/Macau, China, pp. 757-761, 2009.
[17]N. Otsu, “A threshold selection method from gray level histogram,” IEEE Trans. on Systems, Man, and Cybernetics, SMC-8, 1978, pp. 62-66.
[18]L. He, Y. Chao, K. Suzuki and K. Wu, “Fast connected-component labeling,” Pattern Recognition, vol. 42, no. 9, pp.1977-1987, 2008.

簡易檢索 / 詳目顯示

相關論文