音樂遊戲也稱節奏遊戲，是一種讓玩家隨著音樂與節奏做出動作的遊戲類型。遊戲 的內容不外乎一個關鍵，我們稱作「譜面」或「拍譜」，當中定義了該如何遊玩歌曲的一 系列動作。將歌曲編輯成可供玩家遊玩的拍譜可說是音樂遊戲最核心的部分。現今音樂 遊戲的拍譜通常都是以人工的方式進行編輯，而整個遊戲拍譜編輯過程是十分繁瑣且曠 日費時，同時也需消耗大量人力資源，光是一首歌曲可能就要花費數小時的時間來編輯 其譜面。

我們提出了一套方法能夠自動化的根據難易度進行編譜。該方法主要由兩大部分 所構成，遊戲拍點放置以及遊戲拍點類型選擇。其中遊戲拍點放置部分，我們會先使用 預訓練好的模型將音樂的所有 onset (音樂發聲位置) 找出，接著再透過深度學習的方式 將 onset 根據當前我們所選擇難易度的譜面來進行篩選，藉此得到歌曲的合適於當前難 易度的拍點時間資訊。提出一種全新的訓練資料前處理方法及訓練用結構「Onset Filter Chart」，使訓練能更加穩定並大幅減緩訓練資料不平衡之問題。於遊戲拍點類型選擇部 分，我們也使用了深度學習的方法來學習開發人員對於譜面的編排方式，並且也加入音 訊特徵來使模型能更一步的學習到音樂與編排方式的對應關係。

我們的方法讓使用者不需經過繁瑣的人工編輯過程，即可快速且自動化的生成遊戲 拍譜。根據實驗結果，我們的拍點放置模型能夠將較高難易度的譜面所標記近九成以上 的拍點找出，於較低難易度下也可找出近八成左右的拍點。其中，遊戲拍點放置模型是 可獨立使用的，模型給出的拍點放置時間位置資訊對於所有音樂遊戲來說是十分實用的。 而為了確保遊戲拍譜的趣味性，使用人工進行拍譜的最終編輯與檢查是不可避免的，因 此我們的方法旨在輔助開發人員並加速編譜效率，進而減少人力成本。

Rhythm games is a type of game that allows players to make actions in accordance with music and rhythm. The content of the game that we call ”Beatmap” or ”Chart”, which defines a series of actions for how to play. Editing a song into a chart that can be played by players is the core part of a rhythm game. Today’s rhythm game chart are usually edited manually, and the entire chart editing process is very time­consuming and also consumes a lot of human resources. It may take several hours to edit the chart of a song alone

We proposed a method to automatic generate chart according to the difficulty. Our method is composed of two parts, step placement and step selection. In the step placement part, we will use the pre­trained model to find all the onsets (music sounding positions) of the music, and then use deep learning to filter the onsets according to the difficulty. We propose a new training data pre­processing method and training structure ”Onset Filter Chart” to enable training that more stable and deals with data imbalance. In the step selection part, we also used the deep learning to learn the developer’s arrangement of the chart, and also added audio features to enable the model to further learn the correspondence between music and the arrangement.

Our method allows users to quickly and automatically generate chart without going through a tedious manual editing process. According to the experimental results, our step placement model can find nearly 90% or more of the note marked on the chart with a higher difficulty level, and about 80% of the note marked by a lower difficulty level. Among them, the step placement model can be used independently. The timing of the step placement given is very practical for all rhythm games. In order to ensure the interesting of the chart, it is inevitable to use manual for the final editing and checking of the chart. Therefore, our method aims to assist developers and accelerate the efficiency of chart editing, thereby reducing labor costs.

[1] C. Donahue, Z. C. Lipton, and J. McAuley, “Dance dance convolution,” in Interna­ tional conference on machine learning, pp. 1039–1048, PMLR, 2017.
[2] Y. Liang, W. Li, and K. Ikeda, “Procedural content generation of rhythm games using deep learning methods,” in Joint International Conference on Entertainment Computing and Serious Games, pp. 134–145, Springer, 2019.
[3] R. Luc, “Deepsaber: Deep learning for high dimensional choreography,” Masaryk University, 2020.
[4] Wikipedia, “Cytus.” https://zh.wikipedia.org/wiki/Cytus. accessed May 12, 2021.
[5] Wikipedia, “Deemo.” https://zh.wikipedia.org/wiki/Deemo. accessed May 12, 2021.
[6] Wikipedia, “Lanota.” https://zh.wikipedia.org/wiki/Lanota. accessed May 12, 2021.
[7] Wikipedia, “唯舞獨尊.” https://zh.wikipedia.org/wiki/唯舞獨尊Online. accessed May 12, 2021.
[8] J. Schlüter and S. Böck, “Improved musical onset detection with convolutional neu­ ral networks,” in 2014 IEEE international conference on acoustics, speech and sig­ nal processing (icassp), pp. 6979–6983, IEEE, 2014.
[9] S. Böck, F. Korzeniowski, J. Schlüter, F. Krebs, and G. Widmer, “Madmom: A new python audio and music signal processing library,” in Proceedings of the 24th ACM international conference on Multimedia, pp. 1174–1178, 2016.
[10] M.­T.Luong,H.Pham,andC.D.Manning,“Effectiveapproachestoattention­based neural machine translation,” arXiv preprint arXiv:1508.04025, 2015.
[11] S.HochreiterandJ.Schmidhuber,“Longshort­termmemory,”Neuralcomputation, vol. 9, no. 8, pp. 1735–1780, 1997.
35
[12] S. Böck, A. Arzt, F. Krebs, and M. Schedl, “Online real­time onset detection with recurrent neural networks,” in Proceedings of the 15th International Conference on Digital Audio Effects (DAFx­12), York, UK, 2012.
[13] M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE transactions on Signal Processing, vol. 45, no. 11, pp. 2673–2681, 1997.
[14] G.Parascandolo,H.Huttunen,andT.Virtanen,“Recurrentneuralnetworksforpoly­ phonic sound event detection in real life recordings,” in 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 6440–6444, IEEE, 2016.
[15] N.Kalchbrenner,I.Danihelka,andA.Graves,“Gridlongshort­termmemory,”arXiv preprint arXiv:1507.01526, 2015.
[16] P. Hamel, Y. Bengio, and D. Eck, “Building musically­relevant audio features through multiple timescale representations,” in Proc. 13th Int. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), pp. 553–558, 2012.
[17] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International conference on machine learn­ ing, pp. 448–456, PMLR, 2015.