本論文嘗試結合HTS的頻譜HMM模型與ANN抖音模型，來建造一個顯著提升自然度之國語歌聲合成系統。在訓練階段，使用STRAIGHT去分析各音框的基頻，接著設計問題集，以使用HTS軟體來訓練音高HMM及訓練頻譜HMM模型與決策樹。在合成階段，先令HTS engine合成出初始歌聲，再令ANN抖音模型依時長資訊去產生各音節具有抖音特性的音高軌跡，然後取代HTS原本產生的音高軌跡，如此就可讓HTS合成出音高正確且具有抖音的歌聲。此外，我們也考慮了音符滿度的設定，調整合成歌聲的振幅，以消除刺耳的聲音。對於合成的歌聲，我們量測了MGC頻譜係數的誤差，並且進行主觀聽測實驗，結果顯示，修改音高後的合成歌聲，其自然度會比HTS原本合成的歌聲提高很多。

In this thesis, a Mandarin singing voice synthesis system is constructed by combining HTS (HMM-based speech synthesis system) trained HMM spectrum models and ANN (artificial neural network) vibrato models. This system is intended to promote the naturalness level of the synthesized singing voice. In the training stage, STRAIGHT is used to analyze the fundamental frequency of each signal frame. Then, we design question sets for the software, HTS, to train HMM fundamental frequency models, HMM spectrum models and decision trees. In the synthesis stage, we first command the HTS engine to synthesize an initial singing voice. Then, the HMM state-staying durations are sent to the ANN vibrato models to generate a natural pitch contour for each lyric syllable. Next, the pitch contour generated by HTS are replaced so that HTS can be enforced to synthesize a new singing voice that not only has correct melody but also has vibrato characteristic. In addition, we consider the occupying rate of a note’s duration, adjust the amplitude of the synthetic singing voice to decrease harsh noise. As to the quality of the synthetic singing voice, average spectral error in terms of MGC (mel generalized cepstrum) coefficients is measured, and listening tests are conducted. The results show that the synthetic singing voice with pitch-replacing is better in naturalness level than the original synthetic singing voice by HTS.

[1] B. Boashash, “Estimating and interpreting the instantaneous frequency of a signal, Part I: Fundamentals”, Proceedings of the IEEE, Vol. 80, pp. 519-538, April 1992.
[2] B. Boashash, “Estimating and interpreting the instantaneous frequency of a signal. Part 2: Algorithms and applications”, Proceedings of the IEEE, Vol. 80, pp. 539-568, April 1992.
[3] C. Dodge, and T. A. Jerse, Computer Music: Synthesis, composition, and performance, second ed., Schirmer Books, 1997.
[4] E. Prame, “Vibrato extent and intonation in professional western lyric singing”, J. Acoust. Soc. Am., Vol. 102, pp. 616-621, 1997.
[5] E. Prame, “Measurements of the vibrato rate of ten singers”, J. Acoust. Soc. Am., Vol. 96, pp. 1979-1984, 1994.
[6] E. Prame, “Vibrato extent and intonation in professional western lyric singing”, J. Acoust. Soc. Am., Vol. 102, pp. 616-621, 1997.
[7] H. Kawahara, I. Masuda-Katsuse and A. de Cheveigne’, “Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequencybased F0 extraction,” Speech Communication 27, pp. 187–207, 1999.
[8] H. Zen, K. Tokuda, K. Oura, K. Hashimoto, S. Shiota, S. Takaki, J. Yamagishi, T. Toda, T. Nose, S. Sako, Alan W. Black, HMM-based Speech Synthesis System (HTS), http://hts.sp.nitech.ac.jp/
[9] HTK, “Forced Alignment”, https://netfiles.uiuc.edu/tyoon/www/ForcedAlignment.htm.
[10] I. Arroabarren, et al.,“Measurement of vibrato in lyric singers”, IEEE instrumentation and measurement technology conference, pp. 1529-1534, 2001.
[11] J. C. Brown, and K. V. VAUGHN, “Pitch center of stringed instrument vibrato tones“, J. Acoust. Soc. Am. 100, 1728–1735. 1996.
[12] J. Bonada, X. Serra, “Synthesis of the singing voice by performance sampling and spectral models,” IEEE Signal Processing Magazine March 2007.
[13] J. Schoukens, R. Pintelon, and H. Van Hamme,“The interpolated fast fourier transform: A comparative study”, IEEE trans. Instrum. Meas., Vol. 41, pp. 226-232, April 1992.
[14] J. Sundberg, “Effects of the vibrato and the ‘singing formant’ on pitch”, Musica Slovaca VI, 1978, Bratislava, 51–69; also J. Res. Singing 5(2), 5–17. 1978.
[15] J. Sundberg, E. Prame, and J. Iwarsson, “Replicability and Accuracy of Pitch Patterns in Professional Singers“, in Vocal Fold Physiology, edited by P. J. Davis and N. H. Fletcher (Singular, San Diego), 1996.
[16] K. Tokuda, H. Zen, and A.W. Black. “An hmm-based speech synthesis system applied to english,”Proc. IEEE 2002 Workshop on Speech Synthesis, Santa Monica, USA, Sep. 2002.
[17] K. Kato, et al., “Blending vocal music with the sound field - the effective duration of autocorrelation function of western professional singing voices with different vowels and pitches”, International Symposium on Musical Acoustics (ISMA2004), Nara, Japan, 2004.
[18] K. Saino, H. Zen, Y. Nankaku, A. Lee, K. Tokuda, “An HMM-based Singing Voice Synthesis System,” INTERSPEECH 2006 – ICSLP.
[19] K. Sjolander and J. Beskow, Centre of Speech Technolodge at KTH, http://www.speech.kth.se/wavesurfer/.
[20] MATLAB, http://www.mathworks.com/products/matlab/.
[21] P. Howes, et al.,“The relationship between measured vibrato characteristics and perception in western operatic singing”, Journal of Voice, Vol. 18, pp. 216-230, 1997.
[22] Park, Younsung, Sungrack Yun, and Chang Dong Yoo. "Parametric emotional singing voice synthesis." Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on. IEEE, 2010.
[23] Rabiner, Lawrence, and B. Juang. "An introduction to hidden Markov models." ASSP Magazine, IEEE 3.1 (1986): 4-16.
[24] S. Imaizumi, H. Saida, Y. Shimura and H. Hirose, “Harmonic analysis of the singing voice:—Acoustic characteristics of vibrato“, in Proceedings of the Stockholm Music Acoustics Conference (SMAC93) Royal Swedish Academy of Music, Stockholm, pp. 197–200. 1994.
[25] S. W. Lee, Shen Ting Ang, Minghui Dong, and Haizhou Li, "Generalized F0 modelling with absolute and relative pitch features for singing voice synthesis", Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on. IEEE, 2012.
[26] S. Young, G. Evermann, T. Hain, D. Kershaw, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, P. Woodland, The HTK Book( for HTK version 3.2.1), Cambridge University Engineering Department, 2002.
[27] Shinoda, Koichi, and Takao Watanabe. "MDL-based context-dependent subword modeling for speech recognition." JOURNAL-ACOUSTICAL SOCIETY OF JAPAN-ENGLISH EDITION- 21.2 (2000): 79-86.
[28] Shinoda, Koichi, and Takao Watanabe. "Speaker adaptation with autonomous model complexity control by MDL principle." Acoustics, Speech, and Signal Processing, 1996. ICASSP-96. Conference Proceedings., 1996 IEEE International Conference on. Vol. 2. IEEE, 1996.
[29] Shonle, J. I., and Horan, K. E. “The pitch of vibrato tones“, J. Acoust. Soc. Am. 67, 246–252. 1980.
[30] Sinsy, “HMM-based Singing Voice Synthesis System,” http://www.sinsy.jp/.
[31] SPTK Working Group, Speech Signal Processing Toolkit (SPTK), http://sp-tk.sourceforge.net/
[32] T. Toda and K. Tokuda, "A speech parameter generation algorithm considering global variance for HMM-based speech synthesis." IEICE TRANSACTIONS on Information and Systems 90.5 (2007): 816-824.
[33] Tokuda, et al. "Mel-generalized cepstral analysis-a unified approach to speech spectral estimation." ICSLP. Vol. 94. 1994.
[34] Y. Horii, “Acoustic analysis of vocal vibrato: a theoretical interpretation of data“, J. Voice 3, 36–43. 1989.
[35] Yamaha, VOCALOID, New Singing Synthesis Technology, http://www.vocaloid.com/en/.
[36] 王如江，基於歌聲表情分析與單元選擇之國語歌聲合成研究，國立台灣科技大學資訊工程研究所碩士論文，2007.
[37] 林正甫，使用ANN抖音參數模型之國語歌聲合成，國立台灣科技大學資訊工程研究所碩士論文，2008.
[38] 陳安璿，整合MIDI伴奏之歌唱聲合成系統，國立台灣科技大學資訊工程研究所碩士論文，台北，2004.
[39] 校園民歌回顧，一品文化出版，台北，1985.
[40] 張世穎，結合HTS頻譜模型與ANN韻律模型之國語語音合成系統，國立台灣科技大學資訊工程研究所碩士論文，台北，2013.
[41] 葉怡成，類神經網路模式應用與實作, 儒林圖書公司, 2006.
[42] 華堃，歌唱聲以及樂器聲合成改進之研究，國立台灣科技大學資訊工程研究所碩士論文，台北，2011.
[43] 廖皇量，國語歌聲合成信號品質改進之研究，國立台灣科技大學資訊工程研究所碩士論文，2006.
[44] 簡延庭，基於 HMM 模型之歌聲合成與音色轉換，國立台灣科技大學資訊工程研究所碩士論文，2013.