本論文研究了一種結合HTS頻譜模型與ANN韻律模型的國語語音合成之系統架構。在訓練階段，關於頻譜係數的分析，我們使用STRAIGHT來求得較準確的頻譜包絡，然後將各音框的頻譜包絡換算成DCC係數；接著使用HTS來訓練頻譜HMM模型及決策樹。在合成階段時，我們自行發展程式來利用HTS訓練之頻譜HMM模型與決策樹，以產生各音框之DCC係數，並且使用ANN韻律模型來產生音長及基週軌跡參數，然後將兩者產生的參數送給HNM信號合成模組，以合成出語音信號。此外我們應用了一種共振峰增強的作法，使得頻譜包絡過度平滑的問題獲得了改善；在音量及基週軌跡方面，我們分別採取了不同內插作法，也使得自然度獲得改善。在系統製作完成之後，我們對於原本的HTS系統和本系統做了一個參數測量及兩項聽測實驗的評估，結果顯示，本論文所製作的系統在合成語音的自然度、清晰度上，都獲得了明顯較佳的表現。

ABSTRACT
In this thesis, a system framework for Mandarin speech synthesis is studied, which combines HTS (HMM-based speech synthesis system) trained spectrum models and ANN (artificial neural network) prosody models. In the training stage, STRAIGHT is used to estimate more accurate spectral envelopes from the training speech frames. The spectral envelope of each frame is converted to DCC (discrete cepstrum coefficients). Then, HMM (hidden Markov model) spectrum models and decision trees are trained by using HTS. In the synthesis stage, we develop programs to utilize HTS trained HMM spectrum models and decision trees in order to generate DCC for each frame. In addition, ANN prosody models are used to generate the parameters of syllable duration and pitch contour. Then, the prosodic parameters and DCC are sent to the HNM (harmonic plus noise model) signal synthesis module to synthesize speech signals. Additionally, we adopt a formant enhancement method to solve the problem of spectral over smoothing. In terms of some interpolation methods to adjust the intensities and pitch heights of the frames around a syllable boundary, the naturalness level is significantly improved. After our system is built, we compare the performances of our system and the original HTS system on DCC generation and two types of listening tests. The results show that our system is not only better on naturalness level but also better on speech clarity level.

[1] D. Cole and S. Sridharan , “Speech enhancement by formant sharpening in the cepstral domain, ” Cole et al. Cepstral formant enhancement,pp.244-249,2002.
[2] C. Dodge, and T. A. Jerse, Computer Music: Synthesis, composition, and performance, 2’nd ed., Schirmer Books, 1997.
[3] C. Huang, Y. shi, J. Zhou, M. Chu, T. Wang and E. Chang, “Segmental tonal modeling for phone set design in mandarin lvcsr,” ICASSP 2004, pp. I – 901- I – 904, 2004.
[4] H. Banno, H. Hata, M. Morise, T. Takahashi, T. Irino and H. Kawahara, “ Implementation of realtime STRAIGHT speech manipulation system,” Acoust. Sci. & Tech. 2007. Vol.28, No.3, pp.140-146, 2007.
[5] H. Kawahara, I. Masuda-Katsuse and A. de Cheveigne’, “Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequencybased F0 extraction,” Speech Communication 27, pp. 187–207 , 1999.
[6] H. Zen, K. Tokuda, K. Oura, K. Hashimoto, S. Shiota, S. Takaki, J. Yamagishi, T. Toda, T. Nose, S. Sako, Alan W. Black, HMM-based Speech Synthesis System (HTS), http://hts.sp.nitech.ac.jp/.
[7] illinois university, HTK, “Forced Alignment,” https://netfiles.uiuc.edu/tyoon/www/ForcedAlignment.htm.
[8] K. Shinoda and T. Watanabe, “Acoustic modeling based on the mdl principle for speech recognition,” Rhodes, Greece, September 22-25, ISCA, 1997.
[9] K. Sjolander and J. Beskow, Centre of Speech Technolodge at KTH, http://www.speech.kth.se/wavesurfer/.
[10] K. Sjolander, The Snack Sound Toolkit, http://www.speech.kth.se/snack/
[11] K. Tokuda, H. Zen, and A.W. Black. “An hmm-based speech synthesis system applied to english,”Proc. IEEE 2002 Workshop on Speech Synthesis, Santa Monica, USA, Sep. 2002.
[12] MathWorks, MATLAB, http://www.mathworks.com/products/matlab/.
[13] P. Birkholz, D. Jackel and B. J. Kroger, “Simulation of losses due to turbulence in the time-varying vocal system,” IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 4, MAY 2007.
[14] P.Woodland, G. Evermann, M. Gales , HTK,http://htk.eng.cam.ac.uk/
[15] S. Imai, “Cepstral anaslysis synthesis on the mel frequency scale,” in Proc. ICASSP-83, Boston, Masachusetts, USA, pp93-96, 1983.
[16] SPTK Working Group , Speech Signal Processing Toolkit (SPTK), http://sp-tk.sourceforge.net/
[17] S. Young, G. Evermann, T. Hain, D. Kershaw, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, P. Woodland, The HTK Book( for HTK version 3.2.1), Cambridge University Engineering Department, 2002.
[18] T. Yoshimuray, K. Tokuday, T. Masukoyy, T. Kobayashiyyand T. Kitamuray, “Duration modeling for hmm-based speech synthesis, ” in Proc. ICSLP, Sydney, Australia,pp.29-32,1998.
[19] T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi and T. Kitamura, “Duration modeling for hmm-dased speech synthesis system,” Proc. of ICSLP, Sydney, Australia, Vol. 2, pp.29-32, 1998.
[20] wikiPedia, “Lagrange polynomial, ”
[21] Y. Stylianou, “Applying the harmonic plus noise model in concatenative speech synthesis,” IEEE Trans. Speech and Audio Processing, Vol. 9, No. 1, pp. 21-29, 2001.
[22] Y.Stylinaou, “Harmonic plus noise models for speech, combined with statistical methods, for speech and speaker modification”, Ph. D. thesis, Ecole National Superieure des Telecommunications, Paris, France, 1996.
[23] Y. Stylianou, “Modeling speech based on harmonic plus noise models”, in Nonlinear Speech Modeling and Applications, eds. G. Chollet et al., Springer-Verlag, Berlin, pp. 244-260, 2005.
[24] 李振宇、林奇嶽 ，“使用隱藏式馬可夫模型為基礎建立中文語音合成系統”，ICL TECHNICAL JOURNAL pp.88-94, 2010.
[25] 吳昌益，使用頻譜演進模型之國語語音合成研究，國立台灣科技大學資訊工程研究所碩士論文，2007.
[26] 陳安璿，整合MIDI伴奏之歌唱聲合成系統，國立台灣科技大學資訊工程研究所碩士論文，台北，2004.
[27] 曹亦岑，使用小型語料類神經網路之國語語音合成韻律參數產生，國立台灣科技大學資訊工程研究所碩士論文，1999.
[28] 賴名彥，結合HMM頻譜模型與ANN韻律模型之國語語音合成系統，2009.
[29] 蔡松峰，GMM為基礎之語音轉換法的改進，國立台灣科技大學資訊工程研究所碩士論文，2009.
[30] 簡延庭，基於HMM模型之歌聲合成與音色轉換，國立台灣科技大學資訊工程研究所碩士論文，2013.
[31] H.-Y. Gu, S.-F. Tsai,“A Discrete-cepstrum Based Spectrum-envelope Estimation Scheme and Its Example Application of Voice Transformation”, Computational Linguistics and Chinese Language Processing ,Vol. 14, No. 4, pp. 363-382., 2009.