在這篇論文，我們提出了兩種適用於平滑支持向量機之赫氏反矩陣更新策略，以及一些實作上的加速計算的技術，來幫助我們快速獲得我們想要知道的資訊，例如：赫氏反矩陣、牛頓方向…等。在實驗的章節，我們利用幾個較具代表性的資料集來測試我們提出的更新策略，結果都顯示出了我們提出的更新策略較原先計算方法有顯著地改進。除此之外，我們提出的這兩種更新策略並不僅僅能用在平滑支持向量機，還能應用在不同種類的支持向量機，例如：最小平方支持向量機。
未來的研究方向，我們將著重在如果利用我們提出的更新策略來應用在逐次學習上面，尤其是單輪逐次學習，我們期望我們提出的更新策略將能發展出更為精確的單輪逐次學習演算法以因應大規模資料。

In this thesis, we proposed two Hessian inverse updating strategies to improve the computational cost of finding Hessian inverse on SSVM and some implementation details to speed-up the performance of Hessian inverse updating and computing Newton direction. In addition, there are some interesting things in Chapter 4.

In our experiments, we demonstrate the effectiveness and speed of SSVM with the Hessian inverse updating strategies by comparing it numerically with SSVM without Hessian inverse updating strategies and LIBSVM.

In addition, our proposed updating strategies not only can apply to updating Hessian inverse efficiently for SSVM, but also can extend to the variants of the SVMs. For example, to improve the performance of the training and performing cross-validation for SVM in the primal and LSSVM, etc.

Furthermore, the Hessian inverse updating strategies is consists of incremental/decremental procedure. Therefore, it can be one of the incremental and decremental SVM types. When the instance is added, it can efficiently update the Hessian inverse by the Hessian inverse updating strategies. In future work, we try to develop a single pass online learning algorithm by our proposed Hessian inverse updating strategies.

[1] MIT Face Recognition Database. Online, 2000.
[2] U Alon. Broad patterns of gene expression revealed by clustering analysis of tu-
mor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the
National Academy of Science, 96:6745{6750, 1999.
[3] C.M. Bishop. Neural networks for pattern recognition. Oxford University Press, USA,
1995.
[4] L. Bottou. Online learning and stochastic approximations. On-line learning in neural
networks, pages 9{42, 1998.
[5] L. Bottou and O. Bousquet. The tradeo®s of large scale learning. Advances in neural
information processing systems, 20:161{168, 2008.
[6] L. Bottou and Y. LeCun. Large scale online learning. Advances in neural information
processing systems, 16:217{224, 2004.
[7] R.L. Burden and J.D. Faires. Numerical Analysis. 2001. Brooks/cole.
[8] Christopher J. C. Burges. A tutorial on support vector machines for pattern recog-
nition. Data Mining and Knowledge Discover, 2(2):121{167, 1998.
[9] G. Cauwenberghs and T. Poggio. Incremental and decremental support vector ma-
chine learning. In Advances in neural information processing systems 13: proceedings
of the 2000 conference, page 409. The MIT Press, 2001.
[10] G.C. Cawley and N.L.C. Talbot. Fast exact leave-one-out cross-validation of sparse
least-squares support vector machines. Neural networks, 17(10):1467{1475, 2004.
[11] N. Cesa-Bianchi, A. Conconi, and C. Gentile. A second-order perceptron algorithm.
In Computational Learning Theory, pages 129{140. Springer, 2002.
[12] C.C. Chang and C.J. Lin. LIBSVM: a library for support vector machines, 2001.
Software available at http://www. csie. ntu. edu. tw/cjlin/libsvm, 2001.
[13] Olivier Chapelle. Training a support vector machine in the primal. Neural Compu-
tation, 19(5):1155{1178, 2007.
75
[14] K.S. Chua. E±cient computations for large least square support vector machine
classi‾ers. Pattern Recognition Letters, 24(1-3):75{80, 2003.
[15] Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Ma-
chines: and other kernel-based learning methods. Cambridge University Press, New
York, NY, USA, 1999.
[16] H. Demuth and M. Beale. Neural network toolbox. MathWorks Version, 4, 2001.
[17] C. Domeniconi and D. Gunopulos. Incremental support vector machine construction.
In Proceedings of the 2001 IEEE International Conference on Data Mining, pages
589{592. Citeseer, 2001.
[18] N. Egidi and P. Maponi. A Sherman-Morrison approach to the solution of linear sys-
tems. Journal of Computational and Applied Mathematics, 189(1-2):703{718, 2006.
[19] Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin.
Liblinear: A library for large linear classi‾cation. J. Mach. Learn. Res., 9:1871{1874,
2008.
[20] V. Franc, P. Laskov, and K.R. M}uller. Stopping conditions for exact computation of
leave-one-out error in support vector machines. In Proceedings of the 25th interna-
tional conference on Machine learning, pages 328{335. ACM, 2008.
[21] G. Fung and O.L. Mangasarian. Incremental support vector machine classi‾cation.
In Proceedings of the Second SIAM International Conference on Data Mining, Ar-
lington, Virginia, pages 247{260. Citeseer, 2002.
[22] T.R. Golub, D.K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J.P. Mesirov,
H. Coller, M.L. Loh, J.R. Downing, M.A. Caligiuri, et al. Molecular classi‾cation of
cancer: class discovery and class prediction by gene expression monitoring. science,
286(5439):531, 1999.
[23] F. Gustavson, A. Karaivanov, J. Wasniewski, and P. Yalamov. Application of the
Sherman-Morrison-Woodbury Formula to a Recursive Algorithm for Symmetric In-
de‾nite Linear Systems? submitted to EuroPar, 99.
[24] William W. Hager. Updating the inverse of a matrix. SIAM Review, 31:221{239,
1989.
[25] N.J. Higham. Accuracy and stability of numerical algorithms. Society for Industrial
Mathematics, 2002.
[26] T. Joachims. Estimating the generalization performance of an SVM e±ciently. In
MACHINE LEARNING-INTERNATIONAL WORKSHOP THEN CONFERENCE-
, pages 431{438. Citeseer, 2000.
[27] J. Kivinen, A.J. Smola, and R.C. Williamson. Online learning with kernels. IEEE
Transactions on Signal Processing, 52(8):2165{2176, 2004.
76
[28] Yuh-Jye Lee, Wen-Feng Hsieh, and Chien-Ming Huang. "{SSVR: A smooth support
vector machine for "{insensitive regression. IEEE Transactions on Knowledge and
Data Engineering, 17(5):678{685., 2005.
[29] Yuh-Jye Lee and Su-Yun Huang. Reduced support vector machines: A statistical
theory. IEEE Transactions on Neural Networks, 18(1):1{13, 2007.
[30] Yuh Jye Lee, Hung-Yi Lo, and Su-Yun Huang. Incremental reduced support vector
machines. In Proceeding of International Conference on Informatics Cybernetics and
System, 2003.
[31] Yuh-Jye Lee and Olvi L. Mangasarian. Rsvm: Reduced support vector machines. In
Proceedings of the First SIAM International Conference on Data Mining, 2001.
[32] Yuh-Jye Lee and Olvi L. Mangasarian. SSVM: A smooth support vector machine for
classi‾cation. Computational Optimization and Applications, 20(1):5{22, 2001.
[33] Li Y. Liang Z. Incremental support vector machine learning in the primal and ap-
plications. Neurocomputing, 72:2249{2258, 2009.
[34] Olvi L. Mangasarian. Generalized support vector machines. Advances in Large
Margin Classi‾ers, pages 135{146, 2000.
[35] MATLAB. User's guide. MathWorks, Inc., Natick, MA, 1992.
[36] K.B. Petersen and M.S. Pedersen. The matrix cookbook. Citeseer, 2006.
[37] B. Sch}olkopf, A.J. Smola, R.C. Williamson, and P.L. Bartlett. New support vector
algorithms. Neural Computation, 12(5):1207{1245, 2000.
[38] S. Smale and Y. Yao. Online learning algorithms. Foundations of computational
mathematics, 6(2):145{170, 2006.
[39] J. A. K. Suykens and J. Vandewalle. Least squares support vector machine classilers.
Neural Processing Letters, 9(3):293{300., 1999.
[40] A. Tveit, M. Hetland, and H. Engum. Incremental and decremental proximal sup-
port vector classi‾cation using decay coe±cients. Data Warehousing and Knowledge
Discovery, pages 422{423, 2003.
[41] V. Vapnik and O. Chapelle. Bounds on error expectation for support vector machines.
Neural computation, 12(9):2013{2036, 2000.
[42] Vladimir Naumovich Vapnik. The Nature of Statistical Learning Theory. Springer,
2000.
[43] M.A. Woodbury. Inverting modi‾ed matrices. Memorandum report, 42:106, 1950.
[44] Yi-Ren Yeh Yuh-Jye Lee and Hsing-Kuo Pao. Introduction to support vector ma-
chines and their applications in bankruptcy prognosis. Handbook of Computational
Finance, expected to publish in November 2010 (invited book chapter).
77
[45]