簡易檢索 / 詳目顯示

研究生: 陳士杰
Shih-Chieh Chen
論文名稱: 應用通用啟發式演算法、資料探勘演算法與合併預測以增加分類正確率之研究
A Study of Applying Meta-heuristic Algorithms, Data Mining Algorithms and Combining Predictors to Increase the Classification Accuracy Rate
指導教授: 周碩彥
Shuo-Yan Chou
口試委員: 王孔政
Kung-Jeng Wang
王福琨
Fu-Kwun Wang
陳振明
Jen-Ming Chen
謝中奇
Chung-Chi Hsieh
謝光進
Kong-King Shieh
張聖麟
Sheng-lin Chang
學位類別: 博士
Doctor
系所名稱: 管理學院 - 工業管理系
Department of Industrial Management
論文出版年: 2010
畢業學年度: 98
語文別: 英文
論文頁數: 101
中文關鍵詞: 參數調整屬性篩選啟發式演算法資料探勘演算法合併預測
外文關鍵詞: Parameter tuning, Feature selection, Meta-heuristic algorithm, Data mining algorithm, Combining predictors
相關次數: 點閱:437下載:14
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 分類問題為常見於日常生活且重要的研究議題,例如醫生或許希望在醫學檢測前知道病人是否有癌症,使用者在下載電子郵件前知道是正常郵件或垃圾郵件。資料探勘中決策樹(decision tree)、區別分析(linear discriminant analysis)、倒傳遞類神經網路(back-propagation network)、支援向量機(support vector machine)皆已成功應用在各種領域分類問題中。然而資料探勘演算法在面對不同特性問題時需要不同的參數設定以建立出好的模型,於是試誤法常被用以解決參數決定問題。但其除費時外更可能因參數設定不佳而得到不好的結果,另一方面資料集中通常包含許多屬性但並非所有屬性皆對於建立分類模型有益。
    通用啟發式演算法(meta-heuristic algorithm)已被成功的應用在最佳化問題之中,本論文利用啟發式演算法中模擬退火演算法(simulated-annealing)、粒子群最佳化演算法(particle swarm optimization algorithm)與分散式搜尋演算法(scatter search algorithm)找出在不同問題下對資料探勘演算法合適的參數組合與屬性集合以建立更佳的模型。
    另外不同資料探勘演算法在面對不同特性問題時有其各自的優點與缺點,若能將這些演算法預測整合可得到更佳結果,此種結合多演算法預測稱之為合併預測(combining predictors)。因此本論文利用合併多個經由良好參數設定與屬性篩選之分類器由整併多個分類器之預測更進一步的強化正確率。
    本論文為衡量所提出方法之效果利用UCI公開資料庫中多個資料集做為測試基準與多個改良資料探勘文獻相比較。實驗結果顯示本論文所提出之方法能比其他的研究提供更佳的分類正確率,顯示本論文提出之方法能在不同特性的問題下皆能提供良好的分類正確率。


    Classification problem is an important research issue of real world. For example, the doctor may want to detect the patient has cancer or not before medical test. The user wants to know an email is legitimate or spam before download the mail. The decision tree (DT), linear discriminant analysis (LDA), back-propagation network (BPN), and support vector machine (SVM) of data mining algorithms are popular and can be applied to various areas successfully. However, different problems typically require different parameter settings. Rule of thumb or trial-and-error methods are generally utilized to determine parameter settings. However, these methods may result in poor parameter settings and unsatisfactory results. On the other hand, although a dataset contain many features, not all features are beneficial for classification in data mining algorithms. Due to the population-based meta-heuristics have been applied to various optimization problems successful, three meta-heuristics, including simulated-annealing (SA), particle swarm optimization (PSO), and scatter search (SS), are proposed to obtain appropriate parameters and select the beneficial subsets of features for data mining algorithms in this study.
    Moreover, the different data mining algorithms have their respective advantages and disadvantages, and suitability is influenced by the characteristics of the problem. If the algorithms can function together in the so-called combining predictors, it is expected that better results can be obtained. Therefore, this study adapts combining predictors to further enhance the classification accuracy rate.
    In order to evaluate the performance of the proposed approach, datasets in UCI (University of California, Irvine) data mining database were applied as the test problem set. The experimental results were compared to several well-known, published approaches. The comparative study shows that the proposed approach improved the classification accuracy rate in most datasets. Thus, the proposed approach can be useful to different problems and provided good classification accuracy rate.

    CONTENTS 摘要........................................................................I ABSTRACT ..................................................................II 誌謝......................................................................III CONTENTS...................................................................IV LIST OF TABLES............................................................VII CHAPTER 1 INTRODUCTION......................................................1 1.1. Background.............................................................1 1.2. Research Objectives....................................................3 1.3 Limits of This Dissertation.............................................4 1.4. Organization of Dissertation...........................................5 CHAPTER 2 LITERATURE REVIEW.................................................6 2.1. Four Classification Methods of Data Mining.............................6 2.1.1. Decision Tree Algorithm..............................................6 2.1.2 Discriminant Analysis................................................10 2.1.3 Back-propagation Network.............................................13 2.1.4. Support Vector Machine..............................................15 2.2. Feature Selection.....................................................17 2.3. Combining Predictors (Ensemble).......................................19 2.4. Related Research of Four Classification Methods, Feature Selection and Combining Predictors.......................................................21 CHAPTER 3 THE PROPOSED APPROACHES..........................................28 3.1. Meta-heuristic Approaches.............................................28 3.1.1. Simulated-annealing Algorithm.......................................28 3.1.2. Particle Swarm Optimization Algorithm...............................31 3.1.3 Scatter Search Algorithm.............................................32 3.2. Data Description......................................................34 3.3. Calculation of the Objective Function Value...........................37 3.4. The Proposed SSDT Approach............................................38 3.4.1. The Proposed SSDT Coding Schema.....................................38 3.4.2. The Procedure of SSDT...............................................39 3.4.3. Demonstration of SSDT Procedure.....................................40 3.5. The Proposed PSOLDA Approach..........................................43 3.5.1. The Proposed PSOLDA Coding Schema...................................44 3.5.2. The Procedure of PSOLDA.............................................44 3.5.3. Demonstration of PSOLDA Procedure...................................45 3.6. The Proposed PSOBPN Approach..........................................47 3.6.1. The Proposed PSOBPN Coding Schema...................................47 3.6.2. The Procedure of PSOBPN.............................................48 3.7. The Proposed SASVM Approach...........................................48 3.7.1. The Proposed SASVM Coding Schema....................................49 3.7.2. The Procedure of SASVM..............................................49 3.8. The Proposed Ensemble Approach........................................52 3.8.1. Demonstration of Proposed Ensemble Approach.........................53 3.8.2. Procedure of the Proposed Meta-heuristic Based Ensemble Approach....54 CHAPTER 4 EXPERIMENTAL RESULTS.............................................57 4.1. SSDT Experimental Results.........................................57 4.2. PSOLDA Experimental Results...........................................64 4.3. PSOBPN Experimental Results...........................................68 4.4. SASVM Experimental Results............................................75 4.5. The Ensemble Experimental Results.....................................79 CHAPTER 5 CONCLUSIONS AND FUTURE RESEARCH..................................82 REFERENCES.................................................................87 INDEX......................................................................97 作者簡介...................................................................99

    REFERENCES
    Ahluwalia, M. and Bull, L., Coevolving functions in genetic programming: classification using k-nearest-neighbour, In Banzhaf, W., Daida, J. and Eiben, G., (eds), GECCO-99: Proceedings of the Genetic and Evolutionary Computation Conference, Morgan Kaufmann, Florida, 1999.
    Aitkenhead, M. J., A co-evolving decision tree classification method, Expert Systems with Applications, 2008, 34, 18-25.
    Amanda, J. C., Combining Artificial Neural Nets: Ensemble and Modular Multi-net Systems, Springer, London, 1999.
    Auri, S. K. and Chakraborty, S., Improved recognition of control chart patterns using artificial neural networks, International Journal of Advanced Manufacturing Technology, 36, 2008, 1191-1201.
    Bauer, E. and Kohavi, R., An empirical comparison of voting classification algorithm: bagging, boosting and variants, Data Mining, 1999, 36, 105-139.
    Belhumeur, P. N., Hespanda, J. and Kriegeman, D., Eigenfaces vs fisherfaces: recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 1997, 711-720.
    Berry, M. J. A. and Linoff, G., Data Mining Techniques: for Marking, Sales and Customer Support, John Wiley and Sons, New York, 2004.
    Breiman, L., Stacked regression, Data Mining, 1996, 24, 49-64.
    Breiman, L., Arcing classifiers, Annals of Statistics, 1998, 26, 801-824.
    Breiman, L., Prediction games and arcing algorithm, Neural Computation, 1999, 11, 1493-1517.
    Burgers, C. J. C., A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 1998, 2, 121-167.
    Cadima, J. and Jolliffe, I. T., Loadings and correlations in the interpretation of principal components, Journal of Applied Statistics, 1995, 22, 203-214.
    Carvalho, D. R. and Freitas, A. A., A genetic-algorithm for discovering small-disjunct rules in data mining, Applied Soft Computing, 2002, 2, 75-88.
    Castillo, P. A., Carpio, J. and Merelo, J. J., Evolving multilayer perceptrons, Neural Processing Letters, 2000a, 12, 115-127.
    Castillo, P. A., Merelo, J. J. and Prieto, A., G-prop: global optimization of multilayer perceptrons using Gas, Neurocomputing, 2000b, 35,149-163.
    Chapelle, O., Vapnik, V., Bousquet, O. and Mukherjee, S., Choosing multiple parameters for support vector machines, Data Mining, 2002, 46, 131-159.
    Chen, R. -C. and Hsieh, C. -H., Web page classification based on a support vector machine using a weighed vote schema, Expert Systems with Applications, 2006, 31, 427-435.
    Chiang, L. H. and Pell, R. J., Genetic algorithms combined with discriminant analysis for key variable identification, Journal of Process Control, 2004, 14, 143-155.
    Coello, C. A. C., Pulido, G. T. and Lechuga, M. S., Handling multiple objectives with particle swarm optimization, IEEE Transactions on Evolutionary Computation, 2004, 8, 256-279.
    Dixon, P. W., Corne, D. W. and Oates, M. J., A preliminary investigation of modified XCS as a generic data mining tool, In Lanai, P. L., Stolzmann, W. and Wilson, S., (eds), Advertences in Learning Classifier Systems, Springer, Berlin, 2002.
    Džeroski, S. and Ženko, B., Is combining classifiers better than selecting the best one, Data Mining, 2004, 54, 255-273.
    Eberhart, R. C. and Shi, Y., Comparing inertia weights and constriction factors in particle swarm optimization, Proceedings of IEEE on Evolutionary Computation, 2000, 1, 84-88.
    Fausett, L., Fundamentals of Neural Networks: Architecture, Algorithm and Applications, Prentice-Hall, New Jersey, 1994.
    Fernandez, G. C. J., Discriminate analysis: a powerful classification technique in data mining, Proceedings of the SAS Users International Conference, 2002, 247-256.
    Freitas, A. A., Data Mining and Knowledge Discovery with Evolutionary Algorithm, Springer, USA, 2002.
    Fung, G. and Mangasarian, O. L., Finite Newton method for Lagrangian support vector machine classification, Neurocomputing, 2003, 55, 39-55.
    García-Cuesta, E., Galván, I. M. and de Castro, A. J., Multilayer perceptron as inverse model in a ground-based remote sensing temperature retrieval problem, Engineering Applications of Artificial Intelligence, 2008, 21, 26-34.
    Gardner, R. M., Bieker, J., Elwell, S., Thalman, R. and Rivera, E., Solving tough semiconductor manufacturing problems using data mining, Proceedings of IEEE/SEMI Advanced Semiconductor Manufacturing Conference and Workshop, 2002, 46-55.
    Gashler, M., Giraud-Carrier, C. and Martinez, T., Decision tree ensemble: small heterogeneous is better than large homogeneous, Proceedings of the Seventh International Conference on Data Mining and Applications, 2008, 900-905.
    Geethanjali, M., Slochanal, S. M. R. and Bhavani, R., PSO trained ANN-based differential protection scheme for power transformers, Neurocomputing, 2008, 71, 904-918.
    Ghosh, A. and Jain, L. C., Evolutionary Computation in Data Mining: Studies in Fuzziness and Soft Computing, Springer, Germany, 2004.
    Ghosh, R. and Verma, B., A hierarchical method for finding optimal architecture and weights using evolutionary least square based learning, International Journal of Neural Systems, 2003, 13, 13-24.
    Ghosh-Dastidar, S., Adeli, H. and Dadmehr, N., Principal component analysis-enhanced cosine radial basis function neural network for robust epilepsy and seizure detection, IEEE Transactions on Biomedical Engineering, 2008, 55, 512-518.
    Glover, F., Heuristics for integer programming using surrogate constraints, Decision Science, 1977, 8, 156-166.
    Glover, F., A template for scatter search and path relinking, In Hao, J. K., Lutton, E., Ronald, E., Schoenauer, M. and Snyers, D., (eds), Artificial Evolution, Lecture Notes in Computer Science, Springer, USA, 1998.
    Gold, C., Holub, A. and Sollich, P., Bayesian approach to feature selection and parameter tuning for support vector machine classifiers, Neural Networks, 2005, 18, 693-701.
    Gray, J. B. and Fan, G., Classification tree analysis using TARGET, Computational Statistics and Data Analysis, 2008, 52, 1362-1372.
    Gupta, J. N. D. and Sexton, R. S., Comparing backpropagation with a genetic algorithm for neural network training, The International Journal of Management Science, 1999, 27, 679-684.
    Guyon, I., Weston, J., Barnhill, S. and Vapnik, V., Gene selection for cancer classification using support vector machines, Data Mining, 2002, 46, 389-422.
    Han, J. and Kamber, M., Data Mining: Concepts and Techniques, Morgan Kaufmann, San Francisco, 2007.
    Haq, N. A. and Ramanan, R. T., A bicriterian flow shop scheduling using artificial neural network, International Journal of Advanced Manufacturing Technology, 2006, 30, 1132-1138.
    Heo, G. -S. and Oh, I. -S., Simultaneous node pruning of input and hidden layers using genetic algorithms, Proceedings of IEEE the Seventh International Conference on Data Mining and Cybernetics, 2008, 3428-3433.
    Hettich, S., Blake, C. L. and Merz, C. J., UCI repository of data mining databases, 1998, Available at http://www.ics.uci.edu/~mlearn/MLRepository.html
    Holsapple, C. W., Handbook on Knowledge Management: Knowledge Directions, Springer, USA, 2004.
    Howland, P. and Park, H., Generalized discriminant analysis using the generalized singular value decomposition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 8, 995-1006.
    Hou, T. -H., Su, C. -H. and Chang, H. -Z., Using neural networks and immune algorithms to find the optimal parameters for an IC wire bonding process, Expert Systems with Applications, 2008, 34, 427-436.
    Hsu, C. -W., Chang, C. -C. and Lin, C. -J., A practical guide to support vector classification, Technical Report, University of National Taiwan, Department of Computer Science and Information Engineering, 2003, 1-12.
    Huang, C. -Y., Chen, L. -H. and Chen, Y. -L., Evaluating the process of a genetic algorithm to improve the back-propagation network: a Monte Carlo study, Expert Systems with Application, 2009, 36, 1459-1465.
    Jack, L. B. and Nandi, A. K., Fault detection using support vector machines and artificial neural networks, augmented by genetic algorithms, Mechanical Systems and Signal Processing, 2002, 16, 373-390.
    John, G. H., Cross-validated C4.5: using error estimation for automatic parameter selection, Technical Report, Computer Science Department, Stanford University, 1994, 1-6.
    Jolliffe, I. T., Principal Component Analysis, Springer, New York, 2002.
    Kalaichelvi, V., Sivakumar, D., Karthikeyan, R. and Palanikumar, K., Prediction of the flow stress of 6061 Al-15% SiC-MMC composites using adaptive network based fuzzy inference system, Materials and Design, 2009, 30, 1362-1370.
    Kennedy, J. and Eberhart, R., Particle swarm optimization, Proceedings of IEEE Conference on Neural Network, 1995, 4, 1942-1948.
    Khaw, J. F. C., Lim, B. S. and Lim, L., Optimal design of neural networks using Taguchi method, Neurocomputing, 1995, 7, 225-245.
    Kim, G. -H., Yoon, J. -E., An, S. -H., Cho, H. -H. and Kang, K. -I., Neural network model incorporating a genetic algorithm in estimating construction costs, Building and Environment, 2004, 39, 1333-1340.
    Kim, H. and Koehler, G. J., Theory and practice of decision tree induction, Omega, 1995, 23, 637-652.
    Kim, K. -J. and Han, I., Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index, Expert System with Application, 2000, 19, 125-132.
    Kirkpatrick, S., Jr. Gelatt, C. D. and Vecchi, M. P., Optimization by simulated annealing, Science, 1983, 220, 671-680.
    Kittler, J., Hatef, M., Duin, R. P. W. and Matas, J., On combining classifiers, IEEE Transactions on Patten Analysis and Machine Intelligence, 1998, 20, 226-239.
    Kohavi, R. and John, G., Automatic parameter selection by minimizing estimated error, In Prieditis, A. and Russell, S., (eds), Data Mining: Proceeding of the Twelfth International Conference, Morgan Kaufmann, USA, 1995.
    Kohavi, R. and John, G. H., Wrappers for feature subset selection, Artificial Intelligence, 1997, 97, 273-324.
    Kuncheva, L. and Whitaker, C., Measures of diversity in classifier ensembles, Data Mining, 2003, 51, 181-207.
    Laguna, M. and Martí, R., Scatter Search: Methodology and Implementations in C, Kluwer Academic Publishers, Boston, 2003.
    Last, M., Kandel, A. and Maimon, O., Information-theoretic algorithm for feature selection, Pattern Recognition Letters, 2001, 22, 799-811.
    Lattin, J., Carroll, D. and Green, P., Analyzing Multivariate Data, Duxbury, USA, 2003.
    Lee, E. T. and Wang, J. W., Statistical Methods for Survival Data Analysis, Wiley, Boston, 2003.
    Lee, T. -L., Back-propagation neural network for the prediction of the short-term storm surge in Taichung harbor, Taiwan, Engineering Applications of Artificial Intelligence, 2008, 21, 63-72.
    Lee, Z. -J., Ying, K. -C., Chen, S. -C. and Lin, S. -W., Applying PSO-based BPN for predicting the yield rate of DRAM modules produced using defective ICs, International Journal of Applied Management and Technology, 2009, 10.1007/s00170-009-2448-2.
    Lezoray, O. and Cardot, H., A neural network architecture for data classification, International Journal of Neural Systems, 2001, 11, 33-42.
    Li, T., Zhu, S. and Ogihara, M., Using discriminant analysis for multi-class classification: an experimental investigation, Knowledge and Information Systems, 2006, 10, 453-472.
    Liang, Z., Li, Y. and Shi, P., A note on two-dimensional linear discriminant analysis, Pattern Recognition Letters, 2008, 29, 2122-2128.
    Liao, Y., Fang, S. -C. and Nuttle, H. L. W., A neural network model with bounded-weights for pattern classification, Computers and Operations Research, 2004, 31, 1411-1426.
    Lin, H. -T. and Lin, C. -J., A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods, Technical Report, University of National Taiwan, Department of Computer Science and Information Engineering, 2003, 1-32.
    Lin, S. -W., Chou, S. -Y. and Chen, S. -C., Irregular shapes classification by back-propagation neural networks, The International Journal of Advanced Manufacturing Technology, 2007, 34, 1164-1172.
    Lin, S. -W., Lee, Z. -J., Chen, S. -C. and Tseng, T. -Y., Parameter determination of support vector machine and feature selection using simulated annealing approach, Applied Soft Computing, 2008a, 8, 1505-1512.
    Lin, S. -W., Tseng, T. -Y., Chou, S. -Y. and Chen, S. -C., A simulated annealing based approach for simultaneous parameter optimization and feature selection of back-propagation networks, Expert Systems with Applications, 2008b, 34, 1491-1499.
    Lin, S. -W., Ying, K. -C., Chen, S. -C. and Lee, Z. -J., Particle swarm optimization for parameter determination and feature selection of support vector machines, Expert Systems with Applications, 2008c, 35, 1817-1824.
    Liu, H. and Motoda, H., Feature Selection for Knowledge Discovery and Data Mining, Kluwer Academic, Boston, 1998.
    López, F. G., Torres, G. M. and Batista, B. M., Solving feature subset selection problem by parallel scatter search, European Journal of Operation Research, 2006, 169, 477-489.
    Malhotra, R. and Malhotra, D. K., Evaluating consumer loans using neural networks, Omega, 2003, 31, 83-96.
    Mangasarian, O. L. and Musicant, D. R., Lagrangian support vector machine, Journal of Data Mining Research, 2001, 1, 161-177.
    Martí, R., Scatter search-wellsprings and challenges, European Journal of Operational Research, 2006, 169, 351-358.
    McClelland, J. L. and Rumelhart, D. E., Explorations in Parallel Distributed Processing: a Handbook of Models, Programs and Exercises, MIT, Cambridge, 1991.
    Melville, P. and Mooney, R. J., Creating diversity in ensembles using artificial data, Information Fusion, 2005, 6, 99-111.
    Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E., Equation of state calculations by fast computing machines, The Journal of Chemical Physics, 1953, 21, 1087-1092.
    Moravej, Z. and Vishwakarma, D. N., ANN-based harmonic restraint differential protection of power transformer, Institution of Engineers, 2003, 84, 1-6.
    Müller, K. R., Mike, S., Rätsch, G., Tsuda, K. and Schölkopf, B., An introduction to kernel-based learning algorithms, IEEE Transactions on Neural Networks, 2001, 12, 181-201.
    Nanni, L. and Lumini, A., Ensemblator: an ensemble of classifiers for reliable classification of biological data, Pattern Recognition Letters, 2006, 28, 622-630.
    Neumann, J., Schnorr, C. and Steidl, G., Combined SVM-based feature selection and classification, Data Mining, 2005, 61, 129-150.
    Nilsson, N. J., Learning Machines: Foundations of Trainable Pattern Classifying Systems, McGraw-Hill, USA, 1965.
    Orsenigo, C. and Vercellis, C., Discrete support vector decision trees via tabu search, Computational Statistics and Data Analysis, 2004, 47, 311-322.
    Osei-Bryson, K. M., Post-pruning in decision tree induction using multiple performance measures, Computers and Operations Research, 2007, 34, 3331-3345.
    Pacheco, J., Casado, S., Núñez, L. and Gómez, O., Analysis of new variable selection methods for discriminant analysis, Computational Statistics and Data Analysis, 2006, 51, 1463-1478.
    Pai, P. -F. and Hong, W. -C., Forecasting regional electricity load based on recurrent support vector machines with genetic algorithms, Electric Power Systems Research, 2005a, 74, 417-425.
    Pai, P. -F. and Hong, W. -C., Support vector machines with simulated annealing algorithms in electricity load forecasting, Energy Conversion and Management, 2005b, 46, 2669-2688.
    Pai, P. -F. and Hong, W. -C., Software reliability forecasting by support vector machines with simulated annealing algorithms, The Journal of Systems and Software, 2006, 79, 747-755.
    Pardo, M. and Sberveglieri, G., Classification of electronic nose data with support vector machines, Sensors and Actuators B Chemical, 2005, 107, 730-737.
    Perner, P. and Apte, C., Empirical evaluation of feature subset selection based on a real-world data set, Engineering Applications of Artificial Intelligence, 2004, 17, 285-288.
    Pudil, P., Novovičová, J. and Kittler, J., Floating search methods in feature selection, Pattern Recognition Letters, 1994, 15, 1119-1125.
    Qin, S., Valle, S. and Piovoso, M., On unifying multi-block analysis with applications to decentralized process monitoring, Journal of Chemometrics, 2001, 15, 715-742.
    Quinlan, J. R., Introduction of decision trees, Data Mining, 1986, 1, 81-106.
    Quinlan, J. R., Simplifying decision trees, International Journal of Man Machine Studies, 1987, 27, 221-234.
    Quinlan, J. R., C4.5: Programs for Data Mining, Morgan Kaufman, New York, 1993.
    Rasha, S. A. W., Monmarché, N., Slimane, M., Moaid, A. F. and Saleh, H. H., A scatter search algorithm for the automatic clustering problem, Lecture Notes in Computer Science, 2006, 4065, 350-364.
    Ravi,V., Reddy, P. J. and Zimmermann, H. -J., Pattern classification with principal component analysis and fuzzy rule bases, European Journal of Operational Research, 2000, 126, 526-533.
    Refenes, A. N., Zapranis, A. and Francis, G., Stock performance modeling using neural networks: a comparative study with regression models, Neural Networks, 1999, 47, 375-388.
    Rencher, A. C., Methods of Multivariate Analysis, Wiley, USA, 2002.
    Rocchi, L., Chiari, L., Cappello, A. and Horak, F. B., Identification of distinct characteristics of postural sway in parkinson's disease: a feature selection procedure based on principal component analysis, Neuroscience Letters, 2006, 394, 140-145.
    Romeijn, H. E. and Smith, R. L., Simulated annealing for constrained global optimization, Journal of Global Optimization, 1994, 5, 101-126.
    Romeijn, H. E., Zabinsky, Z. B., Graesser, D. L. and Neogi, S., New reflection generator for simulated annealing in mixed-integer/continuous global optimization, Journal of Optimization Theory and Applications, 1999, 101, 403-427.
    Salzberg, S. L., On comparing classifiers: pitfalls to avoid and a recommended approach, Data Mining and Knowledge Discovery, 1997, 1, 317-328.
    Samanta, B., Al-Balushi, K. R. and Al-Araimi, S. A., Artificial neural networks and support vector machines with genetic algorithm for bearing fault detection, Engineering Applications of Artificial Intelligence, 2003, 16, 657-665.
    Schittenkopf, C., Deco, G. and Brauer, W., Two strategies to avoid overfitting in feedforward networks, Neural Network, 1997, 10, 505-516.
    SchÖlkopf, B. and Smola, A. J., Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT, London, 2002.
    Sexton, R. S., Dorsery, R. E. and Johnson, J. D., Toward global optimization of neural networks: a comparison of the genetic algorithm and back propagation, Decision Support System, 1998, 22, 171-185.
    Sexton, R. S., Dorsey, R. E. and Johnson, J. D., Optimization of neural networks: a comparative analysis of the genetic algorithm and simulated annealing, European Journal of Operational Research, 1999, 114, 589-601.
    Sexton, R. S., McMurtrey, S. and Cleavenger, D. J., Knowledge discovery using a neural network simultaneous optimization algorithm on a real world classification problem, European Journal of Operational Research, 2006, 168, 1009-1018.
    Sharkey, A. J. C., Combining Artificial Neural Nets: Ensemble and Modular Multi-net Systems, Springer, London, 1999.
    Shon, T., Kim, Y., Lee, C. and Moon, J., A data mining framework for network anomaly detection using SVM and GA, Proceedings of IEEE Workshop on Information Assurance and Security, 2005, 2, 176-183.
    Sivagaminathan, R. K. and Ramakrishnan, S., A hybrid approach for feature subset selection using neural networks and ant colony optimization, Expert Systems with Applications, 2007, 33, 49-60.
    Skapura, D. M., Building Neural Networks, Addison-Wesley Professional, New York, 1995.
    Smilde, A. K., Westerhuis, J. A. and Boqué, R., Multiway multiblock component and covariates regression models, Journal of Chemometrics, 2000, 14, 301-331.
    Smith, M. and Bull, L., GAP: constructing and selection features with evolutionary computing, In Ghosh, A. and Jain, L.C., (eds), Evolutionary Computation in Data Mining, Springer, USA, 2005.
    Sollich, P. and Krogh, A., Learning with ensembles: how overfitting can be useful, Advances in Neural Information Processing Systems, 1996, 8, 190-196.
    Su, C. T. and Shiue, Y. R., Intelligent scheduling controller for shop floor control systems: a hybrid genetic algorithm/decision tree learning approach, International Journal of Production Research, 2003, 41, 2619-2641.
    Su, C. -T., Chen, M. -C. and Chan, H. -L., Applying neural network and scatter search to optimize parameter design with dynamic characteristics, Journal of the Operational Research Society, 2005, 56, 1132-1140.
    Sun, W., Chen, J. and Li, J., Decision tree and PCA-based fault diagnosis of rotating machinery, Mechanical Systems and Signal Processing, 2007, 21, 1300-1317.
    Tsai, H. -C., Hybrid high order neural networks, Applied Soft Computing, 2009, 9, 874-881.
    Tsai, C. -F. and Wu, J. -W., Using neural network ensembles for bankruptcy prediction and credit scoring, Expert Systems with Applications, 2008, 34, 2639-2649.
    van Wezel, M. and Potharst, R., Improved customer choice predictions using ensemble methods, European Journal of Operational Research, 2006, 181, 436-452.
    Verikas, A. and Bacauskiene, M., Feature selection with neural networks, Pattern Recognition Letters, 2002, 23, 1323-1335.
    Wang, H. S., Application of BPN with feature-based models on cost estimation of plastic injection products, Computer and Industrial Engineering, 2007, 53, 79-94.
    Wang, T. Y. and Huang, C. Y., Applying optimized BPN to a chaotic time series problem, Expert Systems with Applications, 2007, 32, 193-200.
    Wang, J., Wu, X. and Zhang, C., Support vector machines based on k-means clustering for real-time business intelligence systems, International Journal of Business Intelligence and Data Mining, 2005, 1, 54-64.
    Wang, G., Zhang, C. and Huang, L., A study of classification algorithm for data mining based on hybrid intelligent systems, Proceeding of IEEE Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 2008, 371-375.
    Wei, Y. and Lin, C. -J., Feature Extraction, Foundations and Applications, Springer, USA, 2005.
    West, D., Dellana, S. and Qian, J., Neural network ensemble strategies for financial decision applications, Computers and Operations Research, 2005, 32, 2543-2559.
    Wold, S., Kettaneh, N. and Tjessem, K., Hierarchical multiblock PLS and PC models for easier model interpretation and as an alternative to variable selection, Journal of Chemometrics, 1996, 10, 463-482.
    Wolpert, D. H., Stacked generalization, Neural Network, 1992a, 5, 241-259.
    Wolpert, D. H., On the connection between in sample testing and generalization error, Complex Systems, 1992b, 6, 47-94.
    Xie, Z. -X., Hu, Q. -H. and Yu, D. -R., Improved feature selection algorithm based on SVM and correlation, Lecture Notes in Computer Science, 2006, 3971, 1373-1380.
    Yang, J. and Honavar, V., Feature subset selection using a genetic algorithm, IEEE Intelligent Systems, 1998, 34, 44-49.
    Yamazaki, A., de Souto, M. C. P. and Ludermir, T. B., Optimization of neural network weights and architectures for odor recognition using simulated annealing, Proceedings of IEEE on Neural Networks, 2002, 1, 547-552.
    Ye, J., Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems, The Journal of Data Mining Research, 2005, 6, 483-502.
    Ye, J. and Li, Q., A two-stage discriminant analysis via QR decomposition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27, 929-941.
    Yeung, D. S. and Zeng, X. -Q., Hidden neuron pruning for multilayer perceptrons using a sensitivity measure, Proceedings of IEEE on Data Mining and Cybernetics, 2002, 4, 1751-1757.
    Zhang, L., Jack, L. B. and Nandi, A. K., Fault detection using genetic programming, Mechanical Systems and Signal Processing, 2005, 19, 271-289.
    Zhou, Z. -H., Jiang, Y., Yang, Y. -B. and Chen, S. -F., Lung cancer cell identification based on artificial neural network ensembles, Artificial Intelligence in Medicine, 2002, 24, 25-36.
    Zhou, Z. -H. and Jiang, Y., Medical diagnosis with C4.5 rule preceded by artificial neural network ensemble, IEEE Transactions on Information Technology in Biomedicine, 2003, 7, 37-42.
    Zurada, J. M., Introduction to Artificial Neural Systems, West Publishing Company, London, 1994.

    QR CODE