本研究全面評估了在多個標竿表格資料集上，由混合萬用啟發式演算法強化的樹狀機器學習模型在超參數優化(hyperparameter optimization; HPO)上的性能。理論研究強調了混合黏菌演算法及粒子群最佳化演算法(particle swarm optimization with slime mold algorithm; PSO_SMA)在多種指標中的卓越性能，證明了其在二元和多元分類任務中的優越性。在7項指標中，PSO_SMA超越了其他基於萬用啟發式演算法的HPO方法，5項取得了第一名，餘下的兩項取得了第二名，這一結果由統計分析結果支持，其p值小於0.05。
這項研究進一步展示了所提方法在實務上的應用，我們引進了一款低代碼網路應用程式 (low-code web application) 在一家中小型B2C企業–台灣行銷研究有限公司 (Taiwan Marketing Research LTD Co.; TMR)。該程式使TMR能夠快速輸入他們的資料（例如：零售數據）並訓練樹狀模型，如Extreme Gradient Boosting Machine (XGBoost)、Light Gradient Boosting Machine (LightGBM)和Categorical Boosting Machine (Catboost)。這些模型能使用財務指定的目標函數進行評估，並且本研究實施了 PSO_SMA 策略，發現其與傳統的全市場策略相比，更改進了轉換率和利潤效果。

This study presents a comprehensive evaluation of the performance of tree-based machine learning models enhanced by hybrid metaheuristic for hyperparameter optimization (HPO) on multiple benchmark tabular datasets. The theoretical investigation highlights the superior performance of the particle swarm optimization with slime mold algorithm (PSO_SMA) across diverse metrics, demonstrating preeminence in both binary and multi-class classification tasks. PSO_SMA outperformed other metaheuristic-based HPO methods, securing top ranks in 5 out of 7 metrics, and second in the remaining two, as corroborated by a statistical analysis yielding a p-value less than 0.05.
The study further demonstrates the application of these findings in a practical scenario, involving Taiwan Marketing Research LTD Co. (TMR), a small-to-medium-sized B2C enterprise. We introduce a low-code web application that enables TMR to input their data (e.g., retailing data) rapidly and train tree-based models, such as Extreme Gradient Boosting Machine (XGBoost), Light Gradient Boosting Machine (LightGBM), Categorical Boosting Machine (CatBoost). These models are evaluated using a financially specified objective function, and the implementation of the PSO_SMA strategy results in improved conversion rates and profitability compared to the traditional all-market strategy.

Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(2).
Bergstra, J., Yamins, D., & Cox, D. D. (2013). Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms. Proceedings of the 12th Python in science conference, June 24-29, Austin, Texas, USA, 13, 20.
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32.
Brin, S., Motwani, R., Ullman, J. D., & Tsur, S. (1997). Dynamic itemset counting and implication rules for market basket data. ACM SIGMOD Record, 26(2), 255-264. https://doi.org/10.1145/253262.253325
Canuto, S. D., Belém, F. M., Almeida, J. M., & Gonçalves, M. A. (2013). A comparative study of learning-to-rank techniques for tag recommendation. Journal of Information and Data Management, 4(3), 453-453. https://doi.org/https://dx.doi.org/10.5753/jidm.2013.1509
Caruana, R., Niculescu-Mizil, A., Crew, G., & Ksikes, A. (2004). Ensemble selection from libraries of models Proceedings of the twenty-first international conference on Machine learning, Banff, Alberta, Canada. https://doi.org/10.1145/1015330.1015432
Cazzaniga, P., Nobile, M. S., & Besozzi, D. (2015). The impact of particles initialization in PSO: Parameter estimation as a case in point. 2015 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), August 12-15, Niagara Falls, ON, Canada, 1-8.
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 13 - 17, San Francisco, California, USA, 785–794.
Cheng, M.-Y., Huang, K.-Y., & Hutomo, M. (2018). Multiobjective Dynamic-Guiding PSO for Optimizing Work Shift Schedules. Journal of Construction Engineering and Management, 144(9), 04018089. https://doi.org/doi:10.1061/(ASCE)CO.1943-7862.0001548
Chung, H. H. (2020). contract Taiwan Marketing Research LTD Co..
Coello, C. A. C., & Montes, E. M. (2002). Constraint-handling in genetic algorithms through the use of dominance-based tournament selection. Advanced Engineering Informatics, 16(3), 193-203. https://doi.org/https://doi.org/10.1016/S1474-0346(02)00011-3
Conover, W. J. (1999). Practical nonparametric statistics (Vol. 350). john wiley & sons.
Cortez, P., Cerdeira, A., Almeida, F., Matos, T., & Reis, J. (2009). Wine Quality Data Set. https://archive.ics.uci.edu/ml/datasets/wine+quality
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Dhindsa, A., Bhatia, S., Agrawal, S., & Sohi, B. S. (2021). An improvised machine learning model based on mutual information feature selection approach for microbes classification. Entropy, 23(2), 257. https://www.mdpi.com/1099-4300/23/2/257
Eggensperger, K., Feurer, M., Hutter, F., Bergstra, J., Snoek, J., Hoos, H., & Leyton-Brown, K. (2013). Towards an empirical foundation for assessing Bayesian optimization of hyperparameters. NIPS Workshop on Bayesian Optimization in Theory and Practice, 1-5.
Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200), 675-701. https://doi.org/10.2307/2279372
Gogna, A., & Tayal, A. (2013). Metaheuristics: review and application. Journal of Experimental & Theoretical Artificial Intelligence, 25(4), 503-526. https://doi.org/10.1080/0952813X.2013.782347
Hancock, J. T., & Khoshgoftaar, T. M. (2020). CatBoost for big data: an interdisciplinary review [Review]. Journal of Big Data, 7(1), 45, Article 94. https://doi.org/10.1186/s40537-020-00369-8
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, June 27-30, Las Vegas, NV, USA, 770-778.
Howard, F. L. (1931). The life history of Physarum polycephalum. American Journal of Botany, 116-133. https://doi.org/https://doi.org/10.1002/j.1537-2197.1931.tb09577.x
Hu, F., Hu, J., Dai, R., Guan, Y., Shen, X., Gao, B., Wang, K., Liu, Y., & Yao, X. (2023). Selection of characteristic wavelengths using SMA for laser induced fluorescence spectroscopy of power transformer oil. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 288, 122140. https://doi.org/https://doi.org/10.1016/j.saa.2022.122140
Itano, F., Sousa, M. A. d. A. d., & Del-Moral-Hernandez, E. (2018). Extending MLP ANN hyper-parameters Optimization by using Genetic Algorithm. 2018 International Joint Conference on Neural Networks (IJCNN), July 8-13, Rio de Janeiro, Brazil, 1-8.
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255-260. https://doi.org/doi:10.1126/science.aaa8415
Kadiwal, A. (2021). Water Quality. https://www.kaggle.com/datasets/adityakadiwal/water-potability
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). Lightgbm: A highly efficient gradient boosting decision tree. Advances In Neural Information Processing Systems, December 4-9, Long Beach, CA, USA, 30,
Kessler, D. (1982). Plasmodial structure and motility. Cell biology of Physarum and Didymium/edited by Henry C. Aldrich, John W. Daniel.
Koza, J. R., & Rice, J. P. (1992). Automatic programming of robots using genetic programming. Proceedings of the Tenth National Conference on Artificial Intelligence, July 12 - 16, San Jose, California, 194–201.I
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174. https://doi.org/https://doi.org/10.2307/2529310
Lessmann, S., Stahlbock, R., & Crone, S. F. (2005). Optimizing hyperparameters of support vector machines by genetic algorithms. IC-AI, June 27-30, Las Vegas, Nevada, USA, 74, 82.
Li, S. M., Chen, H. L., Wang, M. J., Heidari, A. A., & Mirjalili, S. (2020). Slime mould algorithm: A new method for stochastic optimization. Future Generation Computer Systems-the International Journal of Escience, 111, 300-323. https://doi.org/10.1016/j.future.2020.03.055
Lorenzo, P. R., Nalepa, J., Kawulok, M., Ramos, L. S., & Pastor, J. R. (2017). Particle swarm optimization for hyper-parameter selection in deep neural networks. Proceedings of the Genetic and Evolutionary Computation Conference, July 15 - 19, Berlin, Germany, 481–488.
Manias, D. M., Jammal, M., Hawilo, H., Shami, A., Heidari, P., Larabi, A., & Brunner, R. (2019). Machine learning for performance-aware virtual network function placement. 2019 IEEE Global Communications Conference (GLOBECOM), December 9-13, Waikoloa, HI, USA, 1-6.
MÖBIUS. (2020). HR Analytics. https://www.kaggle.com/datasets/arashnic/hr-analytics-job-change-of-data-scientists
Moro, S., Cortez, P., & Rita, P. (2014). A data-driven approach to predict the success of bank telemarketing. Decision Support Systems, 62, 22-31. https://doi.org/10.1016/j.dss.2014.03.001
Nayak, J., Naik, B., Behera, H. S., & Ieee. (2014). A Hybrid PSO-GA based Pi Sigma neural network (PSNN) with standard back propagation gradient descent learning for classification. 2014 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT), July 10-11, Kanyakumari, INDIA, 878-885.
Nematzadeh, S., Kiani, F., Torkamanian-Afshar, M., & Aydin, N. (2022). Tuning hyperparameters of machine learning algorithms and deep neural networks using metaheuristics: A bioinformatics study on biomedical and biological cases. Computational Biology and Chemistry, 97, Article 107619. https://doi.org/https://doi.org/10.1016/j.compbiolchem.2021.107619
Nguyen, L. T. K., Chung, H. H., Tuliao, K. V., & Lin, T. M. Y. (2020). Using XGBoost and skip-gram model to predict online review popularity. Sage Open, 10(4), 17. https://doi.org/10.1177/2158244020983316
Oord, A. v. d., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499.
Örnek, B. N., Aydemir, S. B., Düzenli, T., & Özak, B. (2022). A novel version of slime mould algorithm for global optimization and real world engineering problems: Enhanced slime mould algorithm. Mathematics and Computers in Simulation, 198, 253-288. https://doi.org/https://doi.org/10.1016/j.matcom.2022.02.030
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., & Dubourg, V. (2011). Scikit-learn: Machine learning in Python. the Journal of Machine Learning Research, 12, 2825-2830.
Popescu, P. S., & Cojocaru, I. (2022). Driving Behavior. https://www.kaggle.com/datasets/outofskills/driving-behavior
Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V., & Gulin, A. (2018). CatBoost: unbiased boosting with categorical features. Proceedings of the 32nd International Conference on Neural Information Processing Systems, December 3-8, Montréal, Canada, 6639–6649.
Raji, I. D., Bello-Salau, H., Umoh, I. J., Onumanyi, A. J., Adegboye, M. A., & Salawudeen, A. T. (2022). Simple deterministic selection-based genetic algorithm for hyperparameter tuning of machine learning models. Applied Sciences-Basel, 12(3), Article 1186. https://doi.org/10.3390/app12031186
Safavian, S. R., & Landgrebe, D. (1991). A survey of decision tree classifier methodology. IEEE Transactions on Systems, Man, and Cybernetics, 21(3), 660-674. https://doi.org/10.1109/21.97458
Salojärvi, J., Puolamäki, K., Simola, J., Kovanen, L., Kojo, I., & Kaski, S. (2005). Inferring relevance from eye movements: Feature extraction. Workshop at NIPS 2005, in Whistler, BC, Canada, on December 10, 2005., 45.
Samrin, N. A., & Pramanik, M. A. (2022). Students Adaptability Level in Online Education. https://www.kaggle.com/datasets/mdmahmudulhasansuzan/students-adaptability-level-in-online-education
Sanders, S., & Giraud-Carrier, C. (2017). Informing the Use of Hyperparameter Optimization Through Metalearning. 2017 IEEE International Conference on Data Mining (ICDM), November 18-21, New Orleans, LA, USA, 1051-1056.
Sharma, A. K., Saxena, A., & Palwalia, D. K. (2023). Oppositional Slime Mould Algorithm: Development and application for designing demand side management controller. Expert Systems with Applications, 214, 18, Article 119002. https://doi.org/10.1016/j.eswa.2022.119002
Shi, Y., & Eberhart, R. C. (1998). Parameter selection in particle swarm optimization. Evolutionary Programming VII, 1998//, Berlin, Heidelberg, 591-600.
Shwartz-Ziv, R., & Armon, A. (2022). Tabular data: Deep learning is not all you need. Information Fusion, 81, 84-90. https://doi.org/10.1016/j.inffus.2021.11.011
Statnikov, A., Wang, L., & Aliferis, C. F. (2008). A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification. BMC Bioinformatics, 9(1), 1-10.
Steinholtz, O. S. (2018). A comparative study of black-box optimization algorithms for tuning of hyper-parameters in deep neural networks. Luleå University of Technology.
Tang, R., Wang, W., Tu, Z., & Lin, J. (2018). An Experimental Analysis of the Power Consumption of Convolutional Neural Networks for Keyword Spotting. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), April 15-20, Calgary, AB, Canada, 5479-5483.
Tomczak, M., & Tomczak, E. (2014). The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in Sport Sciences, 1(21), 19-25.
Truong, Q.-T., Nguyen, M., Dang, H., & Mei, B. (2020). Housing Price Prediction via Improved Machine Learning Techniques. Procedia Computer Science, 174, 433-442. https://doi.org/https://doi.org/10.1016/j.procs.2020.06.111
Whitley, D. (1994). A genetic algorithm tutorial. Statistics and Computing, 4(2), 65-85. https://doi.org/10.1007/BF00175354
Wolberg, W., Street, W., & Mangasarian, O. (1995). Breast Cancer Wisconsin (Diagnostic). https://archive-beta.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+diagnostic
Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67-82. https://doi.org/10.1109/4235.585893
Wu, J. H., Man, W. S., Zhang, Z. Y., & Ieee. (2018). Short-term load forecasting based on GA-PSO optimized extreme learning machine. 2018 2nd IEEE Conference on Energy Internet and Energy System Integration (EI2), October 20-22, Beijing, China,
Wu, S. B., Heidari, A. A., Zhang, S. Y., Kuang, F. J., & Chen, H. L. (2023). Gaussian bare-bone slime mould algorithm: performance optimization and case studies on truss structures. Artificial Intelligence Review, 37. https://doi.org/10.1007/s10462-022-10370-7
Yan, J., Xu, Y. T., Cheng, Q., Jiang, S. Q., Wang, Q., Xiao, Y. J., Ma, C., Yan, J. B., & Wang, X. F. (2021). LightGBM: accelerated genomically designed crop breeding through ensemble learning. Genome Biology, 22(1), 24, Article 271. https://doi.org/10.1186/s13059-021-02492-y
Yang, L., & Shami, A. (2020). On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing, 415, 295-316. https://doi.org/10.1016/j.neucom.2020.07.061
Yao, Q., Wang, M., Chen, Y., Dai, W., Li, Y.-F., Tu, W.-W., Yang, Q., & Yu, Y. (2018). Taking human out of learning applications: A survey on automated machine learning. arXiv preprint arXiv:1810.13306.
Zöller, M.-A., & Huber, M. F. (2021). Benchmark and survey of automated machine learning frameworks. Journal of Artificial Intelligence Research, 70, 409-472. https://doi.org/https://doi.org/10.1613/jair.1.11854
Zhang, H., Petitjean, F., & Buntine, W. (2020). Bayesian network classifiers using ensembles and smoothing. Knowledge and Information Systems, 62(9), 3457-3480. https://doi.org/10.1007/s10115-020-01458-z