特徵轉換和選擇是探索分類模型相關新特徵的重要工作。本文研究了兩種非線性特徵變換和一種特徵選擇方案，以使用 Linear Discriminant Analysis (LDA) 作為分類器來提高分類性能。為了探索原始特徵和響應變量之間的非線性關係，我們使用 power 和 logit 函數從每個原始特徵中生成新特徵，其中應用 Gram-Schmidt 過程來生成不相關的新特徵。由於原始特徵通常是相關的，從每個原始特徵生成的不相關的新特徵可能是相關的，導致難以選擇有希望的特徵進行分類。在特徵選擇之前，Principal Component Analysis（PCA）用於將所有特徵轉換為不相關的主成分。
隨後，選擇與響應變量的平方相關排序的不相關主成分作為 LDA 分類模型的有希望的特徵。為了對所提出的方案進行性能評估，我們在 LDA 模型上應用了十倍交叉驗證。這項研究使用了來自 UCI 機器學習存儲庫和Kaggle 的三個醫學數據集（Haberman’s Survival、Diabetic Retinopathy Debrecen 和 Heart Disease）。實驗結果表明，非線性特徵變換和特徵選擇對於提高分類性能起著至關重要的作用。

Feature transformations and selection are important jobs to explore the relevant
new features for classification models. This paper investigates two
non-linear feature transformations and a feature selection scheme to enhance
classification performance using Linear Discriminant Analysis (LDA)
as a classifier. To explore the non-linear relationship between the original
features and the response variable, we used power transformation and the
logit function to generate new features from each original feature where
Gram-Schmidt process is applied for generating uncorrelated new features.
Since the original feature are often correlated, the uncorrelated new features
generated from each original feature may be correlated, resulting in
difficulty in selecting promising features for classification. Before feature
selection, Principal Component Analysis (PCA) is used to transform all
features into uncorrelated principal components. Subsequently, the uncorrelated
principal components which are ranked by the squared correlations
with the response variable are selected as the promising features for LDA
classification model. For performance evaluation of the proposed scheme,
we applied ten-fold cross-validation on the LDA model. This study used
three medical data sets (Haberman’s Survival, Diabetic Retinopathy Debrecen,
and Heart Disease) from the UCI machine learning repository and
Kaggle. Experimental results showed that non-linear feature transformations
and feature selection play an essential role for improving the classification
performance.

[1] M. Z. Abedin, G. Chi, M. M. Uddin, M. S. Satu, M. I. Khan, and P. Hajek, “Tax default prediction using feature transformation-based machine learning,” IEEE Access, vol. 9, pp. 19864–19881, 2020.
[2] D. Jain and V. Singh, “A two-phase hybrid approach using feature selection and adaptive svm for chronic disease classification,” International Journal of Computers and Applications, vol. 43, no. 6, pp. 524–536, 2021.
[3] C. Sahu and M. Banavar, “A nonlinear feature transformation-based multi-user classification algorithm for keystroke dynamics,” in 2021 55th Asilomar Conference on Signals, Systems, and Computers, pp. 1448–1452, IEEE, 2021.
[4] A. G. Asuero, A. Sayago, and A. Gonzalez, “The correlation coefficient: An overview,” Critical reviews in analytical chemistry, vol. 36, no. 1, pp. 41–59, 2006.
[5] J. Spilka, V. Chudacek, M. Koucky, Michal, L. Lhotska, M. Huptych, P. Janku, G. Georgoulas, and C. Stylios, “Using nonlinear features for fetal heart rate classification,” Biomedical signal processing and control, vol. 7, no. 4, pp. 350–357, 2012.
[6] B. Hosseinifard, M. H. Moradi, and R. Rostami, “Classifying depression patients and normal subjects using machine learning techniques and nonlinear features from eeg signal,” Computer methods and programs in biomedicine, vol. 109, no. 3, pp. 339–345, 2013.
[7] H. Li, X. Feng, L. Cao, C. Zhang, C. Tang, E. Li, H. Liang, and X. Chen, “Heartbeat classification using different classifiers with non-linear feature extraction,” Transactions of the Institute of Measurement and Control, vol. 38, no. 9, pp. 1033–1040, 2016.
[8] M. Yamada, J. Tang, J. Lugo-Martinez, E. Hodzic, R. Shrestha, A. Saha, H. Ouyang, D. Yin, H. Mamitsuka, C. Sahinalp, et al., “Ultra high-dimensional nonlinear feature selection for big biological data,” IEEE Transactions on Knowledge and Data Engineering, vol. 30, no. 7, pp. 1352–1365, 2018.
[9] C. Yan, “A simple transformation for nonlinear waves,” Physics Letters A, vol. 224, no. 1-2, pp. 77–84, 1996.
[10] M. R. Marler, P. Gehrman, J. L. Martin, and S. Ancoli-Israel, “The sigmoidally transformed cosine curve: a mathematical model for circadian rhythms with symmetric non-sinusoidal shapes,” Statistics in medicine, vol. 25, no. 22, pp. 3893–3904, 2006.
[11] Z. Hu, Y. Bao, T. Xiong, and R. Chiong, “Hybrid filter–wrapper feature selection for short-term load forecasting,” Engineering Applications of Artificial Intelligence, vol. 40, pp. 17–27, 2015.
[12] Y. Xue, L. Zhang, B. Wang, Z. Zhang, and F. Li, “Nonlinear feature selection using gaussian kernel svm-rfe for fault diagnosis,” Applied Intelligence, vol. 48, no. 10, pp. 3306–3331, 2018.
[13] V. Kumar and S. Minz, “Feature selection: a literature review,” SmartCR, vol. 4, no. 3, pp. 211–229, 2014.
[14] A. Kalousis, J. Prados, and M. Hilario, “Stability of feature selection algorithms: a study on highdimensional spaces,” Knowledge and information systems, vol. 12, no. 1, pp. 95–116, 2007.
[15] H. Liu and H. Motoda, “Feature transformation and subset selection,” IEEE Intell Syst Their Appl, vol. 13, no. 2, pp. 26–28, 1998.
[16] E. Bloedorn and R. S. Michalski, “Data-driven constructive induction: methodology and applications,” in Feature Extraction, Construction and Selection, pp. 51–68, Springer, 1998.
[17] B. Zupan, M. Bohanec, J. Demšar, and I. Bratko, “Feature transformation by function decomposition,” in Feature Extraction, Construction and Selection, pp. 325–340, Springer, 1998.
[18] J. Huang, G. Li, Q. Huang, and X. Wu, “Joint feature selection and classification for multilabel learning,” IEEE transactions on cybernetics, vol. 48, no. 3, pp. 876–889, 2017.
[19] F. Song, D. Mei, and H. Li, “Feature selection based on linear discriminant analysis,” in 2010 international conference on intelligent system design and engineering application, vol. 1, pp. 746–749, IEEE, 2010.
[20] A. Shadvar and A. Erfanian, “Mutual information-based fisher discriminant analysis for feature extraction and recognition with applications to medical diagnosis,” in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, pp. 5811–5814, IEEE, 2010.
[21] R. S. El-Sayed, “Linear discriminant analysis for an efficient diagnosis of heart disease via attribute filtering based on genetic algorithm.,” J. Comput., vol. 13, no. 11, pp. 1290–1299, 2018.
[22] K. Alabdulwahhab, W. Sami, T. Mehmood, S. Meo, T. Alasbali, and F. Alwadani, “Automated detection of diabetic retinopathy using machine learning classifiers,” European Review for Medical and Pharmacological Sciences, vol. 25, no. 2, pp. 583–590, 2021.
[23] R. Spencer, F. Thabtah, N. Abdelhamid, and M. Thompson, “Exploring feature selection and classification methods for predicting heart disease,” Digital health, vol. 6, p. 2055207620914777, 2020.
[24] M. U. Emon, R. Zannat, T. Khatun, M. Rahman, M. S. Keya, et al., “Performance analysis of diabetic retinopathy prediction using machine learning models,” in 2021 6th International Conference on Inventive Computation Technologies (ICICT), pp. 1048–1052, IEEE, 2021.
[25] B. Zamani, A. Akbari, and B. Nasersharif, “Evolutionary combination of kernels for nonlinear feature transformation,” Information Sciences, vol. 274, pp. 95–107, 2014.
[26] Y. Peng, Z. Wu, and J. Jiang, “A novel feature selection approach for biomedical data classification,” Journal of Biomedical Informatics, vol. 43, no. 1, pp. 15–23, 2010.
[27] M. M. Kabir, M. M. Islam, and K. Murase, “A new wrapper feature selection approach using neural network,” Neurocomputing, vol. 73, no. 16-18, pp. 3273–3283, 2010.
28] E. Zorarpaci and S. A. Ozel, “A hybrid approach of differential evolution and artificial bee colony for feature selection,” Expert Systems with Applications, vol. 62, pp. 91–103, 2016.
[29] M. Aladeemy, S. Tutun, and M. T. Khasawneh, “A new hybrid approach for feature selection and support vector machine model selection based on self-adaptive cohort intelligence,” Expert Systems with Applications, vol. 88, pp. 118–131, 2017.
[30] R. Kohavi and G. H. John, “Wrappers for feature subset selection,” Artificial intelligence, vol. 97, no. 1-2, pp. 273–324, 1997.
[31] N. Sanchez-Marono, A. Alonso-Betanzos, and M. Tombilla-Sanroman, “Filter methods for feature selection–a comparative study,” in International Conference on Intelligent Data Engineering and Automated Learning, pp. 178–187, Springer, 2007.
[32] R. K. Sivagaminathan and S. Ramakrishnan, “A hybrid approach for feature subset selection using neural networks and ant colony optimization,” Expert systems with applications, vol. 33, no. 1, pp. 49– 60, 2007.
[33] A. Kusiak, “Feature transformation methods in data mining,” IEEE Transactions on Electronics packaging manufacturing, vol. 24, no. 3, pp. 214–221, 2001.
[34] T. Mu, A. K. Nandi, and R. M. Rangayyan, “Classification of breast masses via nonlinear transformation of features based on a kernel matrix,” Medical & Biological Engineering & Computing, vol. 45, no. 8, pp. 769–780, 2007.
[35] G. Zu, W. Ohyama, T. Wakabayashi, and F. Kimura, “Accuracy improvement of automatic text classification based on feature transformation,” in Proceedings of the 2003 ACM symposium on Document engineering, pp. 118–120, 2003.
[36] J.-D. Lee, H.-R. Su, P. E. Cheng, M. Liou, J. A. Aston, A. C. Tsai, and C.-Y. Chen, “Mr image segmentation using a power transformation approach,” IEEE transactions on medical imaging, vol. 28, no. 6, pp. 894–905, 2009.
[37] L. Tichy, S. M. Hennekens, P. Novak, J. S. Rodwell, J. H. Schaminee, and M. Chytry, “Optimal transformation of species cover for vegetation classification,” Applied Vegetation Science, vol. 23, no. 4, pp. 710–717, 2020.
[38] H. M. Parsons, C. Ludwig, U. L. Gunther, and M. R. Viant, “Improved classification accuracy in 1-and 2-dimensional nmr metabolomics data using the variance stabilising generalised logarithm transformation,” BMC bioinformatics, vol. 8, no. 1, pp. 1–16, 2007.
[39] A. Cubero-Fernandez, F. Rodriguez-Lozano, R. Villatoro, J. Olivares, J. M. Palomares, et al., “Efficient pavement crack detection and classification,” EURASIP Journal on Image and Video Processing, vol. 2017, no. 1, pp. 1–11, 2017.
[40] A. Singh and N. Kingsbury, “Dual-tree wavelet scattering network with parametric log transformation for object classification,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2622–2626, IEEE, 2017.
[41] V. Bolon-Canedo, N. Sanchez-Marono, and A. Alonso-Betanzos, “A review of feature selection methods on synthetic data,” Knowledge and information systems, vol. 34, no. 3, pp. 483–519, 2013.
[42] V. Singh and S. Pathak, “Feature selection using classifier in high dimensional data,” arXiv preprint arXiv:1401.0898, 2014.
[43] J. Li, K. Cheng, S. Wang, F. Morstatter, R. P. Trevino, J. Tang, and H. Liu, “Feature selection: A data perspective,” ACM computing surveys (CSUR), vol. 50, no. 6, pp. 1–45, 2017.
[44] M. Shokouhifar and S. Sabet, “A hybrid approach for effective feature selection using neural networks and artificial bee colony optimization,” in 3rd international conference on machine vision (ICMV 2010), pp. 502–506, 2010.
[45] J. Bennet, C. Ganaprakasam, and N. Kumar, “A hybrid approach for gene selection and classification using support vector machine.,” International Arab Journal of Information Technology (IAJIT), vol. 12, 2015.
[46] F. Kherif and A. Latypova, “Principal component analysis,” in Machine Learning, pp. 209–225, Elsevier, 2020.
[47] H. Abdi and L. J. Williams, “Principal component analysis,” Wiley interdisciplinary reviews: computational statistics, vol. 2, no. 4, pp. 433–459, 2010.
[48] T. Kurita, “Principal component analysis (pca),” Computer Vision: A Reference Guide, pp. 1–4, 2019.
[49] O. Altay and M. Ulas, “Prediction of the autism spectrum disorder diagnosis with linear discriminant analysis classifier and k-nearest neighbor in children,” in 2018 6th international symposium on digital forensic and security (ISDFS), pp. 1–4, IEEE, 2018.
[50] A. Neale, M. Kummert, and M. Bernier, “Discriminant analysis classification of residential electricity smart meter data,” Energy and Buildings, vol. 258, p. 111823, 2022.
[51] P. Xanthopoulos, P. M. Pardalos, and T. B. Trafalis, “Linear discriminant analysis,” in Robust data mining, pp. 27–33, Springer, 2013.
[52] B. Kolukisa, H. Hacilar, G. Goy, M. Kus, B. Bakir-Gungor, A. Aral, and V. C. Gungor, “Evaluation of classification algorithms, linear discriminant analysis and a new hybrid feature selection methodology for the diagnosis of coronary artery disease,” in 2018 IEEE International Conference on Big Data (Big Data), pp. 2232–2238, IEEE, 2018.
[53] S. Rathi, B. Kaur, and R. Agrawal, “Selection of relevant visual feature sets for enhanced depression detection using incremental linear discriminant analysis,” Multimedia Tools and Applications, vol. 81, no. 13, pp. 17703–17727, 2022.
[54] Y. B. Wah, N. Ibrahim, H. A. Hamid, S. Abdul-Rahman, and S. Fong, “Feature selection methods: Case of filter and wrapper approaches for maximising classification accuracy.,” Pertanika Journal of Science & Technology, vol. 26, no. 1, 2018.
[55] D. A. Aljawad, E. Alqahtani, A.-K. Ghaidaa, N. Qamhan, N. Alghamdi, S. Alrashed, J. Alhiyafi, and S. O. Olatunji, “Breast cancer surgery survivability prediction using bayesian network and support vector machines,” in 2017 International Conference on Informatics, Health & Technology (ICIHT), pp. 1–6, IEEE, 2017.
[56] N. Cahyana, S. Khomsah, and A. S. Aribowo, “Improving imbalanced dataset classification using oversampling and gradient boosting,” in 2019 5th International Conference on Science in Information Technology (ICSITech), pp. 217–222, IEEE, 2019.