研究生: |
蔡唯翔 Wei-shiang Tsai |
---|---|
論文名稱: |
基於生成影像與多模型集成之皮膚病變分類 Skin Lesion Classification Based on Multi-Model Ensemble with Generated Levels-of-Detail Images |
指導教授: |
林昌鴻
Chang-Hong Lin |
口試委員: |
林宗男
Tsungnan Lin 吳沛遠 Pei-Yuan Wu 呂政修 Jenq-Shiou Leu |
學位類別: |
碩士 Master |
系所名稱: |
電資學院 - 電子工程系 Department of Electronic and Computer Engineering |
論文出版年: | 2020 |
畢業學年度: | 108 |
語文別: | 英文 |
論文頁數: | 57 |
中文關鍵詞: | 皮膚病變分類 、深度學習 、資料擴增 、資料平衡 、模型合奏 |
外文關鍵詞: | Skin Lesion Classification, Deep Learning, Data Augmentation, Data Balancing, Model Ensemble |
相關次數: | 點閱:249 下載:0 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
皮膚癌已經多年排在癌症發生率的第一名,根據世界衛生組織統計,隨著紫外線每年不斷地增強,皮膚癌的人數也會持續增加,由於皮膚癌的發生過程仍然沒有一個可參考的依據,導致醫生難以評估皮膚癌的惡化程度, 所以使用深度學習為皮膚病變做分類已經成為了現行的方法。然而在皮膚病變的影像上,良性的資料非常的常見且容易收集,但惡性的資料卻往往因為較少見且病人不願意提供詳細資訊,使得惡性的資料非常難以收集,因此導致醫學資料集在各類別的影像數量相當的不平衡,而這些極度不平衡的資料更使得深度神經網路(Deep Neural Network,簡稱DNN)在學習的過程中,偏向去預測屬於數量較多的類別。
在本篇論文提出一種數據平衡系統來解決數據不平衡的問題,我們的數據平衡系統可分成幾個方法,首先使用對抗生成網路(Generative Adversarial Network,簡稱GAN)的生成器來產生三種不同解析度的影像,再將資料集個別使用其中一種解析度來平衡,並將這些被不同種解析度平衡的資料集拿來訓練深度網路獲得各別的模型,最後再將這些模型對於測試資料集的預測值集合後再評估為最終的預測機率,這樣可以有效的提升平均召回率以及準確率。藉由我們提出的方法分別實驗在ISIC-2018 [1]與ISIC-2019 [2]的資料集上,並在測試集上取得82.1%與62.5%各別的平均召回率。
Skin cancer has been the top one of global cancer incidence. With regard to that the reliable registration of these cancers has not been achieved yet, the temporal trends of the incidence of skin cancers are difficult to determine. Therefore, using deep learning has become the essential element of the skin lesion classification. Even though, the benign skin lesion is common, patients with malignant skin lesions are reluctant to provide the information, which leads to extremely unbalanced skin lesion datasets, and the deep learning networks tend to classify the testing data into the category with a larger number of images, which is the benign lesion.
We propose data balanced methods to solve this data imbalanced problem, including data augmentation, data balancing, and the multi-model ensemble. First of all, we use a generator of the Generative Adversarial Network (GANs) to generate images. Second, we use the generated images to balance the number of images in each category. Finally, we adopt the Deep Neural Networks (DNN) to train the model with the balanced data of different resolutions. Moreover, the ensemble of these model’s prediction value can improve the performance of the mean recall and accuracy. With our proposed method, we are able to achieve the mean recall of 82.1% and 62.5% on the test set of ISIC-2018 [1] and ISIC-2019 [2].
[1] "ISIC-2018 web." [Online]. https://challenge2018.isic-archive.com/ (accessed April, 2020).
[2] "ISIC-2019 web." [Online]. https://challenge2019.isic-archive.com/ (accessed April, 2020).
[3] K. Ramlakhan and Y. Shang, "A mobile automated skin lesion classification system," in IEEE 23rd International Conference on Tools with Artificial Intelligence, pp. 138-141, 2011.
[4] B. Ö. Cakir, P. Adamson, and C. Cingi, "Epidemiology and economic burden of nonmelanoma skin cancer," Facial plastic surgery clinics of North America, vol. 20, no. 4, pp. 419-422, 2012.
[5] C. P. Wild, B. W. Stewart, and C. Wild, "World cancer report 2014," World Health Organization Geneva, vol. 5, no. 14, 2014.
[6] L. E. Dubas and A. Ingraffea, "Nonmelanoma skin cancer," Facial Plastic Surgery Clinics, vol. 21, no. 1, pp. 43-53, 2013.
[7] T. Karras, S. Laine, and T. Aila, "A style-based generator architecture for generative adversarial networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4401-4410, 2019.
[8] P. Tschandl, C. Rosendahl, and H. Kittler, "The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions," Scientific Data, vol. 5, p. 180161, 2018.
[9] N. Codella, V. Rotemberg, P. Tschandl, M. E. Celebi, S. Dusza, D. Gutman, B. Helba, A. Kalloo, K. Liopyris, and M. Marchetti, "Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (ISIC)," arXiv preprint arXiv:1902.03368, 2019.
[10] N. C. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, and H. Kittler, "Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC)," in IEEE 15th International Symposium on Biomedical Imaging, pp. 168-172, 2018.
[11] M. Combalia, N. C. Codella, V. Rotemberg, B. Helba, V. Vilaplana, O. Reiter, A. C. Halpern, S. Puig, and J. Malvehy, "Bcn20000: Dermoscopic lesions in the wild," arXiv preprint arXiv:1908.02288, 2019.
[12] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700-4708, 2017.
[13] X. Huang and S. Belongie, "Arbitrary style transfer in real-time with adaptive instance normalization," in Proceedings of the IEEE International Conference on Computer Vision, pp. 1501-1510, 2017.
[14] S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in International Conference on Machine Learning, 2015.
[15] T. Karras, T. Aila, S. Laine, and J. Lehtinen, "Progressive growing of gans for improved quality, stability, and variation," International Conference on Learning Representations, vol. 1710.10196, 2018.
[16] A. Krizhevsky and G. Hinton, "Learning multiple layers of features from tiny images," M.S.thesis,Department of Computer Science, University of Toronto, 2009.
[17] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, "Reading digits in natural images with unsupervised feature learning," Workshop on Neural Information Processing Systems, 2011.
[18] D. Jia, D. Wei, S. Richard, L.-J. Li, K. Li, and F.-F. Li, "Imagenet: A large-scale hierarchical image database," in IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
[19] V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," in Proceedings of the 27th International Conference on Machine Learning, pp. 807-814, 2010.
[20] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
[21] M. Lin, Q. Chen, and S. Yan, "Network in network," in International Conference on Learning Representations, 2013.
[22] G. Lemaître, F. Nogueira, and C. K. Aridas, "Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning," The Journal of Machine Learning Research, vol. 18, no. 1, pp. 559-563, 2017.
[23] E. Ramentol, Y. Caballero, R. Bello, and F. Herrera, "SMOTE-RSB*: a hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using SMOTE and rough sets theory," Knowledge and Information Systems, vol. 33, no. 2, pp. 245-265, 2012.
[24] H. Han, W.-Y. Wang, and B.-H. Mao, "Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning," in International Conference on Intelligent Computing, pp. 878-887, 2005.
[25] H. Haibo and E. A. Garcia, "Learning from Imbalanced Data," IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9, pp. 1263-1284, 2009.
[26] G. King and L. Zeng, "Logistic regression in rare events data," Political Analysis, vol. 9, no. 2, pp. 137-163, 2001.
[27] M. Zhu, J. Xia, X. Jin, M. Yan, G. Cai, J. Yan, and G. Ning, "Class weights random forest algorithm for processing class imbalanced medical data," IEEE Access, vol. 6, pp. 4641-4652, 2018.
[28] Q. Dong, S. Gong, and X. Zhu, "Imbalanced Deep Learning by Minority Class Incremental Rectification," IEEE Trans Pattern Anal Mach Intell, vol. 41, no. 6, pp. 1367-1381, 2019.
[29] G. Mariani, F. Scheidegger, R. Istrate, C. Bekas, and C. Malossi, "Bagan: Data augmentation with balancing gan," arXiv preprint arXiv, vol. 1803.09655, 2018.
[30] C. Shorten and T. M. Khoshgoftaar, " A survey on image data augmentation for deep learning," Journal of Big Data, vol. 6, no. 60, 2019.
[31] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, "Gans trained by a two time-scale update rule converge to a local nash equilibrium," in Advances in Neural Information Processing Systems, pp. 6626-6637, 2017.
[32] Y. Wu, J. Donahue, D. Balduzzi, K. Simonyan, and T. Lillicrap, "LOGAN: Latent Optimisation for Generative Adversarial Networks," arXiv preprint arXiv, vol. 1912.00953, 2019.
[33] A. Brock, J. Donahue, and K. Simonyan, "Large scale gan training for high fidelity natural image synthesis," arXiv preprint arXiv, vol. 1809.11096, 2018.
[34] S. Laine, "Feature-based metrics for exploring the latent space of generative models," International Conference on Learning Representations, vol. 2018, 2018.
[35] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, "Autoaugment: Learning augmentation strategies from data," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 113-123, 2019.
[36] F. Nachbar, W. Stolz, T. Merkle, A. B. Cognetta, T. Vogt, M. Landthaler, P. Bilek, O. Braun-Falco, and G. Plewig, "The ABCD rule of dermatoscopy: high prospective value in the diagnosis of doubtful melanocytic skin lesions," Journal of the American Academy of Dermatology, vol. 30, no. 4, pp. 551-559, 1994.
[37] Y. Xiao, J. Wu, Z. Lin, and X. Zhao, "A deep learning-based multi-model ensemble method for cancer prediction," Computer Methods and Programs in Biomedicine, vol. 153, pp. 1-9, 2018.
[38] J. Li, W. Chen, Y. Sun, Y. Li, and Z. Peng, "Object Detection Based on DenseNet and RPN," in IEEE Chinese Control Conference, pp. 8410-8415, 2019.
[39] Z. Abai and N. Rajmalwar, "DenseNet Models for Tiny ImageNet Classification," arXiv preprint arXiv, vol. 1904.10429, 2019.
[40] D. Opitz and R. Maclin, "Popular ensemble methods: An empirical study," Journal of Artificial Intelligence Research, vol. 11, pp. 169-198, 1999.
[41] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv, vol. 1412.6980, 2014.
[42] "Tensorboard." [Online]. [https://www.tensorflow.org/tensorboard/] (accessed April, 2020).
[43] "TensorFlow " [Online]. [https://www.tensorflow.org/] (accessed April, 2020).
[44] "Keras Website." [Online]. [https://keras.io] (accessed April, 2020).
[45] K. M. Li and E. C. Li, "Skin lesion analysis towards melanoma detection via end-to-end deep learning of convolutional neural networks," arXiv preprint arXiv:1807.08332, 2018.
[46] A. Bissoto, F. Perez, V. Ribeiro, M. Fornaciali, S. Avila, and E. Valle, "Deep-learning ensembles for skin-lesion segmentation, analysis, classification: RECOD titans at ISIC challenge 2018," arXiv preprint arXiv, vol. 1808.08480, 2018.
[47] A. Aldwgeri and N. F. Abubacker, "Ensemble of Deep Convolutional Neural Network for Skin Lesion Classification in Dermoscopy Images," in International Visual Informatics Conference, pp. 214-226, 2019.
[48] Y. C. Lee, S.-H. Jung, and H.-H. Won, "WonDerM: Skin lesion classification with fine-tuned neural networks," arXiv preprint arXiv, vol. 1808.03426, 2018.
[49] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in International Conference on Learning Representations, pp. 1-14, 2015.
[50] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, "Inception-v4, inception-resnet and the impact of residual connections on learning," in Thirty-first AAAI Conference on Artificial Intelligence, 2017.
[51] Y. Pan and Y. Xia, "Residual Network based Aggregation Model for Skin Lesion Classification," arXiv preprint arXiv, vol. 1807.09150, 2018.
[52] M. Goyal and J. C. Rajapakse, "Deep neural network ensemble by data augmentation and bagging for skin lesion classification," arXiv preprint arXiv, vol. 1807.05496, 2018.
[53] A. G. Pacheco, A.-R. Ali, and T. Trappenberg, "Skin cancer detection based on deep learning and entropy to detect outlier samples," arXiv preprint arXiv, vol. 1909.04525, 2019.
[54] Y. Li and L. Shen, "Skin lesion analysis towards melanoma detection using deep learning network," Sensors, vol. 18, no. 2, p. 556, 2018.
[55] N. Gessert, M. Nielsen, M. Shaikh, R. Werner, and A. Schlaefer, "Skin lesion classification using ensembles of multi-resolution EfficientNets with meta data," Elsevier MethodsX, p. 100864, 2020.
[56] M. Tan and Q. V. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," arXiv preprint arXiv, vol. 1905.11946, 2019.