基於生成影像與多模型集成之皮膚病變分類｜國立臺灣科技大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	蔡唯翔 Wei-shiang Tsai
論文名稱：	基於生成影像與多模型集成之皮膚病變分類 Skin Lesion Classification Based on Multi-Model Ensemble with Generated Levels-of-Detail Images
指導教授：	林昌鴻 Chang-Hong Lin
口試委員:	林宗男 Tsungnan Lin 吳沛遠 Pei-Yuan Wu 呂政修 Jenq-Shiou Leu
學位類別：	碩士 Master
系所名稱：	電資學院 - 電子工程系 Department of Electronic and Computer Engineering
論文出版年：	2020
畢業學年度：	108
語文別：	英文
論文頁數：	57
中文關鍵詞：	皮膚病變分類、深度學習、資料擴增、資料平衡、模型合奏
外文關鍵詞：	Skin Lesion Classification, Deep Learning, Data Augmentation, Data Balancing, Model Ensemble
相關次數：	點閱：237 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

皮膚癌已經多年排在癌症發生率的第一名，根據世界衛生組織統計，隨著紫外線每年不斷地增強,皮膚癌的人數也會持續增加，由於皮膚癌的發生過程仍然沒有一個可參考的依據,導致醫生難以評估皮膚癌的惡化程度，所以使用深度學習為皮膚病變做分類已經成為了現行的方法。然而在皮膚病變的影像上，良性的資料非常的常見且容易收集，但惡性的資料卻往往因為較少見且病人不願意提供詳細資訊，使得惡性的資料非常難以收集，因此導致醫學資料集在各類別的影像數量相當的不平衡，而這些極度不平衡的資料更使得深度神經網路(Deep Neural Network，簡稱DNN)在學習的過程中，偏向去預測屬於數量較多的類別。
在本篇論文提出一種數據平衡系統來解決數據不平衡的問題，我們的數據平衡系統可分成幾個方法，首先使用對抗生成網路(Generative Adversarial Network，簡稱GAN)的生成器來產生三種不同解析度的影像，再將資料集個別使用其中一種解析度來平衡，並將這些被不同種解析度平衡的資料集拿來訓練深度網路獲得各別的模型，最後再將這些模型對於測試資料集的預測值集合後再評估為最終的預測機率，這樣可以有效的提升平均召回率以及準確率。藉由我們提出的方法分別實驗在ISIC-2018 [1]與ISIC-2019 [2]的資料集上，並在測試集上取得82.1%與62.5%各別的平均召回率。

Skin cancer has been the top one of global cancer incidence. With regard to that the reliable registration of these cancers has not been achieved yet, the temporal trends of the incidence of skin cancers are difficult to determine. Therefore, using deep learning has become the essential element of the skin lesion classification. Even though, the benign skin lesion is common, patients with malignant skin lesions are reluctant to provide the information, which leads to extremely unbalanced skin lesion datasets, and the deep learning networks tend to classify the testing data into the category with a larger number of images, which is the benign lesion.
We propose data balanced methods to solve this data imbalanced problem, including data augmentation, data balancing, and the multi-model ensemble. First of all, we use a generator of the Generative Adversarial Network (GANs) to generate images. Second, we use the generated images to balance the number of images in each category. Finally, we adopt the Deep Neural Networks (DNN) to train the model with the balanced data of different resolutions. Moreover, the ensemble of these model’s prediction value can improve the performance of the mean recall and accuracy. With our proposed method, we are able to achieve the mean recall of 82.1% and 62.5% on the test set of ISIC-2018 [1] and ISIC-2019 [2].

摘要    I
ABSTRACT    II
致謝    III
LIST OF CONTENTS    IV
LIST OF FIGURES    VI
LIST OF TABLES    VII
CHAPTER 1    INTRODUCTIONS    1
1.1    Motivation    1
1.2    Contributions    2
1.3    Thesis Organization    3
CHAPTER 2    RELATED WORKS    4
2.1    Generative Adversarial Network    4
2.1.1    StyleGAN [7]    5
2.2    DenseNet [12]    6
CHAPTER 3    PROPOSED METHOD    10
3.1    Data Balancing    11
3.1.1    Generating Image    11
3.1.1.1    The Style-based Generator Architecture    14
3.1.2    Data Augmentation    17
3.2    Multi-model Ensemble    19
3.2.1    DenseNet-201    21
3.2.2    Final Prediction    24
CHAPTER 4    EXPERIMENTAL RESULTS    26
4.1    Experimental Environment    26
4.2    Skin Lesion Datasets    27
4.2.1    ISIC-2018 dataset [8, 9]    27
4.2.2    ISIC-2019 dataset [8, 10, 11]    29
4.3    Evaluation Performance    31
4.4    Comparison and Analyses    35
4.4.1    Evaluation of the Proposed Method on ISIC-2018 [1, 8, 9] Datasets    35
4.4.1.1    Comparison of LOD Dataset Composition    35
4.4.1.2    Comparison of Multi-Model Ensemble    38
4.4.1.3    Comparison with other methods on the ISIC-2018 [1] Test Set    40
4.4.2    Comparing with other methods on the ISIC-2019 [2] Test Set    41
CHAPTER 5    CONCLUSIONS AND FUTURE WORKS    42
5.1    Conclusions    42
5.2    Future Works    43
REFERENCES    44


 
LIST OF FIGURES
Figure 2.1 The architecture of a GAN.    4
Figure 2.2 An flowchart of style transfer.    5
Figure 2.3 The illustration of the four-layer in Dense Block.    7
Figure 2.4 The architectures of the DenseNet-201 [12].    8
Figure 3.1 Flowchart of the data balancing system.    10
Figure 3.2 The illustration of image generation system.    12
Figure 3.3 Examples of generated images (Dermatofibroma) scale to 2242 pixels in resolution 162-2562.    13
Figure 3.4 The architecture of our mapping network.    14
Figure 3.5 The architecture of our style-base generator.    15
Figure 3.6 Examples of Vascular lesion (VL) generated images. (a) No noise in all layers.    16
Figure 3.7 Samples of Melanoma (MEL) data augmentation.    18
Figure 3.8 The architecture of the multi-model detection network ensemble.    20
Figure 3.9 The illustration of the softmax layer.    21
Figure 3.10 The flow chart of the multi-model ensemble (a) Traditional ensemble (b) The proposed method.    23
Figure 3.11 The example of the multi-model ensemble (a) Traditional ensemble (b) The proposed method.    25
Figure 4.1 The illustration of different categories in the ISIC-2018 [8, 9].    28
Figure 4.2 The example of Squamous Cell Carcinoma (SCC).    30
 
LIST OF TABLES
Table 2.1 The DenseNet-201 [12] architectures in detail.    9
Table 4.4.1 The specifications of the experiment platform.    26
Table 4.2 Numbers of images of different categories in the training set of ISIC-2018 [8, 9].    29
Table 4.3 Numbers of images of different categories in the training set of ISIC-2019 [8, 10, 11].    30
Table 4.4 Confusion matrix for the i-th category.    31
Table 4.5 Example of the imbalanced multi-categories confusion matrix.    33
Table 4.6 The compositions of LOD0 datasets.    36
Table 4.7 The performance of LOD0 datasets in DenseNet-201 [12].    37
Table 4.8 The performances on the ISIC-2018 [8, 9] validation set.    38
Table 4.9 The performances on the ISIC-2018 [1] test set.    39
Table 4.10 The mean recall on ISIC-2018 challenge website [1].    40
Table 4.11 The mean recall and parameters on ISIC-2019 challenge website [2]    41


                                

[1] "ISIC-2018 web." [Online]. https://challenge2018.isic-archive.com/ (accessed April, 2020).
[2] "ISIC-2019 web." [Online]. https://challenge2019.isic-archive.com/ (accessed April, 2020).
[3] K. Ramlakhan and Y. Shang, "A mobile automated skin lesion classification system," in IEEE 23rd International Conference on Tools with Artificial Intelligence, pp. 138-141, 2011.
[4] B. Ö. Cakir, P. Adamson, and C. Cingi, "Epidemiology and economic burden of nonmelanoma skin cancer," Facial plastic surgery clinics of North America, vol. 20, no. 4, pp. 419-422, 2012.
[5] C. P. Wild, B. W. Stewart, and C. Wild, "World cancer report 2014," World Health Organization Geneva, vol. 5, no. 14, 2014.
[6] L. E. Dubas and A. Ingraffea, "Nonmelanoma skin cancer," Facial Plastic Surgery Clinics, vol. 21, no. 1, pp. 43-53, 2013.
[7] T. Karras, S. Laine, and T. Aila, "A style-based generator architecture for generative adversarial networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4401-4410, 2019.
[8] P. Tschandl, C. Rosendahl, and H. Kittler, "The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions," Scientific Data, vol. 5, p. 180161, 2018.
[9] N. Codella, V. Rotemberg, P. Tschandl, M. E. Celebi, S. Dusza, D. Gutman, B. Helba, A. Kalloo, K. Liopyris, and M. Marchetti, "Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (ISIC)," arXiv preprint arXiv:1902.03368, 2019.
[10] N. C. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, and H. Kittler, "Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC)," in IEEE 15th International Symposium on Biomedical Imaging, pp. 168-172, 2018.
[11] M. Combalia, N. C. Codella, V. Rotemberg, B. Helba, V. Vilaplana, O. Reiter, A. C. Halpern, S. Puig, and J. Malvehy, "Bcn20000: Dermoscopic lesions in the wild," arXiv preprint arXiv:1908.02288, 2019.
[12] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700-4708, 2017.

[13] X. Huang and S. Belongie, "Arbitrary style transfer in real-time with adaptive instance normalization," in Proceedings of the IEEE International Conference on Computer Vision, pp. 1501-1510, 2017.
[14] S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in International Conference on Machine Learning, 2015.
[15] T. Karras, T. Aila, S. Laine, and J. Lehtinen, "Progressive growing of gans for improved quality, stability, and variation," International Conference on Learning Representations, vol. 1710.10196, 2018.
[16] A. Krizhevsky and G. Hinton, "Learning multiple layers of features from tiny images," M.S.thesis,Department of Computer Science, University of Toronto, 2009.
[17] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, "Reading digits in natural images with unsupervised feature learning," Workshop on Neural Information Processing Systems, 2011.
[18] D. Jia, D. Wei, S. Richard, L.-J. Li, K. Li, and F.-F. Li, "Imagenet: A large-scale hierarchical image database," in IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
[19] V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," in Proceedings of the 27th International Conference on Machine Learning, pp. 807-814, 2010.
[20] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
[21] M. Lin, Q. Chen, and S. Yan, "Network in network," in International Conference on Learning Representations, 2013.
[22] G. Lemaître, F. Nogueira, and C. K. Aridas, "Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning," The Journal of Machine Learning Research, vol. 18, no. 1, pp. 559-563, 2017.
[23] E. Ramentol, Y. Caballero, R. Bello, and F. Herrera, "SMOTE-RSB*: a hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using SMOTE and rough sets theory," Knowledge and Information Systems, vol. 33, no. 2, pp. 245-265, 2012.
[24] H. Han, W.-Y. Wang, and B.-H. Mao, "Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning," in International Conference on Intelligent Computing, pp. 878-887, 2005.
[25] H. Haibo and E. A. Garcia, "Learning from Imbalanced Data," IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9, pp. 1263-1284, 2009.
[26] G. King and L. Zeng, "Logistic regression in rare events data," Political Analysis, vol. 9, no. 2, pp. 137-163, 2001.
[27] M. Zhu, J. Xia, X. Jin, M. Yan, G. Cai, J. Yan, and G. Ning, "Class weights random forest algorithm for processing class imbalanced medical data," IEEE Access, vol. 6, pp. 4641-4652, 2018.
[28] Q. Dong, S. Gong, and X. Zhu, "Imbalanced Deep Learning by Minority Class Incremental Rectification," IEEE Trans Pattern Anal Mach Intell, vol. 41, no. 6, pp. 1367-1381, 2019.
[29] G. Mariani, F. Scheidegger, R. Istrate, C. Bekas, and C. Malossi, "Bagan: Data augmentation with balancing gan," arXiv preprint arXiv, vol. 1803.09655, 2018.
[30] C. Shorten and T. M. Khoshgoftaar, " A survey on image data augmentation for deep learning," Journal of Big Data, vol. 6, no. 60, 2019.
[31] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, "Gans trained by a two time-scale update rule converge to a local nash equilibrium," in Advances in Neural Information Processing Systems, pp. 6626-6637, 2017.
[32] Y. Wu, J. Donahue, D. Balduzzi, K. Simonyan, and T. Lillicrap, "LOGAN: Latent Optimisation for Generative Adversarial Networks," arXiv preprint arXiv, vol. 1912.00953, 2019.
[33] A. Brock, J. Donahue, and K. Simonyan, "Large scale gan training for high fidelity natural image synthesis," arXiv preprint arXiv, vol. 1809.11096, 2018.
[34] S. Laine, "Feature-based metrics for exploring the latent space of generative models," International Conference on Learning Representations, vol. 2018, 2018.
[35] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, "Autoaugment: Learning augmentation strategies from data," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 113-123, 2019.
[36] F. Nachbar, W. Stolz, T. Merkle, A. B. Cognetta, T. Vogt, M. Landthaler, P. Bilek, O. Braun-Falco, and G. Plewig, "The ABCD rule of dermatoscopy: high prospective value in the diagnosis of doubtful melanocytic skin lesions," Journal of the American Academy of Dermatology, vol. 30, no. 4, pp. 551-559, 1994.
[37] Y. Xiao, J. Wu, Z. Lin, and X. Zhao, "A deep learning-based multi-model ensemble method for cancer prediction," Computer Methods and Programs in Biomedicine, vol. 153, pp. 1-9, 2018.
[38] J. Li, W. Chen, Y. Sun, Y. Li, and Z. Peng, "Object Detection Based on DenseNet and RPN," in IEEE Chinese Control Conference, pp. 8410-8415, 2019.
[39] Z. Abai and N. Rajmalwar, "DenseNet Models for Tiny ImageNet Classification," arXiv preprint arXiv, vol. 1904.10429, 2019.
[40] D. Opitz and R. Maclin, "Popular ensemble methods: An empirical study," Journal of Artificial Intelligence Research, vol. 11, pp. 169-198, 1999.
[41] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv, vol. 1412.6980, 2014.

[42] "Tensorboard." [Online]. [https://www.tensorflow.org/tensorboard/] (accessed April, 2020).
[43] "TensorFlow " [Online]. [https://www.tensorflow.org/] (accessed April, 2020).
[44] "Keras Website." [Online]. [https://keras.io] (accessed April, 2020).
[45] K. M. Li and E. C. Li, "Skin lesion analysis towards melanoma detection via end-to-end deep learning of convolutional neural networks," arXiv preprint arXiv:1807.08332, 2018.
[46] A. Bissoto, F. Perez, V. Ribeiro, M. Fornaciali, S. Avila, and E. Valle, "Deep-learning ensembles for skin-lesion segmentation, analysis, classification: RECOD titans at ISIC challenge 2018," arXiv preprint arXiv, vol. 1808.08480, 2018.
[47] A. Aldwgeri and N. F. Abubacker, "Ensemble of Deep Convolutional Neural Network for Skin Lesion Classification in Dermoscopy Images," in International Visual Informatics Conference, pp. 214-226, 2019.
[48] Y. C. Lee, S.-H. Jung, and H.-H. Won, "WonDerM: Skin lesion classification with fine-tuned neural networks," arXiv preprint arXiv, vol. 1808.03426, 2018.
[49] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in International Conference on Learning Representations, pp. 1-14, 2015.
[50] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, "Inception-v4, inception-resnet and the impact of residual connections on learning," in Thirty-first AAAI Conference on Artificial Intelligence, 2017.
[51] Y. Pan and Y. Xia, "Residual Network based Aggregation Model for Skin Lesion Classification," arXiv preprint arXiv, vol. 1807.09150, 2018.
[52] M. Goyal and J. C. Rajapakse, "Deep neural network ensemble by data augmentation and bagging for skin lesion classification," arXiv preprint arXiv, vol. 1807.05496, 2018.
[53] A. G. Pacheco, A.-R. Ali, and T. Trappenberg, "Skin cancer detection based on deep learning and entropy to detect outlier samples," arXiv preprint arXiv, vol. 1909.04525, 2019.
[54] Y. Li and L. Shen, "Skin lesion analysis towards melanoma detection using deep learning network," Sensors, vol. 18, no. 2, p. 556, 2018.
[55] N. Gessert, M. Nielsen, M. Shaikh, R. Werner, and A. Schlaefer, "Skin lesion classification using ensembles of multi-resolution EfficientNets with meta data," Elsevier MethodsX, p. 100864, 2020.
[56] M. Tan and Q. V. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," arXiv preprint arXiv, vol. 1905.11946, 2019.

全文公開日期 2025/07/23 (校內網路)
全文公開日期本全文未授權公開 (校外網路)
全文公開日期本全文未授權公開 (國家圖書館：臺灣博碩士論文系統)

簡易檢索 / 詳目顯示

相關論文