新型冠狀病毒(COVID-19)疫情已持續延燒數年，及早分流輕、重症與降低高風險族群重症死亡率是重要關鍵，過往的文獻表明肺部超音波比起胸腔X光檢查更能精確地診斷出新冠肺炎，隔離病房內使用超音波檢查胸腔，除提供即時、非侵入性影像降低病毒傳染風險，更能協助醫療團隊有效掌握重症患者變化多端的病況。超音波影像的非侵入性與即時顯像之特性，能快速幫助醫事人員給予病患適當治療。此外，深度學習辨識醫療影像於近年高度關注及研究；因此本研究以深度學習神經網路，配合注意力機制模組，優化與可視化三分類COVID-19相關肺炎檢測任務。
主要以網路上的COVID-19肺部超音波資料庫，擷取共2,147張包含健康肺部、一般肺炎及COVID-19肺部超音波影像作為訓練驗證集。於神經網路模型建構部分，採用二維捲積層、批次統一層及ReLU激勵函數層為一捲積主結構，並配合平均池化層進行特徵採樣，共包含五層作為基底模型。而其中四種注意力機制被採用，空間注意力機制模組、空間通道注意力機制模組、非局部特徵注意力機制模組及本研究提出的局部全局注意力機制模組，並加入於不同結構層間做模型訓練及分析。
在模型訓練並分析後，以本研究提出之局部結合全局注意力機制模組於第4個捲積主結構層中配合基底模型，準確率能達到90.68%，精確度則為89.11% 、召回率為89.99% 及F1 score 為89.50%。相較於傳統CNN模型，本研究提出之模型亦表現出較好的分析指標表現與視覺化辨識。從實驗結果來看，提出之注意力機制配合基底模型之輕量化模型能夠有效地辨別三分類COVID-19相關肺炎超音波影像，以協助醫事人員能快速準確的做出診斷並給予治療。

The novel coronavirus (COVID-19) epidemic has been spreading for several years. Early diagnosis of mild and severe cases and reduction of severe mortality in high-risk groups are the key points. Past literature has shown that lung ultrasound is more accurate than lung X-ray examination. After diagnosing COVID-19, ultrasound examination of the chest cavity either provides real-time, non-invasive images to reduce the risk of virus infection or assists the medical team to effectively grasp the changing conditions of critically ill patients. The non-invasive and real-time characteristics of ultrasound systems can quickly help medical staff to give patients appropriate treatment. In addition, deep learning to identify medical images has been highly concerned and studied in recent years; therefore, this study uses deep learning neural networks and an attention module to optimize and visualize three-category COVID-19-related pneumonia detection tasks.
The COVID-19 lung ultrasound database on the Internet was downloaded to extract a total of 2,147 ultrasound images including healthy lungs, general pneumonia and COVID-19 lungs as a training and verification set. In the part of DNN architecture establishment, convolution layer, batch normalization and ReLU activation was used and combined as a main structure block, and came with average pooling layer for feature sampling. The base model comprised 5 main structure blocks. And also 4 different attention modules were introduced including spatial attention module, channel and spatial attention module, non-local attention module and local-global interaction attention module our proposed. These modules were integrated and analyzed between different structural layers during model training. After model training and analysis, using the local combined global attention module proposed in this study with the base model in the fourth convolutional main structure layer, the accuracy rate can reach 90.68%, the precision is 89.11%, and the recall rate It is 89.99% and F1 score is 89.50%. Compared with the traditional CNN model, the model proposed in this study also presented better performance and visual recognition. From the experimental results, the proposed attention module combined with the light weight model of the base model can effectively distinguish three categories of ultrasound images of COVID-19-related pneumonia, so as to assist medical staff to quickly and accurately diagnose and give treatment.

[1] 疾病介紹. 衛生福利部疾病管制署. (n.d.). https://www.cdc.gov.tw/Category/Page/vleOMKqwuEbIMgqaTeXG8A
[2] World Health Organization. (n.d.). Coronavirus. World Health Organization. https://www.who.int/health-topics/coronavirus#tab=tab_1
[3] Centers for Disease Control and Prevention. (n.d.). About covid-19. Centers for Disease Control and Prevention. https://www.cdc.gov/coronavirus/2019-ncov/your-health/about-covid-19.html
[4] Centers for Disease Control and Prevention. (n.d.-b). Symptoms of COVID-19. Centers for Disease Control and Prevention. https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html
[5] Mayo Foundation for Medical Education and Research. (2020, June 13). Pneumonia. Mayo Clinic.https://www.mayoclinic.org/diseases-conditions/pneumonia/symptoms-causes/syc-20354204
[6] World Health Organization. (n.d.-a). Coronavirus disease (covid-19): How is it transmitted?. World Health Organization. https://www.who.int/news-room/questions-and-answers/item/coronavirus-disease-covid-19-how-is-it-transmitted
[7] Lotfi, M., Hamblin, M. R., & Rezaei, N. (2020). Covid-19: Transmission, prevention, and potential therapeutic opportunities. Clinica Chimica Acta, 508, 254–266. https://doi.org/10.1016/j.cca.2020.05.044
[8] Yu, X., & Yang, R. (2020). Covid‐19 transmission through asymptomatic carriers is a challenge to containment. Influenza and Other Respiratory Viruses, 14(4), 474–475. https://doi.org/10.1111/irv.12743
[9] Islam, M. S., Rahman, K. M., Sun, Y., Qureshi, M. O., Abdi, I., Chughtai, A. A., & Seale, H. (2020). Current knowledge of COVID-19 and infection prevention and control strategies in healthcare settings: A global analysis. Infection Control & Hospital Epidemiology, 41(10), 1196–1206. https://doi.org/10.1017/ice.2020.237
[10] Filchakova, O., Dossym, D., Ilyas, A., Kuanysheva, T., Abdizhamil, A., & Bukasov, R. (2022). Review of covid-19 testing and diagnostic methods. Talanta, 244, 123409. https://doi.org/10.1016/j.talanta.2022.123409
[11] Singh, B., Datta, B., Ashish, A., & Dutta, G. (2021). A comprehensive review on current COVID-19 detection methods: From lab care to point of care diagnosis. Sensors International, 2, 100119. https://doi.org/10.1016/j.sintl.2021.100119
[12] 台灣醫檢雜誌 - 台灣醫檢會報第18卷第2期-社團法人台灣醫事檢驗學會. 台灣醫檢雜誌 - 台灣醫檢會報第18卷第2期-社團法人台灣醫事檢驗學會 Taiwan Society of Laboratory Medicine. (n.d.). https://www.labmed.org.tw/bjournal04.asp?IsAno=7&IsBno=16&IsCno=54&IsDno=143
[13] 毛玻璃樣病灶. 毛玻璃樣病灶-胸腔外科-三軍總醫院 - tri-service general hospital. (n.d.). https://wwwv.tsgh.ndmctsgh.edu.tw/unit/10041/14482
[14] Silverstone, L. (2023, February 25). Ground-glass opacification: Radiology reference article. Radiopaedia Blog RSS. https://radiopaedia.org/articles/ground-glass-opacification-3
[15] professional, C. C. medical. (n.d.). Covid-19 pneumonia: Symptoms, treatment & recovery. Cleveland Clinic. https://my.clevelandclinic.org/health/diseases/24002-covid-pneumonia
[16] Knox, D. B., Hirshberg, E. L., Orme, J., Peltan, I., & Lanspa, M. J. (2022). Effect of COVID 19 pneumonia on hyperglycemia: Is it different from non COVID pneumonia?. Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 16(2), 102407.
[17] Radiological Society of North America (RSNA) and American College of Radiology (ACR). (n.d.). General ultrasound. Radiologyinfo.org. https://www.radiologyinfo.org/en/info/genus
[18] U.S. Department of Health and Human Services. (n.d.). Ultrasound. National Institute of Biomedical Imaging and Bioengineering. https://www.nibib.nih.gov/science-education/science-topics/ultrasound#pid-931
[19] Gargani, L. (2011). Lung Ultrasound: A new tool for the cardiologist. Cardiovascular Ultrasound, 9(1). https://doi.org/10.1186/1476-7120-9-6
[20] Allinovi, M., Parise, A., Giacalone, M., Amerio, A., Delsante, M., Odone, A., ... & Mangia, A. (2020). Lung ultrasound may support diagnosis and monitoring of COVID-19 pneumonia. Ultrasound in medicine & biology, 46(11), 2908-2917.
[21] Lichtenstein, D. A. (2015). BLUE-protocol and FALLS-protocol: two applications of lung ultrasound in the critically ill. Chest, 147(6), 1659-1670.
[22] Khalili N, Haseli S, Iranpour P. Lung Ultrasound in COVID-19 Pneumonia: Prospects and Limitations. Acad Radiol. 2020 Jul;27(7):1044-1045. doi: 10.1016/j.acra.2020.04.032. Epub 2020 May 3. PMID: 32444253; PMCID: PMC7196391.
[23] Goldsmith, A. J., Al Saud, A., Duggan, N. M., Ma, I. W., Huang, C. K., Eke, O., ... & Liteplo, A. S. (2022). Point-of-Care Lung Ultrasound for Differentiating COVID-19 From Influenza. Cureus, 14(1).
[24] Volpicelli, G., Lamorte, A., & Villén, T. (2020). What’s new in lung ultrasound during the COVID-19 pandemic. Intensive care medicine, 46, 1445-1448.
[25] Albawi, S., Mohammed, T. A., & Al-Zawi, S. (2017). Understanding of a convolutional neural network. 2017 International Conference on Engineering and Technology (ICET). https://doi.org/10.1109/icengtechnol.2017.8308186
[26] Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., & Chen, T. (2018). Recent advances in Convolutional Neural Networks. Pattern Recognition, 77, 354–377. https://doi.org/10.1016/j.patcog.2017.10.013
[27] Shen, D., Wu, G., & Suk, H.-I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19(1), 221–248. https://doi.org/10.1146/annurev-bioeng-071516-044442
[28] Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017). A survey on Deep Learning in medical image analysis. Medical Image Analysis, 42, 60–88. https://doi.org/10.1016/j.media.2017.07.005
[29] Anaya-Isaza, A., Mera-Jiménez, L., & Zequera-Diaz, M. (2021). An overview of deep learning in medical imaging. Informatics in Medicine Unlocked, 26, 100723. https://doi.org/10.1016/j.imu.2021.100723
[30] Yu, H., Yang, L. T., Zhang, Q., Armstrong, D., & Deen, M. J. (2021). Convolutional Neural Networks for Medical Image Analysis: State-of-the-art, comparisons, improvement and perspectives. Neurocomputing, 444, 92–110. https://doi.org/10.1016/j.neucom.2020.04.157
[31] Niu, Z., Zhong, G., & Yu, H. (2021b). A review on the attention mechanism of Deep Learning. Neurocomputing, 452, 48–62. https://doi.org/10.1016/j.neucom.2021.03.091
[32] Bahdanau, D., Cho, K., & Bengio, Y. (2016, May 19). Neural machine translation by jointly learning to align and translate. arXiv.org. https://arxiv.org/abs/1409.0473
[33] Guo, MH., Xu, TX., Liu, JJ. et al. Attention mechanisms in computer vision: A survey. Comp. Visual Media 8, 331–368 (2022). https://doi.org/10.1007/s41095-022-0271-y
[34] Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19).
[35] Born, J., Wiedemann, N., Cossio, M., Buhre, C., Brändle, G., Leidermann, K., Goulet, J., Aujayeb, A., Moor, M., Rieck, B., & Borgwardt, K. (2021). Accelerating detection of lung pathologies with explainable ultrasound image analysis. Applied Sciences, 11(2), 672. https://doi.org/10.3390/app11020672
[36] Diaz-Escobar, J., Ordóñez-Guillén, N. E., Villarreal-Reyes, S., Galaviz-Mosqueda, A., Kober, V., Rivera-Rodriguez, R., & Lozano Rizk, J. E. (2021). Deep-learning based detection of covid-19 using lung ultrasound imagery. PLOS ONE, 16(8). https://doi.org/10.1371/journal.pone.0255886
[37] Awasthi, N., Dayal, A., Cenkeramaddi, L. R., & Yalavarthy, P. K. (2021). Mini-covidnet: Efficient Lightweight Deep Neural Network for ultrasound based point-of-care detection of covid-19. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 68(6), 2023–2037. https://doi.org/10.1109/tuffc.2021.3068190
[38] Rahhal, M. M., Bazi, Y., Jomaa, R. M., Zuair, M., & Melgani, F. (2022). Contrasting EfficientNet, VIT, and GMLP for COVID-19 detection in ultrasound imagery. Journal of Personalized Medicine, 12(10), 1707. https://doi.org/10.3390/jpm12101707
[39] La Salvia, M., Secco, G., Torti, E., Florimbi, G., Guido, L., Lago, P., Salinaro, F., Perlini, S., & Leporati, F. (2021). Deep learning and lung ultrasound for covid-19 pneumonia detection and severity classification. Computers in Biology and Medicine, 136, 104742. https://doi.org/10.1016/j.compbiomed.2021.104742
[40] Muhammad, G., & Shamim Hossain, M. (2021). Covid-19 and non-covid-19 classification using multi-layers fusion from lung ultrasound images. Information Fusion, 72, 80–88. https://doi.org/10.1016/j.inffus.2021.02.013
[41] Schlemper, J., Oktay, O., Schaap, M., Heinrich, M., Kainz, B., Glocker, B., & Rueckert, D. (2019). Attention gated networks: Learning to leverage salient regions in medical images. Medical image analysis, 53, 197-207.
[42] Du, J., Guan, K., Zhou, Y., Li, Y., & Wang, T. (2022). Parameter-Free Similarity-Aware Attention Module for Medical Image Classification and Segmentation. IEEE Transactions on Emerging Topics in Computational Intelligence.
[43] Joutard, S., Dorent, R., Isaac, A., Ourselin, S., Vercauteren, T., & Modat, M. (2019). Permutohedral attention module for efficient non-local neural networks. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI 22 (pp. 393-401). Springer International Publishing.
[44] Zhou, Q., Huang, Z., Ding, M., & Zhang, X. (2023). Medical image classification using light-weight CNN with spiking cortical model based attention module. IEEE Journal of Biomedical and Health Informatics, 27(4), 1991-2002.
[45] Huang, Q., Lei, Y., Xing, W., He, C., Wei, G., Miao, Z., ... & Chen, J. (2022). Evaluation of pulmonary edema using ultrasound imaging in patients with COVID-19 pneumonia based on a non-local Channel attention ResNet. Ultrasound in medicine & biology, 48(5), 945-953.
[46] Centers for Disease Control and Prevention. (n.d.). SARS-COV-2 variant classifications and definitions. Centers for Disease Control and Prevention. https://www.cdc.gov/coronavirus/2019-ncov/variants/variant-classifications.html
[47] Hadj Hassine, I. (2021). Covid‐19 vaccines and variants of concern: A Review. Reviews in Medical Virology, 32(4). https://doi.org/10.1002/rmv.2313
[48] World Health Organization. (n.d.). Tracking sars-COV-2 variants. World Health Organization. https://www.who.int/activities/tracking-SARS-CoV-2-variants
[49] Jannisborn. (n.d.). Jannisborn/covid19_ultrasound: Open source lung ultrasound (LUS) data collection initiative for covid-19. GitHub. https://github.com/jannisborn/covid19_ultrasound
[50] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998, doi: 10.1109/5.726791.
[51] Wang, X., Girshick, R., Gupta, A., & He, K. (2018). Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7794-7803).
[52] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
[53] Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360..
[54] Berrar, D. (2019). Cross-Validation.
[55] Narkhede, S. (2018). Understanding auc-roc curve. Towards Data Science, 26(1), 220-227.
[56] Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of machine learning research, 9(11).
[57] Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision (pp. 618-626).
[58] Gare, G. R., Tran, H. V., deBoisblanc, B. P., Rodriguez, R. L., & Galeotti, J. M. (2022). Weakly Supervised Contrastive Learning for Better Severity Scoring of Lung Ultrasound. arXiv preprint arXiv:2201.07357.
[59] Barros, B., Lacerda, P., Albuquerque, C., & Conci, A. (2021). Pulmonary COVID-19: learning spatiotemporal features combining CNN and LSTM networks for lung ultrasound video classification. Sensors, 21(16), 5486.
[60] Magrelli, S., Valentini, P., De Rose, C., Morello, R., & Buonsenso, D. (2021). Classification of lung disease in children by using lung ultrasound images and deep convolutional neural network. Frontiers in physiology, 12, 693448.