Endometrial cancer (EC) diagnosis traditionally relies on tumor morphology and nuclear grade, but personalized therapy requires a deeper understanding of tumor mutational burden (TMB), i.e. an established predictive biomarker for immune checkpoint inhibition and has been shown to be a predictive biomarker for response to immunotherapy. However, traditional methods for predicting TMB require sequencing comprehensive panels, exomes, or whole genomes, which are high-cost and not commonly available in healthcare settings. Addressing this challenge, we present three multilayer attention multiple instance deep learning models designed to the classification of aggressive and non-aggressive EC and predict TMB status in the non-aggressive and aggressive ECs using commonly acquired H&E-stained whole slide images. Our models were evaluated on a large dataset of gigapixel histopathology images from The Cancer Genome Atlas (TCGA). The proposed method 1 achieved outstanding performance in classification of the aggressive and non-aggressive ECs, with 97%, 93%, 89% and 89% for area under the receiver operating characteristic curve (AUROC), sensitivity, mean of sensitivity and specificity (MSS) and accuracy, respectively. For TMB prediction in the aggressive ECs, the proposed method 3 achieves 85% and 80% for the sensitivity and AUROC, respectively, and the proposed method 2 obtains 73% and 78% for the MSS and AUROC, respectively.
In predicting TMB in the non-aggressive ECs, the proposed method 3 also demonstrated the best performance in comparison to the seven SOTA approaches, achieving 76% of sensitivity and 70% of AUROC. Fisher's exact test further validated that the association between the proposed model prediction and the actual cancer subtype or TMB status is both strong (p < 0.01).
Furthermore, according to the Fisher's exact test, the association between the proposed model prediction and the actual cancer subtype or TMB status in the aggressive group is both extremely strong (p < 0.001), and the association between the prediction of the proposed model 3 and the actual TMB status in the non-aggressive group is strong (p<0.01). Notably, our methods consistently outperformed seven state-of-the-art benchmarked approaches in computational pathology.
Importantly, according to the Kaplan-Meier survival analysis, the results show that the proposed method 1 successfully differentiates patients with longer disease-specific survival (DSS) and overall survival (OS) with significant difference (p < 0.01 for DSS, p < 0.05 for OS) between the TMB predicted classes in the aggressive EC. These compelling findings highlight the potential of the proposed methods to guide personalized treatment decisions by accurately predicting the EC cancer subtype and the TMB status for effective immunotherapy planning for EC patients. Moreover, a run time analysis demonstrates that the proposed method achieves high efficiency in inference time, taking only 26.21 seconds per slide on average, which makes the proposed methods feasible for practical clinical usage. Additionally, we explored the impact of incorporating a stable factor as a model covariate, resulting in an average performance improvement of 7% and 9% in the MSS and AUROC, respectively. This finding provides valuable insight for researchers aiming to optimize the performance of deep learning-based frameworks in future research.

[1] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
[2] O. Raglan, I. Kalliala, G. Markozannes, S. Cividini, M. J. Gunter, J. Nautiyal, H. Gabra, E. Paraskevaidis, P. Martin-Hirsch, K. K. Tsilidis, et al., “Risk factors for endometrial cancer: An umbrella review of the literature,” International journal of cancer, vol. 145, no. 7, pp. 1719–1730, 2019.
[3] A. Santoro, G. Angelico, A. Travaglino, F. Inzani, D. Arciuolo, M. Valente, N. D’Alessandris, G. Scaglione, V. Fiorentino, A. Raffone, et al., “New pathological and clinical insights in endometrial cancer in view of the updated esgo/estro/esp guidelines,” Cancers, vol. 13, no. 11, p. 2623, 2021.
[4] J. S. Berek, X. Matias-Guiu, C. Creutzberg, C. Fotopoulou, D. Gaffney, S. Kehoe, K. Lindemann, D. Mutch, N. Concin, F. W. C. C. Endometrial Cancer Staging Subcommittee, et al., “Figo staging of endometrial cancer: 2023,” International Journal of Gynecology & Obstetrics, 2023.
[5] S. F. Lax, E. S. Pizer, B. M. Ronnett, and R. J. Kurman, “Comparison of estrogen and progesterone receptor, ki-67, and p53 immunoreactivity in uterine endometrioid carcinoma and endometrioid carcinoma with squamous, mucinous, secretory, and ciliated cell differentiation,” Human pathology, vol. 29, no. 9, pp. 924–931, 1998.
[6] J. V. Bokhman, “Two pathogenetic types of endometrial carcinoma,” Gynecologic oncology, vol. 15, no. 1, pp. 10–17, 1983.
[7] M. A. Voss, R. Ganesan, L. Ludeman, K. McCarthy, R. Gornall, G. Schaller, W. Wei, and S. Sundar, “Should grade 3 endometrioid endometrial carcinoma be considered a type 2 cancer—a clinical and pathological evaluation,” Gynecologic oncology, vol. 124, no. 1, pp. 15–20, 2012.
[8] T. de Bortoli, M. Benary, P. Horak, M. Lamping, S. Stintzing, I. Tinhofer, S. Leyvraz, R. Sch ̈afer, F. Klauschen, U. Keller, et al., “Tumour mutational burden and survival with molecularly matched therapy,” European Journal of Cancer, vol. 190, p. 112925, 2023.
[9] D. T. Rieke, T. de Bortoli, M. Benary, P. Horak, M. Lamping, S. Stintzing, I. Tinhofer, S. Leyvraz, R. Sch ̈afer, F. Klauschen, et al., “Tumor mutational burden as a predictive biomarker for molecularly matched therapy in two independent pan-cancer cohorts.,” 2023.
[10] W. Cao, X. Ma, J. V. Fischer, C. Sun, B. Kong, and Q. Zhang, “Immunotherapy in endometrial cancer: rationale, practice and perspectives,” Biomarker Research, vol. 9, no. 1, pp. 1–30, 2021.
[11] S. Lee, O. Lara, H. Karpel, and B. Pothuri, “The association of tumor mutational burden, microsatellite stability, and mismatch repair deficiency in an endometrial cancer patient cohort (194),” Gynecologic Oncology, vol. 166, p. S111, 2022.
[12] X. Guo, X. Liang, Y. Wang, A. Cheng, H. Zhang, C. Qin, and Z. Wang, “Significance of tumor mutation burden combined with immune infiltrates in the progression and prognosis of advanced gastric cancer,” Frontiers in Genetics, vol. 12, p. 642608, 2021.
[13] R. T. Lawlor, P. Mattiolo, A. Mafficini, S.-M. Hong, M. L. Piredda, S. V. Taormina, G. Malleo, G. Marchegiani, A. Pea, R. Salvia, et al., “Tumor mutational burden as a potential biomarker for immunotherapy in pancreatic cancer: systematic review and still-open questions,” Cancers, vol. 13, no. 13, p. 3119, 2021.
[14] L. Fancello, S. Gandini, P. G. Pelicci, and L. Mazzarella, “Tumor mutational burden quantification from targeted gene panels: major advancements and challenges,” Journal for immunotherapy of cancer, vol. 7, pp. 1–13, 2019.
[15] B. Mel ́endez, C. Van Campenhout, S. Rorive, M. Remmelink, I. Salmon, and N. D’Haene, “Methods of measurement for tumor mutational burden in tumor tissue,” Translational lung cancer research, vol. 7, no. 6, p. 661, 2018.
[16] R. B ̈uttner, J. W. Longshore, F. L ́opez-R ́ıos, S. Merkelbach-Bruse, N. Normanno, E. Rouleau, and F. Penault-Llorca, “Implementing tmb measurement in clinical practice: considerations on assay requirements,” ESMO open, vol. 4, no. 1, p. e000442, 2019.
[17] Z. R. Chalmers, C. F. Connelly, D. Fabrizio, L. Gay, S. M. Ali, R. Ennis, A. Schrock, B. Campbell, A. Shlien, J. Chmielecki, et al., “Analysis of 100,000 human cancer genomes reveals the landscape of tumor mutational burden,” Genome medicine, vol. 9, pp. 1–14, 2017.
[18] A. Sadhwani, H.-W. Chang, A. Behrooz, T. Brown, I. Auvigne-Flament, H. Patel, R. Findlater, V. Velez, F. Tan, K. Tekiela, et al., “Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images,” Scientific reports, vol. 11, no. 1, p. 16605, 2021.
[19] Y. Niu, L. Wang, X. Zhang, Y. Han, C. Yang, H. Bai, K. Huang, C. Ren, G. Tian, S. Yin, et al., “Predicting tumor mutational burden from lung adenocarcinoma histopathological images using deep learning,” Frontiers in Oncology, vol. 12, p. 927426, 2022.
[20] K. Huang, B. Lin, J. Liu, Y. Liu, J. Li, G. Tian, and J. Yang, “Predicting colorectal cancer tumor mutational burden from histopathological images and clinical information using multi-modal deep learning,” Bioinformatics, vol. 38, no. 22, pp. 5108–5115, 2022.
[21] H. Xu, J. R. Clemenceau, S. Park, J. Choi, S. H. Lee, and T. H. Hwang, “Spatial heterogeneity and organization of tumor mutation burden with immune infiltrates within tumors based on whole slide images correlated with patient survival in bladder cancer,” Journal of Pathology Informatics, vol. 13, p. 100105, 2022.
[22] M. Y. Lu, T. Y. Chen, D. F. Williamson, M. Zhao, M. Shady, J. Lipkova, and F. Mahmood, “Ai-based pathology predicts origins for cancers of unknown primary,” Nature, vol. 594, no. 7861, pp. 106–110, 2021.
[23] A. V. Konstantinov and L. V. Utkin, “Multi-attention multiple instance learning,” Neural Computing and Applications, vol. 34, no. 16, pp. 14029–14051, 2022.
[24] K. Tomczak, P. Czerwi ́nska, and M. Wiznerowicz, “Review the cancer genome atlas (tcga): an immeasurable source of knowledge,” Contemporary Oncology/Wsp ́olczesna Onkologia, vol. 2015, no. 1, pp. 68–77, 2015.
[25] N. Coudray, P. S. Ocampo, T. Sakellaropoulos, N. Narula, M. Snuderl, D. Feny ̈o, A. L. Moreira, N. Razavian, and A. Tsirigos, “Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning,” Nature medicine, vol. 24, no. 10, pp. 1559–1567, 2018.
[26] G. Campanella, M. G. Hanna, L. Geneslaw, A. Miraflor, V. Werneck Krauss Silva, K. J. Busam, E. Brogi, V. E. Reuter, D. S. Klimstra, and T. J. Fuchs, “Clinical-grade computational pathology using weakly supervised deep learning on whole slide images,” Nature medicine, vol. 25, no. 8, pp. 1301–1309, 2019.
[27] M. Y. Lu, D. F. Williamson, T. Y. Chen, R. J. Chen, M. Barbieri, and F. Mahmood, “Data-efficient and weakly supervised computational pathology on whole-slide images,” Nature biomedical engineering, vol. 5, no. 6, pp. 555–570, 2021.
[28] C.-W. Wang, Y.-C. Lee, Y.-J. Lin, N. P. Firdi, H. Muzakky, T.-C. Liu, P.-J.Lai, C.-H. Wang, Y.-C. Wang, M.-H. Yu, et al., “Deep learning can predict bevacizumab therapeutic effect and microsatellite instability directly from histology in epithelial ovarian cancer,” Laboratory Investigation, vol. 103, no. 11, p. 100247, 2023.
[29] Y. Zheng, J. Li, J. Shi, F. Xie, J. Huai, M. Cao, and Z. Jiang, “Kernel attention transformer for histopathology whole slide image analysis and assistant cancer diagnosis,” IEEE Transactions on Medical Imaging, 2023.
[30] Z. Shao, H. Bian, Y. Chen, Y. Wang, J. Zhang, X. Ji, et al., “Transmil: Transformer based correlated multiple instance learning for whole slide image classification,” Advances in neural information processing systems, vol. 34, pp. 2136–2147, 2021.
[31] H. Xiang, J. Shen, Q. Yan, M. Xu, X. Shi, and X. Zhu, “Multi-scale representation attention based deep multiple instance learning for gigapixel whole slide image analysis,” Medical Image Analysis, vol. 89, p. 102890, 2023.
[32] D. N. Louis, M. Feldman, A. B. Carter, A. S. Dighe, J. D. Pfeifer, L. Bry, J. S. Almeida, J. Saltz, J. Braun, J. E. Tomaszewski, et al., “Computational pathology: a path ahead,” Archives of pathology & laboratory medicine, vol. 140, no. 1, pp. 41–50, 2016.
[33] C.-L. Chen, C.-C. Chen, W.-H. Yu, S.-H. Chen, Y.-C. Chang, T.-I. Hsu, M. Hsiao, C.-Y. Yeh, and C.-Y. Chen, “An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning,” Nature communications, vol. 12, no. 1, p. 1193, 2021.
[34] J. Wang and F. Biljecki, “Unsupervised machine learning in urban studies: A systematic review of applications,” Cities, vol. 129, p. 103925, 2022.
[35] V. Rani, S. T. Nabi, M. Kumar, A. Mittal, and K. Kumar, “Self-supervised learning: A succinct review,” Archives of Computational Methods in Engineering, vol. 30, no. 4, pp. 2761–2775, 2023.
[36] C.-W. Wang, Y.-C. Lee, Y.-J. Lin, C.-C. Chang, C.-H. Wang, T.-K. Chao, et al., “Interpretable attention-based deep learning ensemble for personalized ovarian cancer treatment without manual annotations,” Computerized Medical Imaging and Graphics, vol. 107, p. 102233, 2023.
[37] M. Tu, J. Huang, X. He, and B. Zhou, “Multiple instance learning with graph neural networks,” arXiv preprint arXiv:1906.04881, 2019.
[38] Y. Xiong, Z. Zeng, R. Chakraborty, M. Tan, G. Fung, Y. Li, and V. Singh, “Nystr ̈omformer: A nystr ̈om-based algorithm for approximating self-attention,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 14138–14148, 2021.
[39] J. Shreffler and M. R. Huecker, “Diagnostic testing accuracy: Sensitivity, specificity, predictive values and likelihood ratios,” 2020.
[40] D. M. Naeger, M. P. Kohi, E. M. Webb, A. Phelps, K. G. Ordovas, and T. B. Newman, “Correctly using sensitivity, specificity, and predictive values in clinical practice: how to avoid three common pitfalls,” American journal of roentgenology, vol. 200, no. 6, pp. W566–W570, 2013.
[41] A. G. Glaros and R. B. Kline, “Understanding the accuracy of tests with cutting scores: The sensitivity, specificity, and predictive value model,” Journal of clinical psychology, vol. 44, no. 6, pp. 1013–1023, 1988.
[42] E. Bolin and W. Lam, “A review of sensitivity, specificity, and likelihood ratios: evaluating the utility of the electrocardiogram as a screening tool in hypertrophic cardiomyopathy,” Congenital Heart Disease, vol. 8, no. 5, pp. 406–410, 2013.
[43] J. Davis and M. Goadrich, “The relationship between precision-recall and roc curves,” in Proceedings of the 23rd international conference on Machine learning, pp. 233–240, 2006.
[44] S. Inc, “Spss for windows, rel. 15.0. 1,” 2006.
[45] H.-Y. Kim, “Statistical notes for clinical researchers: Chi-squared test and fisher’s exact test,” Restorative dentistry & endodontics, vol. 42, no. 2, pp. 152–155, 2017.
[46] A. Nowacki, “Chi-square and fisher’s exact tests (from the” biostatistics and epidemiology lecture series, part 1”).,” Cleveland Clinic journal of medicine, vol. 84, no. 9 Suppl 2, pp. e20–e25, 2017.
[47] B. Damato and A. Taktak, “Survival after treatment of intraocular melanoma,”in Outcome Prediction in Cancer, pp. 27–41, Elsevier, 2007.
[48] M. Kang, H. Song, S. Park, D. Yoo, and S. Pereira, “Benchmarking self-supervised learning on diverse pathology datasets,” CVPR Open Access, 2023.
[49] H.-Y. Zhou, Y. Yu, C. Wang, S. Zhang, Y. Gao, J. Pan, J. Shao, G. Lu, K. Zhang, and W. Li, “A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics,” Nature Biomedical Engineering, pp. 1–13, 2023.
[50] F. Shamshad, S. Khan, S. W. Zamir, M. H. Khan, M. Hayat, F. S. Khan, and H. Fu, “Transformers in medical imaging: A survey,” Medical Image Analysis, p. 102802, 2023.
[51] Y. Zhang, J. Wang, J. M. Gorriz, and S. Wang, “Deep learning and vision transformer for medical image analysis,” 2023.
[52] X. Wang, Y. Du, S. Yang, J. Zhang, M. Wang, J. Zhang, W. Yang, J. Huang, and X. Han, “Retccl: clustering-guided contrastive learning for whole-slide image retrieval,” Medical image analysis, vol. 83, p. 102645, 2023.
[53] M. Caron, K. Misra, H. Touati, J. Steiner, H. Gurari, M. Uberti, and J. Raison, “Emerging properties in self-supervised vision transformers,” arXiv preprint arXiv:2104.14294, 2021.
[54] E. Cengil and A. C ̧ ınar, “The effect of deep feature concatenation in the classification problem: an approach on covid-19 disease detection,” International journal of imaging systems and technology, vol. 32, no. 1, pp. 26–40, 2022.
[55] Y.-J. Kang, S. O’Haire, F. Franchini, M. IJzerman, J. Zalcberg, F. Macrae, K. Canfell, and J. Steinberg, “A scoping review and meta-analysis on the prevalence of pan-tumour biomarkers (dmmr, msi, high tmb) in different solid tumours,” Scientific reports, vol. 12, no. 1, p. 20495, 2022.
[56] K. Choucair, S. Morand, L. Stanbery, G. Edelman, L. Dworkin, and J. Nemunaitis, “Tmb: a promising immune-response biomarker, and potential spearhead in advancing targeted therapy trials,” Cancer gene therapy, vol. 27, no. 12, pp. 841–853, 2020.
[57] C. S. Perone and J. Cohen-Adad, “Promises and limitations of deep learning for medical image segmentation,” Journal of Medical Artificial Intelligence, vol. 2, 2019.
[58] C. R. Boland, S. N. Thibodeau, S. R. Hamilton, D. Sidransky, J. R. Eshleman, R. W. Burt, S. J. Meltzer, M. A. Rodriguez-Bigas, R. Fodde, G. N. Ranzani, et al., “A national cancer institute workshop on microsatellite instability for cancer detection and familial predisposition: development of international criteria for the determination of microsatellite instability in colorectal cancer,” Cancer research, vol. 58, no. 22, pp. 5248–5257, 1998.