The amount of overexpression of the human epidermal growth factor receptor 2 (HER2) is a predictive biomarker for metastatic breast cancer that may be used to evaluate therapeutic treatments. In HER2 immunohistochemical scores 2+ equivocal cases, accurate HER2 testing is critical for determining the most suitable precision treatment. HER2 in situ hybridization has generally been carried out using fluorescence in situ hybridization techniques (FISH). In recent years, brightfield dual in situ hybridization (DISH) has emerged as an efficient and viable alternative approach to replace fluorescent in situ hybridization (FISH) in various labs, including ours. The presented deep learning based framework allows effective and precise automated detection of ERBB2 to CEN17 signals ratio and the mean HER2 copies for each nucleus in DISH images for clinical usage, consequently avoid inter-oberserver variability and shortening the time required for routine manual assessment. To the best of author's knowledge, this is the first study to explore the use of deep learning technology to automatically detect HER2 overexpression in HER2 DISH images acquired from clinical breast cancer samples. On two datasets, we assess the efficiency of the proposed models. The results illustrate that the proposed method 1 accomplishes better performance than the baseline techniques with an accuracy of 97.11±2.39, precision of 96.92±1.45, recall of 92.49±2.04, F1-score of 94.65±3.59, and Jaccard Index of 88.43±10.27 on dataset 1 and an accuracy of 97.80±1.05, precision of 97.48±1.07, recall of 91.80±3.84, F1-score of 94.56±3.04, and Jaccard Index of 88.39±5.16 for dataset 2. Additionally, using Fisher's LSD, the proposed method 1 outperforms the baseline approaches by a significant margin (P<0.001). With a high degree of accuracy, precision, and sensitivity of over 91%, the proposed model 1 demonstrate that incorporating the deep learning based tools in DISH diagnosis for an accurate evaluation of HER2 overexpression in breast cancer patients, has significant potential to enable precision personalized medicine.

[1] A. Jemal, F. Bray, M. M. Center, J. Ferlay, E. Ward, and D. Forman, Global
cancer statistics, CA: a cancer journal for clinicians, vol. 61, no. 2, pp. 6990,
2011.
[2] A. C. Wol, M. E. H. Hammond, J. N. Schwartz, K. L. Hagerty, D. C. Allred,
R. J. Cote, M. Dowsett, P. L. Fitzgibbons, W. M. Hanna, A. Langer et al.,
American society of clinical oncology/college of american pathologists guideline
recommendations for human epidermal growth factor receptor 2 testing in
breast cancer, Archives of pathology & laboratory medicine, vol. 131, no. 1, pp.
1843, 2007.
[3] S. Kaptain, L. K. Tan, and B. Chen, Her-2/neu and breast cancer, Diagnostic
molecular pathology, vol. 10, no. 3, pp. 139152, 2001.
[4] D. J. Slamon, G. M. Clark, S. G. Wong, W. J. Levin, A. Ullrich, and W. L.
McGuire, Human breast cancer: correlation of relapse and survival with ampli
cation of the her-2/neu oncogene, science, vol. 235, no. 4785, pp. 177182,
1987.
[5] A. K. Tandon, G. M. Clark, G. C. Chamness, A. Ullrich, and W. McGuire, Her-
2/neu oncogene protein and prognosis in breast cancer. Journal of Clinical
Oncology, vol. 7, no. 8, pp. 11201128, 1989.
[6] M. M. Moasser, The oncogene her2: its signaling and transforming functions
and its role in human cancer pathogenesis, Oncogene, vol. 26, no. 45, pp.
64696487, 2007.
[7] Y. Yarden and M. X. Sliwkowski, Untangling the erbb signalling network,
Nature reviews Molecular cell biology, vol. 2, no. 2, pp. 127137, 2001.
[8] N. Benlimame, Q. He, S. Jie, D. Xiao, Y. J. Xu, M. Loignon, D. D. Schlaepfer,
and M. A. Alaoui-Jamali, Fak signaling is critical for erbb-2/erbb-3 receptor
cooperation for oncogenic transformation and invasion, The Journal of cell
biology, vol. 171, no. 3, pp. 505516, 2005.
[9] Z.-H. Liu, K. Wang, D.-Y. Lin, J. Xu, J. Chen, X.-Y. Long, Y. Ge, X.-L. Luo,
K.-P. Zhang, Y.-H. Liu et al., Impact of the updated 2018 asco/cap guidelines
on her2 sh testing in invasive breast cancer: a retrospective study of her2 sh
results of 2233 cases, Breast cancer research and treatment, vol. 175, no. 1, pp.
5157, 2019.
[10] D. J. Slamon, B. Leyland-Jones, S. Shak, H. Fuchs, V. Paton, A. Bajamonde,
T. Fleming, W. Eiermann, J. Wolter, M. Pegram et al., Use of chemotherapy
plus a monoclonal antibody against her2 for metastatic breast cancer that
overexpresses her2, New England journal of medicine, vol. 344, no. 11, pp.
783792, 2001.
[11] M. J. Piccart-Gebhart, M. Procter, B. Leyland-Jones, A. Goldhirsch, M. Untch,
I. Smith, L. Gianni, J. Baselga, R. Bell, C. Jackisch et al., Trastuzumab after
adjuvant chemotherapy in her2-positive breast cancer, New England Journal
of Medicine, vol. 353, no. 16, pp. 16591672, 2005.
[12] H. J. Burstein, L. N. Harris, P. K. Marcom, R. Lambert-Falls, K. Havlin,
B. Overmoyer, R. J. Friedlander Jr, J. Gargiulo, R. Strenger, C. L. Vogel
et al., Trastuzumab and vinorelbine as rst-line therapy for her2-overexpressing
metastatic breast cancer: multicenter phase ii trial with clinical outcomes, analysis
of serum tumor markers as predictive factors, and cardiac surveillance algorithm,
 Journal of clinical oncology, vol. 21, no. 15, pp. 28892895, 2003.
[13] P. P. Rosen, M. L. Lesser, C. D. Arroyo, M. Cranor, P. Borgen, and L. Norton,
Immunohistochemical detection of her2/neu in patients with axillary lymph
node negative breast carcinoma. a study of epidemiologic risk factors, histologic
features, and prognosis, Cancer, vol. 75, no. 6, pp. 13201326, 1995.
[14] N. Pathmanathan and A. M. Bilous, Her2 testing in breast cancer: an overview
of current techniques and recent developments, Pathology, vol. 44, no. 7, pp.
587595, 2012.
[15] R. R. Tubbs, D. G. Hicks, J. Cook, E. Downs-Kelly, J. Pettay, M. B. Hartke,
L. Hood, R. Neelon, J. Myles, G. T. Budd et al., Fluorescence in situ hy-
bridization (sh) as primary methodology for the assessment of her2 status
in adenocarcinoma of the breast: a single institution experience, Diagnostic
Molecular Pathology, vol. 16, no. 4, pp. 207210, 2007.
[16] A. C. Wol, M. E. H. Hammond, D. G. Hicks, M. Dowsett, L. M. McShane,
K. H. Allison, D. C. Allred, J. M. Bartlett, M. Bilous, P. Fitzgibbons et al.,
Recommendations for human epidermal growth factor receptor 2 testing in
breast cancer: American society of clinical oncology/college of american pathologists
clinical practice guideline update, Archives of Pathology and Laboratory
Medicine, vol. 138, no. 2, pp. 241256, 2014.
[17] A. C. Wol, M. E. H. Hammond, K. H. Allison, B. E. Harvey, L. M. McShane,
and M. Dowsett, Her2 testing in breast cancer: American society of clinical
oncology/college of american pathologists clinical practice guideline focused
update summary. Journal of oncology practice, vol. 14, no. 7, pp. 437441,
2018.
[18] J. Bartlett, F. M. Campbell, M. Ibrahim, A. OGrady, E. Kay, C. Faulkes,
N. Collins, J. Starczynski, J. M. Morgan, B. Jasani et al., A uk neqas ish
multicenter ring study using the ventana her2 dual-color ish assay, American
journal of clinical pathology, vol. 135, no. 1, pp. 157162, 2011.
[19] A. Brügmann, G. Lelkaitis, S. Nielsen, K. G. Jensen, and V. Jensen, Testing
her2 in breast cancer: a comparative study on brish, sh, and ihc, Applied
Immunohistochemistry & Molecular Morphology, vol. 19, no. 3, pp. 203211,
2011.
[20] Y. W. Koh, H. J. Lee, J. W. Lee, J. Kang, and G. Gong, Dual-color silverenhanced
in situ hybridization for assessing her2 gene amplication in breast
cancer, Modern Pathology, vol. 24, no. 6, pp. 794800, 2011.
[21] A. S. Manseld, W. R. Sukov, J. E. Eckel-Passow, Y. Sakai, F. J. Walsh,
M. Lonzo, A. E. Wiktor, A. Dogan, and R. B. Jenkins, Comparison of fuorescence
in situ hybridization (sh) and dual-ish (dish) in the determination of
her2 status in breast cancer, American journal of clinical pathology, vol. 139,
no. 2, pp. 144150, 2013.
[22] F. F. Gao, D. J. Dabbs, K. L. Cooper, and R. Bhargava, Bright-eld her2 dual
in situ hybridization (dish) assay vs uorescence in situ hybridization (sh)
focused study of immunohistochemical 2+ cases, American journal of clinical
pathology, vol. 141, no. 1, pp. 102110, 2014.
[23] J. Jacquemier, F. Spyratos, B. Esterni, M.-J. Mozziconacci, M. Antoine,
L. Arnould, S. Lizard, P. Bertheau, J. Lehmann-Che, C. B. Fournier et al.,
Sish/cish or qpcr as alternative techniques to sh for determination of her2
amplication status on breast tumors core needle biopsies: a multicenter experience
based on 840 cases, BMC cancer, vol. 13, no. 1, pp. 111, 2013.
[24] S.-J. Lim, A. Cantillep, and P. M. Carpenter, Validation and workow optimization
of human epidermal growth factor receptor 2 testing using inform
her2 dual-color in situ hybridization, Human pathology, vol. 44, no. 11, pp.
25902596, 2013.
[25] R. Horii, M. Matsuura, T. Iwase, Y. Ito, and F. Akiyama, Comparison of
dual-color in-situ hybridization and uorescence in-situ hybridization in her2
gene amplication in breast cancer, Breast Cancer, vol. 21, no. 5, pp. 598604,
2014.
[26] M. Tang, I. S. Soong, M. Luk, D. T. Suen, F. Hoie, E. P. Man, O. K. Tsun, and
U. Khoo, Comparison of uorescence in-situ hybridisation with dual-colour insitu
hybridisation for assessment of her2 gene amplication of breast cancer in
hong kong, Hong Kong Medical Journal, 2016.
[27] L. J. Layeld, M. L. Wallander, S. R. Tripp, S. Redpath, and P. M. Banks,
Comparison of dual-ish (dish) with uorescence in situ hybridization (sh) and
correlation with immunohistochemical ndings for her2/neu status in breast
carcinoma, Applied immunohistochemistry & molecular morphology, vol. 25,
no. 4, pp. 231236, 2017.
[28] R. Nishimura, N. Okamoto, M. Satou, K. Kojima, S. Tanaka, and N. Yamashita,
Bright-eld her2 dual in situ hybridization (dish) assay on breast cancer cell
blocks: a comparative study with histological sections, Breast Cancer, vol. 23,
no. 6, pp. 917921, 2016.
[29] M. Edelweiss, A. P. M. Sebastiao, H. Oen, M. Kracun, R. Serrette, and D. S.
Ross, Her2 assessment by bright-eld dual in situ hybridization in cell blocks
of recurrent and metastatic breast carcinoma, Cancer cytopathology, vol. 127,
no. 11, pp. 684690, 2019.
[30] B. Z. Clark and R. Bhargava, Bright-eld microscopy for her2 gene assessment:
not just dish-ful thinking? 2013.
[31] S. Kurozumi, M. Padilla, M. Kurosumi, H. Matsumoto, K. Inoue, J. Horiguchi,
I. Takeyoshi, T. Oyama, J. Ranger-Moore, D. C. Allred et al., Her2 intratumoral
heterogeneity analyses by concurrent her2 gene and protein assessment
for the prognosis of her2 negative invasive breast cancer patients, Breast cancer
research and treatment, vol. 158, no. 1, pp. 99111, 2016.
[32] K.-H. Yu, A. L. Beam, and I. S. Kohane, Articial intelligence in healthcare,
Nature biomedical engineering, vol. 2, no. 10, pp. 719731, 2018.
[33] H. Höfener, A. Homeyer, M. Förster, N. Drieschner, H.-U. Schildhaus, and
H. K. Hahn, Automated density-based counting of sh amplication signals
for her2 status assessment, Computer methods and programs in biomedicine,
vol. 173, pp. 7785, 2019.
[34] Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, nature, vol. 521, no.
7553, pp. 436444, 2015.
[35] F. Zakrzewski, W. de Back, M. Weigert, T. Wenke, S. Zeugner, R. Mantey,
C. Sperling, K. Friedrich, I. Roeder, D. Aust et al., Automated detection of
the her2 gene amplication status in uorescence in situ hybridization images
for the diagnostics of cancer tissues, Scientic reports, vol. 9, no. 1, pp. 112,
2019.
[36] K. Bera, K. A. Schalper, D. L. Rimm, V. Velcheti, and A. Madabhushi, Arti
cial intelligence in digital pathologynew tools for diagnosis and precision
oncology, Nature reviews Clinical oncology, vol. 16, no. 11, pp. 703715, 2019.
[37] Y.-J. Lin, T.-K. Chao, M.-A. Khalil, Y.-C. Lee, D.-Z. Hong, J.-J. Wu, and C.-
W. Wang, Deep learning fast screening approach on cytological whole slides
for thyroid cancer diagnosis, Cancers, vol. 13, no. 15, p. 3891, 2021.
[38] C.-W. Wang, Y.-A. Liou, Y.-J. Lin, C.-C. Chang, P.-H. Chu, Y.-C. Lee, C.-H.
Wang, and T.-K. Chao, Articial intelligence-assisted fast screening cervical
high grade squamous intraepithelial lesion and squamous cell carcinoma diagnosis
and treatment planning, Scientic Reports, vol. 11, no. 1, pp. 114,
2021.
[39] C.-W. Wang, S.-C. Huang, Y.-C. Lee, Y.-J. Shen, S.-I. Meng, and J. L. Gaol,
Deep learning for bone marrow cell detection and classication on whole-slide
images, Medical Image Analysis, vol. 75, p. 102270, 2022.
[40] V. Badrinarayanan, A. Kendall, and R. Cipolla, Segnet: A deep convolutional
encoder-decoder architecture for image segmentation, IEEE transactions on
pattern analysis and machine intelligence, vol. 39, no. 12, pp. 24812495, 2017.
[41] Z. Cai and N. Vasconcelos, Cascade r-cnn: High quality object detection and
instance segmentation, IEEE Transactions on Pattern Analysis and Machine
Intelligence, pp. 11, 2019.
[42] T. Falk, D. Mai, R. Bensch, Ö. Çiçek, A. Abdulkadir, Y. Marrakchi, A. Böhm,
J. Deubner, Z. Jäckel, K. Seiwald et al., U-net: deep learning for cell counting,
detection, and morphometry, Nature methods, vol. 16, no. 1, pp. 6770, 2019.
[43] F. Jubayer, J. A. Soeb, A. N. Mojumder, M. K. Paul, P. Barua, S. Kayshar,
S. S. Akter, M. Rahman, and A. Islam, Detection of mold on the food surface
using yolov5, Current Research in Food Science, vol. 4, pp. 724728, 2021.
[44] E. Shelhamer, J. Long, and T. Darrell, Fully convolutional networks for semantic
segmentation, IEEE transactions on pattern analysis and machine in-
telligence, vol. 39, no. 4, pp. 640651, 2017.
[45] Z. Wu, N. Bodla, B. Singh, M. Najibi, R. Chellappa, and L. S. Davis, Soft
sampling for robust object detection, BMVC, p. 225, 2019.
[46] I. Krasin, T. Duerig, N. Alldrin, V. Ferrari, S. Abu-El-Haija, A. Kuznetsova,
H. Rom, J. Uijlings, S. Popov, A. Veit, S. Belongie, V. Gomes, A. Gupta,
C. Sun, G. Chechik, D. Cai, Z. Feng, D. Narayanan, and K. Murphy, Openimages:
A public dataset for large-scale multi-label and multi-class image classi
cation. Dataset available from https://github.com/openimages, 2017.
[47] D. Tellez, G. Litjens, P. Bándi, W. Bulten, J.-M. Bokhorst, F. Ciompi, and
J. van der Laak, Quantifying the eects of data augmentation and stain color
normalization in convolutional neural networks for computational pathology,
Medical image analysis, vol. 58, p. 101544, 2019.
[48] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning. MIT
press Cambridge, 2016, vol. 1, no. 2.
[49] D. Komura and S. Ishikawa, Machine learning methods for histopathological
image analysis, Computational and structural biotechnology journal, vol. 16,
pp. 3442, 2018.
[50] M. Veta, Y. J. Heng, N. Stathonikos, B. E. Bejnordi, F. Beca, T. Wollmann,
K. Rohr, M. A. Shah, D. Wang, M. Rousson et al., Predicting breast tumor
proliferation from whole-slide images: the tupac16 challenge, Medical image
analysis, vol. 54, pp. 111121, 2019.
[51] K. Sirinukunwattana, J. P. Pluim, H. Chen, X. Qi, P.-A. Heng, Y. B. Guo, L. Y.
Wang, B. J. Matuszewski, E. Bruni, U. Sanchez et al., Gland segmentation in
colon histology images: The glas challenge contest, Medical image analysis,
vol. 35, pp. 489502, 2017.
[52] J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei, Imagenet: A
large-scale hierarchical image database, pp. 248255, 2009.
[53] C. Szegedy, S. Ioe, V. Vanhoucke, and A. A. Alemi, Inception-v4, inceptionresnet
and the impact of residual connections on learning, in Thirty-rst AAAI
conference on articial intelligence, 2017.
[54] A. Sohail, A. Khan, H. Nisar, S. Tabassum, and A. Zameer, Mitotic nuclei
analysis in breast cancer histopathology images using deep ensemble classier,
Medical Image Analysis, p. 102121, 2021.
[55] K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition,
 in Proceedings of the IEEE conference on computer vision and pattern
recognition, 2016, pp. 770778.
[56] P. K. Gajaria, S. Tambe, T. Pai, A. Patil, S. B. Desai, T. M. Shet et al., Dualcolor
dual-hapten in situ hybridization (d-dish)comparison with uorescence in
situ hybridization (sh) for her2/neu testing in breast cancer, Indian Journal
of Pathology and Microbiology, vol. 63, no. 2, p. 194, 2020.
[57] S. Kunte, J. Abraham, and A. J. Montero, Novel her2targeted therapies for
her2positive metastatic breast cancer, Cancer, vol. 126, no. 19, pp. 4278
4288, 2020.
[58] M. F. Press, J. A. Seoane, C. Curtis, E. Quinaux, R. Guzman, G. Sauter,
W. Eiermann, J. R. Mackey, N. Robert, T. Pienkowski et al., Assessment of
erbb2/her2 status in her2-equivocal breast cancers by sh and 2013/2014 ascocap
guidelines, JAMA oncology, vol. 5, no. 3, pp. 366375, 2019.
[59] E. J. A. Bowles, R. Wellman, H. S. Feigelson, A. A. Onitilo, A. N. Freedman,
T. Delate, L. A. Allen, L. Nekhlyudov, K. A. Goddard, R. L. Davis et al., Risk
of heart failure in breast cancer patients after anthracycline and trastuzumab
treatment: a retrospective cohort study, Journal of the National Cancer Insti-
tute, vol. 104, no. 17, pp. 12931305, 2012.
[60] N. Mohan, J. Jiang, M. Dokmanovic, and W. J. Wu, Trastuzumab-mediated
cardiotoxicity: current understanding, challenges, and frontiers, Antibody ther-
apeutics, vol. 1, no. 1, pp. 1317, 2018.
[61] F. Meric-Bernstam, A. M. Johnson, E. E. I. Dumbrava, K. Raghav, K. Balaji,
M. Bhatt, R. K. Murthy, J. Rodon, and S. A. Piha-Paul, Advances in her2-
targeted therapy: novel agents and opportunities beyond breast and gastric
cancer, Clinical Cancer Research, vol. 25, no. 7, pp. 20332041, 2019.
[62] G. V. P. Okaly, D. Panwar, K. B. Lingappa, P. Kumari, A. Anand, P. Kumar,
M. H. Chikkalingaiah, R. V. Kumar et al., Fish and her2/neu equivocal
immunohistochemistry in breast carcinoma, Indian journal of cancer, vol. 56,
no. 2, p. 119, 2019.
[63] F.-M. Chen, L.-J. Huang, F. Ou-Yang, J.-Y. Kan, L.-C. Kao, and M.-F. Hou,
Activation of mitochondrial unfolded protein response is associated with her2-
overexpression breast cancer, Breast Cancer Research and Treatment, vol. 183,
no. 1, pp. 6170, 2020.
[64] B. E. Bejnordi, M. Veta, P. J. Van Diest, B. Van Ginneken, N. Karssemeijer,
G. Litjens, J. A. Van Der Laak, M. Hermsen, Q. F. Manson, M. Balkenhol et al.,
Diagnostic assessment of deep learning algorithms for detection of lymph node
metastases in women with breast cancer, Jama, vol. 318, no. 22, pp. 21992210,
2017.
[65] S. Sornapudi, R. J. Stanley, W. V. Stoecker, H. Almubarak, R. Long, S. Antani,
G. Thoma, R. Zuna, and S. R. Frazier, Deep learning nuclei detection
in digitized histology images by superpixels, Journal of pathology informatics,
vol. 9, 2018.
37