第二型人類表皮生長因子受體 (human epidermal growth factor receptor 2, HER2)的過度表現被視為轉移性乳癌的預後標記，並且能用來預測 HER2 標靶藥物的療效。雙色原位雜交法 (Dual in situ hybridization, DISH) 已經由美國食品藥物管理局(FDA) 批准為判斷 HER2 是否過度表現的方法。但此方法在應用上仍面臨幾項挑戰。重疊的細胞及模糊的細胞邊界使分割個別的 HER2 相關細胞變得困難，且HER2 相關細胞的外觀變異度很大。另外，手工的標註通常只針對具備高度信心的細胞作標註，使的一些沒有被標註的 HER2 相關細胞被誤認為背景，可能會降低模型的表現。為了解決以上問題，我們提出一個二階段弱監督式學習的框架來精準的衡量是否有 HER2 的過度表現。
為了證實該模型的有效性，該模型在兩種不同放大倍率的 DISH 數據集上作驗證。在第一組的數據集中，影像數據達到 accuracy 96.78% , precision97.77%,recall 84.86%, Dice Index 90.77% ; 在第二組的數據集中，影像數據達到 accuracy 96.43% , precision 97.82%, recall 87.14%, Dice Index 91.87% 。除此之外，該提出方法的表現優於其他 15 種比較方法，進一步證實該方法在兩組數據集上針對 HER2是否為過度表現能提供優良的結果，並在醫學上輔助 HER2 標靶治療。

Overexpression of human epidermal growth factor receptor 2 (HER2/ERBB2) is
identified as a prognostic marker in metastatic breast cancer and a predictor to determine the effects of HER2­targeted drugs. Accurate HER2 testing is essential in determining the optimal treatment for metastatic breast cancer patients. Brightfield dual in situ hybridization (DISH) was recently authorized by the United States Food and Drug Administration for the assessment of HER2 overexpression, which however is a challenging task due to a variety of reasons. Firstly, the presence of touching clustered and overlapping cells render it difficult for segmentation of individual HER2 related cells, which must contain both HER2 and CEN17 signals. Secondly, the fuzzy cell boundaries make the localization of each HER2 related cell challenging. Thirdly, variation in the appearance of HER2 related cells is large. Fourthly, as manual annotations are usually made on targets with high confidence, causing sparsely labeled data with some unlabeled HER2 related cells defined as background, this will seriously confuse fully supervised AI learning and cause poor model outcomes.
To deal with all issues mentioned above, we propose a two­stage weakly supervised
deep learning framework for accurate and robust assessment of HER2 overexpression.
The effectiveness and robustness of the proposed deep learning framework is evaluated
on two DISH datasets acquired at two different magnifications. The experimental results demonstrate that the proposed deep learning framework achieves an accuracy of
96.78±1.25, precision of 97.77±3.09, recall of 84.86±5.83 and Dice Index of 90.77±4.1
and an accuracy of 96.43±2.67, precision of 97.82±3.99, recall of 87.14±10.17 and Dice
Index of 91.87±6.51 for segmentation of HER2 overexpression on the two experimental
datasets, respectively. Furthermore, the proposed deep learning framework outperforms
15 state­ of­the­art benchmark methods by a significant margin (P < 0.001). Importantly,
the experimental results demonstrate that the proposed deep learning framework generates promising results for segmentation of HER2 overexpression on both datasets, making the proposed framework feasible to assist in guiding HER2 targeted therapies.

參 考 文 獻
[1] R. L. Siegel, K. D. Miller, H. E. Fuchs, A. Jemal, et al., “Cancer statistics, 2021,”
Ca Cancer J Clin, vol. 71, no. 1, pp. 7–33, 2021.
[2] A. C. Wolff, M. E. H. Hammond, J. N. Schwartz, K. L. Hagerty, D. C. Allred, R. J.
Cote, M. Dowsett, P. L. Fitzgibbons, W. M. Hanna, A. Langer, et al., “American society of clinical oncology/college of american pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer,” Archives
of pathology & laboratory medicine, vol. 131, no. 1, pp. 18–43, 2007.
[3] D. J. Slamon, B. Leyland­Jones, S. Shak, H. Fuchs, V. Paton, A. Bajamonde,
T. Fleming, W. Eiermann, J. Wolter, M. Pegram, et al., “Use of chemotherapy plus
a monoclonal antibody against her2 for metastatic breast cancer that overexpresses
her2,” New England journal of medicine, vol. 344, no. 11, pp. 783–792, 2001.
[4] I. Smith, M. Procter, R. D. Gelber, S. Guillaume, A. Feyereislova, M. Dowsett,
A. Goldhirsch, M. Untch, G. Mariani, J. Baselga, et al., “2­year follow­up of
trastuzumab after adjuvant chemotherapy in her2­positive breast cancer: a randomised controlled trial,” The lancet, vol. 369, no. 9555, pp. 29–36, 2007.
[5] G. V. P. Okaly, D. Panwar, K. B. Lingappa, P. Kumari, A. Anand, P. Kumar, M. H.
Chikkalingaiah, R. V. Kumar, et al., “Fish and her2/neu equivocal immunohistochemistry in breast carcinoma,” Indian journal of cancer, vol. 56, no. 2, p. 119,
2019.
[6] F.­M. Chen, L.­J. Huang, F. Ou­Yang, J.­Y. Kan, L.­C. Kao, and M.­F. Hou,
“Activation of mitochondrial unfolded protein response is associated with her2­
overexpression breast cancer,” Breast Cancer Research and Treatment, vol. 183,
no. 1, pp. 61–70, 2020.
[7] F. F. Gao, D. J. Dabbs, K. L. Cooper, and R. Bhargava, “Bright­field her2 dual in
situ hybridization (dish) assay vs fluorescence in situ hybridization (fish) focused
study of immunohistochemical 2+ cases,” American journal of clinical pathology,
vol. 141, no. 1, pp. 102–110, 2014.
[8] D. Zhang, J. Han, G. Cheng, and M.­H. Yang, “Weakly supervised object localization and detection: A survey,” IEEE transactions on pattern analysis and machine
intelligence, vol. 44, no. 9, pp. 5866–5885, 2021.
[9] H. Lee and W.­K. Jeong, “Scribble2label: Scribble­supervised cell segmentation
via self­generating pseudo­labels with consistency,” in International Conference
on Medical Image Computing and Computer­Assisted Intervention, pp. 14–23,
Springer, 2020.
[10] K. Nishimura, C. Wang, K. Watanabe, R. Bise, et al., “Weakly supervised cell instance segmentation under various conditions,” Medical Image Analysis, vol. 73,
p. 102182, 2021.
[11] S.­H. Lee, W. Erber, A. Porwit, M. Tomonaga, L. Peterson, and I. C. S. I. Hematology, “Icsh guidelines for the standardization of bone marrow specimens and reports,”
International journal of laboratory hematology, vol. 30, no. 5, pp. 349–364, 2008.
[12] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and
pattern recognition, pp. 3431–3440, 2015.
[13] H. Chen, X. Qi, L. Yu, and P.­A. Heng, “Dcan: deep contour­aware networks for
accurate gland segmentation,” in Proceedings of the IEEE conference on Computer
Vision and Pattern Recognition, pp. 2487–2496, 2016.
[14] Y.­J. Lin, T.­K. Chao, M.­A. Khalil, Y.­C. Lee, D.­Z. Hong, J.­J. Wu, and C.­W.
Wang, “Deep learning fast screening approach on cytological whole slides for thyroid
cancer diagnosis,” Cancers, vol. 13, no. 15, p. 3891, 2021.
[15] M.­A. Khalil, Y.­C. Lee, H.­C. Lien, Y.­M. Jeng, and C.­W. Wang, “Fast segmentation of metastatic foci in h&e whole­slide images for breast cancer diagnosis,”
Diagnostics, vol. 12, no. 4, p. 990, 2022.
[16] C.­W. Wang, Y.­A. Liou, Y.­J. Lin, C.­C. Chang, P.­H. Chu, Y.­C. Lee, C.­H. Wang,
and T.­K. Chao, “Artificial intelligence­assisted fast screening cervical high grade
squamous intraepithelial lesion and squamous cell carcinoma diagnosis and treatment planning,” Scientific Reports, vol. 11, no. 1, pp. 1–14, 2021.
[17] C.­W. Wang, Y.­C. Lee, C.­C. Chang, Y.­J. Lin, Y.­A. Liou, P.­C. Hsu, C.­C. Chang,
A.­K.­O. Sai, C.­H. Wang, and T.­K. Chao, “A weakly supervised deep learning
method for guiding ovarian cancer treatment and identifying an effective biomarker,”
Cancers, vol. 14, no. 7, p. 1651, 2022.
[18] C.­W. Wang, C.­C. Chang, Y.­C. Lee, Y.­J. Lin, S.­C. Lo, P.­C. Hsu, Y.­A. Liou,
C.­H. Wang, and T.­K. Chao, “Weakly supervised deep learning for prediction of
treatment effectiveness on ovarian cancer from histopathology images,” Computerized Medical Imaging and Graphics, vol. 99, p. 102093, 2022.
[19] O. Ronneberger, P. Fischer, and T. Brox, “U­net: Convolutional networks for
biomedical image segmentation,” in International Conference on Medical image
computing and computer­assisted intervention, pp. 234–241, Springer, 2015.
[20] T. Falk, D. Mai, R. Bensch, Ö. Çiçek, A. Abdulkadir, Y. Marrakchi, A. Böhm,
J. Deubner, Z. Jäckel, K. Seiwald, et al., “U­net: deep learning for cell counting,
detection, and morphometry,” Nature methods, vol. 16, no. 1, pp. 67–70, 2019.
[21] M. Zhang, X. Li, M. Xu, and Q. Li, “Automated semantic segmentation of red blood
cells for sickle cell disease,” IEEE Journal of Biomedical and Health Informatics,
vol. 24, no. 11, pp. 3095–3102, 2020.
[22] V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional
encoder­decoder architecture for image segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481–2495, 2017.
[23] L.­C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder­decoder with
atrous separable convolution for semantic image segmentation,” in Proceedings of
the European conference on computer vision (ECCV), pp. 801–818, 2018.
[24] C.­W. Wang, K.­Y. Lin, Y.­J. Lin, M.­A. Khalil, K.­L. Chu, and T.­K. Chao, “A soft
label deep learning to assist breast cancer target therapy and thyroid cancer diagnosis,” Cancers, vol. 14, no. 21, p. 5312, 2022.
[25] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception­v4, inception­resnet
and the impact of residual connections on learning,” in Thirty­first AAAI conference
on artificial intelligence, 2017.
[26] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
[27] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,”
in Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 770–778, 2016.
[28] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in
Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 1251–1258, 2017.
[29] N. Lessmann, B. Van Ginneken, P. A. De Jong, and I. Išgum, “Iterative fully convolutional neural networks for automatic vertebra segmentation and identification,”
Medical image analysis, vol. 53, pp. 142–155, 2019.
[30] S. Wang, K. He, D. Nie, S. Zhou, Y. Gao, and D. Shen, “Ct male pelvic organ segmentation using fully convolutional networks with boundary sensitive representation,” Medical image analysis, vol. 54, pp. 168–178, 2019.
[31] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r­cnn,” in Proceedings of the
IEEE international conference on computer vision, pp. 2961–2969, 2017.
[32] X. Wang, R. Zhang, T. Kong, L. Li, and C. Shen, “Solov2: Dynamic and fast instance segmentation,” Advances in Neural information processing systems, vol. 33,
pp. 17721–17732, 2020.
[33] L. Ke, Y.­W. Tai, and C.­K. Tang, “Deep occlusion­aware instance segmentation with
overlapping bilayers,” in Proceedings of the IEEE/CVF Conference on Computer
Vision and Pattern Recognition (CVPR), pp. 4019–4028, June 2021.
[34] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of
the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022,
2021.
[35] E. Upschulte, S. Harmeling, K. Amunts, and T. Dickscheid, “Contour proposal networks for biomedical instance segmentation,” Medical Image Analysis, p. 102371,
2022.
[36] H. Chen, X. Qi, L. Yu, Q. Dou, J. Qin, and P.­A. Heng, “Dcan: Deep contour­aware
networks for object instance segmentation from histology images,” Medical image
analysis, vol. 36, pp. 135–146, 2017.
[37] Z. Wu, N. Bodla, B. Singh, M. Najibi, R. Chellappa, and L. S. Davis, “Soft sampling
for robust object detection,” CoRR, vol. abs/1806.06986, 2018.
[38] I. Krasin, T. Duerig, N. Alldrin, V. Ferrari, S. Abu­El­Haija, A. Kuznetsova, H. Rom,
J. Uijlings, S. Popov, A. Veit, et al., “Openimages: A public dataset for large­scale
multi­label and multi­class image classification,” Dataset available from https://
github. com/openimages, vol. 2, no. 3, p. 18, 2017.
[39] C.­W. Wang, S.­C. Huang, M.­A. Khalil, D.­Z. Hong, S.­I. Meng, and Y.­C. Lee,
“CW­NET for multitype cell detection and classification in bone marrow examination and mitotic figure examination,” Bioinformatics, vol. 39, 05 2023. btad344.
[40] C.­W. Wang, M.­A. Khalil, Y.­J. Lin, Y.­C. Lee, and T.­K. Chao, “Detection of erbb2
and cen17 signals in fluorescent in situ hybridization and dual in situ hybridization
for guiding breast cancer her2 target therapy,” Artificial Intelligence in Medicine,
vol. 141, p. 102568, 2023.
[41] Z. Cai and N. Vasconcelos, “Cascade r­cnn: high quality object detection and instance segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 5, pp. 1483–1498, 2019.
[42] J. Deng, W. Dong, R. Socher, L.­J. Li, K. Li, and L. Fei­Fei, “Imagenet: A largescale hierarchical image database,” in 2009 IEEE conference on computer vision and
pattern recognition, pp. 248–255, IEEE, 2009.
[43] C.­W. Wang, S.­M. Ka, and A. Chen, “Robust image registration of biological microscopic images,” Scientific reports, vol. 4, no. 1, pp. 1–12, 2014.
[44] C.­W. Wang and H.­C. Chen, “Improved image alignment method in application to
x­ray images and biological images,” Bioinformatics, vol. 29, no. 15, pp. 1879–1887,
2013.
[45] A. C. Ruifrok, D. A. Johnston, et al., “Quantification of histochemical staining by color deconvolution,” Analytical and quantitative cytology and histology, vol. 23,
no. 4, pp. 291–299, 2001.
[46] P. K. Gajaria, S. Tambe, T. Pai, A. Patil, S. B. Desai, T. M. Shet, et al., “Dual­color
dual­hapten in situ hybridization (d­dish)–comparison with fluorescence in situ hybridization (fish) for her2/neu testing in breast cancer,” Indian Journal of Pathology
and Microbiology, vol. 63, no. 2, p. 194, 2020.
[47] E. J. A. Bowles, R. Wellman, H. S. Feigelson, A. A. Onitilo, A. N. Freedman, T. Delate, L. A. Allen, L. Nekhlyudov, K. A. Goddard, R. L. Davis, et al., “Risk of heart
failure in breast cancer patients after anthracycline and trastuzumab treatment: a retrospective cohort study,” Journal of the National Cancer Institute, vol. 104, no. 17,pp. 1293–1305, 2012.
[48] N. Mohan, J. Jiang, M. Dokmanovic, and W. J. Wu, “Trastuzumab­mediated cardiotoxicity: current understanding, challenges, and frontiers,” Antibody therapeutics, vol. 1, no. 1, pp. 13–17, 2018.
[49] S. Kunte, J. Abraham, and A. J. Montero, “Novel her2–targeted therapies for her2–
positive metastatic breast cancer,” Cancer, vol. 126, no. 19, pp. 4278–4288, 2020.
[50] M. F. Press, J. A. Seoane, C. Curtis, E. Quinaux, R. Guzman, G. Sauter, W. Eiermann,
J. R. Mackey, N. Robert, T. Pienkowski, et al., “Assessment of erbb2/her2 status in
her2­equivocal breast cancers by fish and 2013/2014 asco­cap guidelines,” JAMA
oncology, vol. 5, no. 3, pp. 366–375, 2019.
[51] F. Meric­Bernstam, A. M. Johnson, E. E. I. Dumbrava, K. Raghav, K. Balaji,
M. Bhatt, R. K. Murthy, J. Rodon, and S. A. Piha­Paul, “Advances in her2­targeted
therapy: novel agents and opportunities beyond breast and gastric cancer,” Clinical
Cancer Research, vol. 25, no. 7, pp. 2033–2041, 2019.