簡易檢索 / 詳目顯示

研究生: 陳柏洋
Bo-Yang Chen
論文名稱: 應用無源域自適應方法於醫學影像語義分割
Source-free Domain Adaptation for Medical Image Semantic Segmentation
指導教授: 方文賢
Wen-Hsien Fang
口試委員: 方文賢
Wen-Hsien Fang
陳郁堂
Yie-Tarng Chen
賴坤財
Kuen-Tsair Lay
阮聖彰
Shanq-Jang Ruan
丘建青
Chien-Ching Chiu
學位類別: 碩士
Master
系所名稱: 電資學院 - 電子工程系
Department of Electronic and Computer Engineering
論文出版年: 2023
畢業學年度: 111
語文別: 英文
論文頁數: 46
中文關鍵詞: 語意分割醫療影像風格傳換
外文關鍵詞: Semantic Segmentation, Medical Image, Style Transfer
相關次數: 點閱:139下載:2
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

深度學習在影像語義分割任務上取得了顯著的進展,提高了預測的準確性和效率。
然而,當處理未見過的資料集或具有不同特徵的領域時,其表現可能會受到影響。
在這篇論文中,我們利用傅立葉風格轉換(FST)技術,藉由目標領域中的圖片創建目標風格的源圖片。
為了確保在FST中進行精確的風格轉換,我們使用結構相似性指數(SSIM)生成可靠的風格轉換圖片。
隨後我們根據前一步驟中生成的目標風格源圖片來預測類別比率。此外,我們使用凸包演算法來生成偽標籤。
為了增強分割模型的穩健性,我們引入一個新的損失函數,結合了交叉熵、KL散度和多樣性最大化。
這個損失函數用於計算偽標籤和類別比率與模型預測結果之間的損失。
我們在兩個醫學數據集上驗證了我們的方法,實驗結果證明了它與該領域中其他方法相比的有效性和競爭性。


Deep learning has made remarkable progress in image semantic segmentation, significantly improving the accuracy and efficiency of the task.
However, its performance may be compromised when dealing with unseen datasets or domains with different characteristics.
In this thesis, we adopt Fourier Style Transfer (FST) to create target-style source images from the target domain.
To ensure precise style transformation in FST, we utilize the Structural Similarity Index Measure (SSIM), generating reliable style-transferred images.
We then predict the class-ratio according to the target-style source images from the previous step.
Moreover, we apply the conventional convex hull algorithm to generate connected pseudo-labels.
To enhance the segmentation model's robustness, we introduce a new loss function that combines cross-entropy, KL-divergence, and diversity maximization.
This loss function calculates the loss of the pseudo-labels and the class-ratio with the model's predicted results.
We evaluate our approach on two medical datasets, and the experimental results demonstrate its effectiveness and competitiveness compared to other methods in the field.

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1 Adversarial Learning . . . . . . . . . . . . . . . . . . 5 2.2 Self-Training . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Style-Transfer . . . . . . . . . . . . . . . . . . . . . . 7 2.4 Weakly Supervision . . . . . . . . . . . . . . . . . . . 9 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . 10 3 Proposed Method . . . . . . . . . . . . . . . . . . . . . . . 11 3.1 Proposed Architecture . . . . . . . . . . . . . . . . . 11 3.2 Fourier Style Transfer with SSIM Matching . . . . . . 12 3.3 Pseudo-label Generation by Convex Hull Algorithm . 15 3.4 Loss Functions . . . . . . . . . . . . . . . . . . . . . 16 3.4.1 Self-Entropy Loss . . . . . . . . . . . . . . . . 17 3.4.2 Class-Ratio Loss . . . . . . . . . . . . . . . . 18 3.4.3 Diversity Maximization Loss . . . . . . . . . . 19 3.4.4 Pseudo-Label Loss . . . . . . . . . . . . . . . 20 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . 21 4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.1.1 Multi-Site Prostate Segmentation Dataset . . 22 4.1.2 Multi-Modality Whole Heart Segmentation Dataset 23 4.2 Experimental Settings . . . . . . . . . . . . . . . . . 24 4.3 Evaluation Metrics . . . . . . . . . . . . . . . . . . . 25 4.4 Ablation Studies . . . . . . . . . . . . . . . . . . . . 27 4.4.1 Analysis of Diversity Maximization Loss and Convex Hull Pseudo-label . . . . . . . . . . . 29 4.5 Visualization Results . . . . . . . . . . . . . . . . . . 30 4.6 Comparison with the State-of-the-Art Works . . . . . 34 4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . 37 5 Conclusion and Future Works . . . . . . . . . . . . . . . . 38 5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 38 5.2 Future Works . . . . . . . . . . . . . . . . . . . . . . 38 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

[1] O. Ronneberger and P. Fischer, “U-net: Convolutional
networks for biomedical image segmentation,” in Proceed-
ings of Medical Image Computing and Computer-Assisted
Intervention–MICCAI 2015, pp. 234–241, 2015.
[2] L.-C. Chen, G. Papandreou, et al., “Deeplab: Semantic image
segmentation with deep convolutional nets, atrous convolution,
and fully connected crfs,” IEEE Transactions on Pattern Anal-
ysis and Machine Intelligence, vol. 40, no. 4, pp. 834–848, 2017.
[3] Y. Ganin, V. Lempitsky, et al., “Unsupervised domain adapta-
tion by backpropagation,” in Proceedings of International Con-
ference on Machine Learning, pp. 1180–1189, 2015.
[4] Y.-H. Tsai, W.-C. Hung, S. Schulter, et al., “Learning to adapt
structured output space for semantic segmentation,” in Pro-
ceedings of the IEEE Conference on Computer Vision and Pat-
tern Recognition, pp. 7472–7481, 2018.
[5] Y. Luo, L. Zheng, T. Guan, et al., “Taking a closer look at do-
main shift: Category-level adversaries for semantics consistent
domain adaptation,” in Proceedings of the IEEE/CVF Confer-
ence on Computer Vision and Pattern Recognition, pp. 2507–
2516, 2019.
[6] W.-L. Chang, H.-P. Wang, et al., “All about structure: Adapt-
ing structural information across domains for boosting seman-
tic segmentation,” in Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition, pp. 1900–1909,
2019.
[7] C. Bian, C. Yuan, J. Wang, M. Li, et al., “Uncertainty-
aware domain alignment for anatomical structure segmenta-
tion,” Medical Image Analysis, vol. 64, p. 101732, 2020.
[8] Y. Zou, Z. Yu, et al., “Unsupervised domain adaptation
for semantic segmentation via class-balanced self-training,” in
Proceedings of the European Conference on Computer Vision
(ECCV), pp. 289–305, 2018.
[9] Y. Zhang, P. David, H. Foroosh, and B. Gong, “A curricu-
lum domain adaptation approach to the semantic segmentation
of urban scenes,” IEEE Transactions on Pattern Analysis and
Machine Intelligence, vol. 42, no. 8, pp. 1823–1841, 2019.
[10] M. Bateson, J. Dolz, H. Kervadec, et al., “Constrained do-
main adaptation for image segmentation,” IEEE Transactions
on Medical Imaging, vol. 40, no. 7, pp. 1875–1887, 2021.
[11] Y. Li, L. Yuan, N. Vasconcelos, et al., “Bidirectional learning
for domain adaptation of semantic segmentation,” in Proceed-
ings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition, pp. 6936–6945, 2019.
[12] Y. Huo, Z. Xu, H. Moon, S. Bao, et al., “Synseg-net: Synthetic
segmentation without target modality ground truth,” IEEE
Transactions on Medical Imaging, vol. 38, no. 4, pp. 1016–1025,
2018.
[13] C. Qin, L. Wang, Y. Zhang, et al., “Generatively inferential co-
training for unsupervised domain adaptation,” in Proceedings of
the IEEE/CVF International Conference on Computer Vision
Workshops, pp. 0–0, 2019.
[14] R. Gong, W. Li, Y. Chen, et al., “Dlow: Domain flow for adap-
tation and generalization,” in Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition
(CVPR), pp. 2477–2486, 2019.
[15] Q. Zhou, Z. Feng, Q. Gu, et al., “Uncertainty-aware consistency
regularization for cross-domain semantic segmentation,” Com-
puter Vision and Image Understanding, vol. 221, p. 103448,
2022.
[16] M. Toldo, A. Maracani, et al., “Unsupervised domain adapta-
tion in semantic segmentation: a review,” Technologies, vol. 8,
no. 2, p. 35, 2020.
[17] N. Karani, E. Erdil, K. Chaitanya, et al., “Test-time adapt-
able neural networks for robust medical image segmentation,”
Medical Image Analysis, vol. 68, p. 101907, 2021.
[18] D. Wang, E. Shelhamer, S. Liu, et al., “Tent: Fully test-
time adaptation by entropy minimization,” arXiv preprint
arXiv:2006.10726, 2020.
[19] M. Bateson, H. Kervadec, et al., “Source-relaxed domain adap-
tation for image segmentation,” in Proceedings of the Interna-
tional Conference on Medical Image Computing and Computer-
Assisted Intervention, pp. 490–499, 2020.
[20] M. Bateson, H. Kervadec, et al., “Source-free domain adapta-
tion for image segmentation,” Medical Image Analysis, vol. 82,
p. 102617, 2022.
[21] Y. Yang and S. Soatto, “Fda: Fourier domain adaptation for
semantic segmentation,” in Proceedings of the IEEE/CVF Con-
ference on Computer Vision and Pattern Recognition, pp. 4085–
4095, 2020.
[22] Z. Wang, A. C. Bovik, et al., “Image quality assessment: from
error visibility to structural similarity,” IEEE Transactions on
Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
[23] “Scikit-image: The convex hull of a binary im-
age.” https://scikit-image.org/docs/stable/
auto_examples/edges/plot_convex_hull.html#
sphx-glr-download-auto-examples-edges-plot-convex-hull-py.
Accessed: 2023-07-30.
[24] X. Wu, S. Zhang, Q. Zhou, et al., “Entropy minimization versus
diversity maximization for domain adaptation,” IEEE Trans-
actions on Neural Networks and Learning Systems, 2021.
[25] J.-Y. Zhu, T. Park, et al., “Unpaired image-to-image transla-
tion using cycle-consistent adversarial networks,” in Proceed-
ings of the IEEE/CVF International Conference on Computer
Vision, pp. 2223–2232, 2017.
[26] X. Ouyang, Z. Xue, Y. Zhan, et al., “Weakly supervised seg-
mentation framework with uncertainty: A study on pneumoth-
orax segmentation in chest x-ray,” in Medical Image Computing
and Computer Assisted Intervention–MICCAI 2019, pp. 613–
621, Springer, 2019.
[27] H. Kervadec, J. Dolz, S. Wang, et al., “Bounding boxes for
weakly supervised segmentation: Global constraints get close
to full supervision,” in Medical imaging with deep learning,
pp. 365–381, PMLR, 2020.
[28] S. Mannor, D. Peleg, and R. Rubinstein, “The cross entropy
method for classification,” in Proceedings of the 22nd interna-
tional conference on Machine learning, pp. 561–568, 2005.
[29] E. Tzeng, J. Hoffman, T. Darrell, et al., “Simultaneous deep
transfer across domains and tasks,” in Proceedings of the IEEE
International Conference on Computer Vision, pp. 4068–4076,
2015.
[30] F. Maria Carlucci, L. Porzi, B. Caputo, et al., “Autodial: Au-
tomatic domain alignment layers,” in Proceedings of the IEEE
International Conference on Computer Vision, pp. 5067–5075,
2017.
[31] K. Saito, Y. Ushiku, et al., “Asymmetric tri-training for unsu-
pervised domain adaptation,” in Proceedings of International
Conference on Machine Learning, pp. 2988–2997, 2017.
[32] T. M. Cover and J. A. Thomas, “Information theory and the
stock market,” Elements of Information Theory. Wiley Inc.,
New York, pp. 543–556, 1991.
[33] Q. Liu, Q. Dou, P.-A. Heng, et al., “Shape-aware meta-learning
for generalizing prostate mri segmentation to unseen domains,”
in Proceedings of International Conference on Medical Image
Computing and Computer Assisted Intervention, 2020.
[34] X. Zhuang, L. Li, et al., “Evaluation of algorithms for multi-
modality whole heart segmentation: an open-access grand chal-
lenge,” Medical Image Analysis, vol. 58, p. 101537, 2019.
[35] Q. Dou, C. Ouyang, et al., “Pnp-adanet: Plug-and-
play adversarial domain adaptation network with a bench-
mark at cross-modality cardiac segmentation,” arXiv preprint
arXiv:1812.07907, 2018.
[36] J. Deng, W. Dong, R. Socher, et al., “Imagenet: A large-
scale hierarchical image database,” in Proceedings of 2009
IEEE Conference on Computer Vision and Pattern Recogni-
tion, pp. 248–255, 2009.
[37] D. P. Kingma, J. Ba, et al., “Adam: A method for stochastic
optimization,” arXiv preprint arXiv:1412.6980, 2014.
[38] K. H. Zou, S. K. Warfield, A. Bharatha, et al., “Statistical
validation of image segmentation quality based on a spatial
overlap index1: scientific reports,” Academic radiology, vol. 11,
no. 2, pp. 178–189, 2004.

QR CODE