太空科技蓬勃發展，人類發射衛星用來進行地球觀測，而從衛星影像中提取研究所需的目標資料是一大重點。為減少衛星耗能並獲得有效的地表資訊，我們使用深度學習演算法提取雲特徵，以電腦視覺方法即時分割衛星影像中的雲，相較於過去的雲分割方法，我們所提的模型僅需分析衛星可見光影像這一種資料，且影像沒有地域及時間的限制，還能依照訓練的資料來擴充偵測的物件種類。
本研究提出一套雲分割系統的訓練流程；首先，提出一種以多個影像處理的步驟組成的半自動標註方法，標註出每張影像中每片雲的邊緣點座標及其他相關的資訊，作為實例分割的訓練資料；第二，將前述已標註好的雲資料，使用資料擴增的方法增加深度學習雲特徵的多樣性；最後，以具有多遮罩和多係數的端到端單階段模型進行衛星影像中的即時衛星雲分割。根據實驗結果，我們可以在0.00176至0.0745秒內為每張衛星雲圖以雲遮罩顯示出雲的位置。在本論文中，我們根據雲的特徵重新定義了遮罩重合度和混淆矩陣的計算方法，其中，以幀為單位的準確性為40.27%，而以像素為單位的準確性達97.95%；相較於其他代表性的實例分割方法，雖然我們的模型準確性較低一些，但是分割速度具有更多的優勢，依據實驗結果顯示，此模型可即時幫助衛星獲取有效的地表資訊。
關鍵字：衛星影像、雲分割、影像處理、半自動標註、深度學習、即時實例分割。

Space technology is booming. Humans launch satellites for earth observation. Extracting target data for research from satellite images is a major focus. In order to reduce satellite energy consumption and obtain effective surface information, we use deep learning algorithms to extract cloud features, and use computer vision methods to segment clouds in satellite images in real time. Compared with past cloud segmentation methods, our proposed model only needs to analyze the satellite visible image data, and the image has no geographical and time constraints, and can expand the types of detected objects based on the training data.
This research proposes a training process for a cloud segmentation system. First, a semi-automatic labeling method consisting of multiple image processing steps is proposed to label the contour point coordinates and other relevant information of each cloud in each image as the training data of instance segmentation; second, a data augmentation method is used to increase the diversity of deep learning cloud features; finally, an end-to-end single-stage model with multiple masks and coefficients is used to perform real-time cloud segmentation in satellite images. The experimental results show that the position of the cloud with cloud masks for each satellite image is obtained within 0.00176 to 0.0745 seconds. In this thesis, we redefine the intersection of union (IoU) and confusion matrix based on the characteristics of the cloud. According to this, the frame-based accuracy is 40.27%, while the pixel-based accuracy is 97.95%. Compared to other representative instance segmentation methods, although our model has lower accuracy, the segmentation speed has more advantages. In the light of experimental results, this model can help satellites obtain effective surface information in real time.
Keywords: Satellite Image, Cloud Segmentation, Image Processing, Semi-Automatic Annotation, Deep Learning, Real-time Instance Segmentation.

[1] W. W. Chen, H. K. Chang, and J. C. Liou, “Spatial and temporal distribution of cloud coverage over Taiwan using MTSAT satellite images,” Journal of Photogrammetry and Remote Sensing, vol. 18, no. 3, pp. 153-160, 2014.
[2] K. Y. Lee and C. H. Lin, “Cloud detection based on support vector machine for Landsat 8 imagery,” Journal of Photogrammetry and Remote Sensing, vol. 22, no. 4, pp. 227-241, 2017.
[3] Z. Zhu, S. Wang, and C. E.Woodcock, “Improvement and expansion of the Fmask algorithm: cloud, cloud shadow, and snow detection for Landsats 4–7, 8, and Sentinel 2 images,” Remote Sensing of Environment, vol. 159, no. 15, pp. 269-277, 2015.
[4] S. Ren et al., “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.
[5] K. He et al., “Mask R-CNN,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 386-397, 2020.
[6] L. Liu et al., “Deep learning for generic object detection: A survey,” International Journal of Computer Vision, vol. 128, no. 2, pp. 261-318, 2020.
[7] R. Girshick et al., “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, pp. 580-587, 2014.
[8] J. Redmon et al., “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, pp. 779-788, 2016.
[9] J. R. R. Uijlings et al., “Selective search for object recognition,” International Journal of Computer Vision, vol. 104, no. 2, pp. 154-171, 2013.
[10] R. Girshick, “Fast R-CNN,” in Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, pp. 1440-1448, 2015.
[11] K. He et al., “Spatial pyramid pooling in deep convolutional networks for visual recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1904-1916, 2015.
[12] C. Shorten and T. M. Khoshgoftaar, “A survey on Image Data Augmentation for Deep Learning,” Journal of Big Data, vol. 6, no. 60, pp. 1-48, 2019.
[13] D. Bolya et al., “YOLACT: Real-time instance segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), pp. 9156-9165, 2019.
[14] Y. Li et al., “Fully convolutional instance-aware semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, pp. 4438-4446, 2017.
[15] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, pp. 3431-3440, 2015.
[16] T. Y. Lin et al., “Focal loss for dense object detection,” in Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, pp. 2999-3007, 2017.
[17] T. Y. Lin et al., “Feature pyramid networks for object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, pp. 936-944, 2017.
[18] K. He et al., “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, pp. 770-778, 2016.
[19] J. H. Jeppesen et al., “A cloud detection algorithm for satellite imagery based on deep learning,” Remote Sensing of Environment, vol. 229, pp. 247-259, 2019.
[20] M. Wieland, Y. Li, and S. Martinis, “Multi-sensor cloud and cloud shadow segmentation with a convolutional neural network,” Remote Sensing of Environment, vol. 230, no. 111203, pp. 1-12, 2019.