每年的夏季與秋季，是臺灣的午後對流好發時期，由於提前防範的需求，我們需要一套可靠的午後對流預報系統。過去中央氣象局是使用需要龐大計算的物理模型進行預報，導致耗費大量時間；儘管現在中央氣象局使用模糊邏輯的方式，改善了運算時間的問題，但是有著單一輸入的限制，而且資料處理上會耗費大量人力。因此，本篇論文的目的，就是希望透過自動化的流程來處理標記資料，且能透過多種的異質資料來預測午後對流，進而為防災提前做好準備。
現有同樣使用深度學習進行午後對流預測的方法，通常只使用地面觀測站單一種資料類型，本篇論文嘗試加入天氣圖、衛星雲圖、探空氣球等資料，目的是希望午後對流預測模型能從多元的資料中提取更多的特徵，使預測更為準確。要做到這一點，我們分別設計影像特徵提取器與數值特徵提取器來分別處理影像資料與數值資料，理念是將原始資料產生更多的特徵圖，以利於接續的模型續練。而午後對流預測模型，我們使用LRCN來實作，不同於常見的CNN，它是結合了CNN與LSTM的特點，能夠讓輸入資料也能夠以時序方式輸入。
實驗結果方面，我們使用混淆矩陣中的真陽性、偽陽性、真陰性、偽陰性，來計算精確度、召回率、準確度、F1-score，以此呈現量化結果。我們方法的預測結果不管在弱事件還是強事件上，都有將近90%的準確度，在精確度與F1-score也都超過模糊邏輯法結果的兩倍之高，分別達到43% 與 35%；針對實驗結果，我們還列舉了一好一壞的午後對流預測，進而分析異質資料對模型的影響。
在消融實驗上，我們更驗證了越多異質資料，對模型的學習是越有幫助的。在臺灣北部的實驗上，最好的異質資料組合是天氣圖、地面測站資料、衛星雲圖、探空氣球資料；在臺灣中部與臺灣南部的實驗上，結果不大理想，推究其原因可能有兩個，其一是測站分布相較北部來說，並不是那麼密集，其二是地理環境相較北部而言較為平坦，導致午後對流成因，可能由鋒面與西南氣流影響為深。

Every summer and autumn are good times for afternoon convection in Taiwan. Due to the need for early prevention, we want a reliable afternoon convection forecasting system. In the past, the Central Weather Bureau (CWB) used a physical model that requires huge calculations to make forecasts, resulting in a lot of time-consuming. Although the CWB now uses fuzzy logic to improve the problem of computing time, it has the limitation of a single input data, and data processing will consume a lot of manpower. Therefore, the purpose of this thesis is to process the labeled data via an automated process and to predict afternoon convection through a variety of heterogeneous data, so as to prepare for disaster prevention in advance.
Existing methods that also use deep learning for afternoon convection prediction usually only use a single type of data from ground observation stations. This thesis attempts to add data such as weather maps, satellite cloud images, and sounding balloon data. The purpose is to hope that the afternoon convection prediction model can extract more features from the multivariate data to make the prediction more accurate. To accomplish this, we respectively design an image feature extractor and a numerical feature extractor to process image data and numerical data. The idea is to generate more feature maps from the original data to facilitate subsequent model training. As for the afternoon convection prediction model, we use LRCN to implement it, which is different from the common CNN. The LRCN combines the characteristics of CNN and LSTM so that the input data can also be input in time series.
In terms of experimental results, we use the true positives, false positives, true negatives, and false negatives in the confusion matrix to calculate precision, recall, accuracy, and F1-score to present quantitative results. The prediction results of our method have nearly 90% accuracy in both weak events and strong events, and the precision and F1-score are more than twice as high as those of the fuzzy logic method, reaching 43% and 35%, respectively. In response to the experimental results, we also list good and bad afternoon convection.
In the ablation experiment, we verify that the more heterogeneous data, the more helpful the model learning. In the experiment in northern Taiwan, the best combination of heterogeneous data is weather maps, ground station data, satellite cloud images, and sounding balloon data. In the experiments in central Taiwan and southern Taiwan, the results are not satisfactory. There may be two reasons for this. One is that the distribution of stations is not as dense as that in the north, and the other is that the geographical environment is relatively flat compared to the north. As a result, the cause of convection in the afternoon may be deeply influenced by the front and southwest airflow.

[1] M. Köhler et al., “Cb-LIKE - Cumulonimbus Likelihood: Thunderstorm forecasting with fuzzy logic,” Meteorologische Zeitschrift, vol. 25, pp. 1-19, 2016.
[2] D.R. Nayak, A. Mahapatra, and P. Mishra, “A survey on rainfall prediction using artificial neural network,” International Journal of Computer Applications, vol. 72, no. 16, pp. 32-40, 2013.
[3] C. Cortes and V. Vapnik, “Support vector networks,” Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
[4] S. Haykin, “Neural networks: A comprehensive foundation,” The Knowledge Engineering Review, vol. 13, no. 4, pp. 409-412, 1999.
[5] Y. Lecun et al., “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[6] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” Apr 2015. [online]. Available: https://arxiv.org/abs/1409.1556.
[7] H. Chang et al., “Nowcast guidance of afternoon convection initiation for Taiwan,” Weather and Forecasting, vol. 32, no. 5, pp. 1801-1817, 2017.
[8] P. Lin et al., “Objective prediction of warm season afternoon thunderstorms in northern Taiwan using a fuzzy logic approach,” Weather and Forecasting, pp. 1178-1197, vol. 27, no.5, 2012.
[9] T. Kashiwao et al., “A neural network-based local rainfall prediction system using meteorological data on the Internet: A case study using data from the Japan Meteorological Agency,” Applied Soft Computing, vol. 56, pp. 317-330, 2017.
[10] A. Samad et al., “An approach for rainfall prediction using long short term memory neural network,” in Proceedings of the IEEE 5th International Conference on Computing Communication and Automation, Greater Noida, India, 2020, pp. 190-195.
[11] Y. LeCun et al., “Backpropagation applied to handwritten zip code recognition,” Neural Computation, vol. 1, no. 4, pp. 541-555, 1989.
[12] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” Advances in neural information processing systems, vol. 27, 2014.
[13] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[14] J. Redmon et al., “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, 2016, pp. 779-788.
[15] Y. Taigman et al., “Deepface: Closing the gap to human-level performance in face verification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Ohio, 2014, pp. 1701-1708.
[16] D. G. Lowe, “Object recognition from local scale-invariant features,” in Proceedings of the Seventh IEEE International Conference on Computer Vision, Corfu, Greece, 1999, vol. 2, pp. 1150-1157.
[17] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, California, 2005, vol. 1, pp. 886-893.
[18] H. Bay et al., “SURF: Speeded up robust features,” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346-359, 2008.
[19] Y. Lecun et al., “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[20] C. Szegedy et al., “Going deeper with convolutions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, 2015, pp. 1-9.
[21] K. He et al., “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, 2016, pp. 770-778.
[22] K. Gregor et al., “DRAW: A recurrent neural network for image generation,” May 2015. [Online]. Available: https://arxiv.org/abs/1502.04623.
[23] I. Sutskever et al., “Generating text with recurrent neural networks,” in Proceedings of the 28th International Conference on Machine Learning, Bellevue, Washington, 2011, pp. 1017-1024.
[24] A. Graves, A.-R. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, British Columbia, Canada, 2013, pp. 6645-6649.
[25] M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, 1997.
[26] B. Merriënboer et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” Sep 2014. [Online]. Available: https://arxiv.org/abs/1406.1078.
[27] J. Donahue et al., “Long-term recurrent convolutional networks for visual recognition and description,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, 2015, pp. 2625-2634.
[28] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” Jan 2017. [Online]. Available: https://arxiv.org/abs/1412.6980.