現今在進行城市土木工程前，業界會先透過由航拍影像重建的 3D 實景來查看現場樣貌，以利測量、設計與檢討。在這些城市景觀設計的工程中，以道路為最大面積之工程。在一般道路設計的流程中，必須以人工的方式辨認道路區域以及道路標線的位置，並繪製於場景中。將所有道路資訊繪製完成後，再以人工 3D 建模的方式，將道路以及標線重新建置出來。如此的作業模式，相當的耗費人力!因此我們希望能夠提出一個方法自動提取影像中的資訊，取代人工提取的方式。

本論文提出一套自動化的道路資訊辨識與重建系統，能夠自動提取道路區域以及道路標線並且重建。首先，我們藉由地景模型生成高解析度的俯視圖，並標註我們所需要提取的特徵，道路與道路標線。接著，使用語義分割的神經網路來訓練道路與線條標線的預測模型；使用實例分割的神經網路來訓練圖形標線的預測模型。在訓練完預測模型之後，我們利用這三個預測模型進行道路資訊的提取。我們針對三種預測結果提出各自的擬合方法：藉由預測影像後得到的道路區域，擬合出道路的邊界；藉由預測影像後得到的線條標線區域，擬合出線條標線的線條；藉由預測影像後得到的圖形標線區域，擬合出圖形標線的矩形邊界。最後，將擬合完成的道路資訊匯入 blender 3D 工具，利用 3D 建模的方式生成新的道路以及道路標線。

我們使用 68 個地景模型來進行預測模型的訓練與評測。根據實驗結果，道路區域之精確率、召回率達到90%以上。在線條類型的標線部分，除了黃色實線外，皆有 60%以上的精確率。最後，在圖形類型的標線部分，黃色網狀線、枕木紋行人穿越道、機車停等區，皆有60%以上的召回率，精確率也能達到95%以上。而待轉區與分隔島，召回率的表雖然不高（50%以下），但是在精確率的部分仍有較好的表現。綜合以上實驗結果，在道路的部分，道路邊界比其他研究更加顯著，並且擬合道路邊界，以利道路重建。在道路標線部分，相較於其他研究，像素的種類更多，並且能夠擬合線條標線的線條與圖形標線的邊界，以利道路標線重建。另外，我們也創建一套標註好的道路資訊資料集（AORI），能夠提供給其他研究使用。

Nowadays, before carrying out urban civil engineering, we will look at the scene first through the 3D real scene reconstructed from aerial images to facilitate measurement, design and review. Among these urban landscape design projects, the process of general road design, the road area and the position of road markings must be manually identified and reconstructed in the scene. This method consumes a lot of human resources. Therefore, we hope to propose methods to automatically extract the road information.

First, we use the landscape model to generate dataset for training the semantic segmentation models of road and road marking and for training the instance segmentation model of grouped road marking. Then, these models are used to extract road information. We propose respective fitting methods for the three predicted results: use road segmented mask for fitting the road boundary; use road marking segmented mask for fitting the line of the road marking; use grouped road marking segmented mask for fitting boundary of the grouped road marking. Finally, we reconstruct road and road marking.

We use 68 landscape models for training and evaluating the models. As the experimental results shown, the precision and recall of the road area is above 90%. Except for the yellow single line, all the road markings have precision higher than 60%. The box junction, sleeper crosswalk, and motorcycle waiting zone have recall higher than 60%, and precision can also reach higher than 95%. For the waiting area and divisional island, although the recall is below 50%, it still has a better performance in the precision. In summary, predicted results of road and road marking are better than previous studies, and predicted results can be used to reconstruct road information practically. In addition, we have also created a raod information dataset（AORI）, which can be used for other research.

[1] Y. Li, B. Peng, L. He, K. Fan, and L. Tong, “Road segmentation of unmanned aerial vehicle remote sensing images using adversarial network with multiscale context aggregation,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, pp. 2279–2287, 2019.

[2] S. M. Azimi, P. Fischer, M. Körner, and P. Reinartz, “Aerial lanenet: Lanemarking semantic segmentation in aerial imagery using waveletenhanced
costsensitive symmetric fully convolutional neural networks,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 57, pp. 2920–2938, 2019.

[3] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770– 778, 2016.

[4] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269, 2017.

[5] S. Karen and Z. Andrew, “Very deep convolutional networks for largescale
image recognition,” arXiv 1409.1556, 2014.

[6] K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask rcnn,”
in IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988, 2017.

[7] J. Redmon and A. Farhadi, “Yolo9000: Better, faster, stronger,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525, 2017.

[8] S. Ren, K. He, R. Girshick, and J. Sun, “Faster rcnn: Towards realtime object detection with region proposal networks,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, pp. 1137–1149, 2017.

[9] L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic image segmentation with deep convolutional nets and fully connected crfs,” arXiv: 1412.7062, 2014.

[10] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440, 2015.

[11] O. Ronneberger, P. Fischer, and T. Brox, “Unet: Convolutional networks for biomedical image segmentation,” arXiv:1505.04597, 2014.

[12] J. H. Kim, H. Lee, S. J. Hong, S. Kim, J. Park, J. Y. Hwang, and J. P. Choi, “Objects segmentation from highresolution aerial images using unet with pyramid pooling layers,” in IEEE Geoscience and Remote Sensing Letters, vol. 16, pp. 115–119, 2019.

[13] I. G, M. Mehdi, B. Xu, W. David, O. Sherjil, C. Aaron, and B. Yoshua “Generative adversarial nets,” in Advances in neural information processing systems, pp. 2672– 2680, 2014.

[14] H. Jin, M. Miska, E. Chung, X. Li, and Y. Feng, “Feature extraction from high resolution aerial images upon rural regions based on multiresolution image analysis and gabor filters,” in Remote Sensing Advanced Techniques and Platforms, pp. 387–414, 2012.

[15] H. Jin and Y. Feng, “Automated road pavement marking detection from high resolution aerial images based on multiresolution image analysis and anisotropic gaussian filtering,” in International Conference on Signal Processing Systems (ICSPS), pp. V1–337, 2010.

[16] S. Hinz and A. Baumgartner, “Automatic extraction of urban road networks from multiview aerial imagery,” in International Society for Photogrammetry and Remote Sensing (ISPRS), pp. 83–98, 2003.

[17] G. Mattyus, S. Wang, S. Fidler, and R. Urtasun, “Hd maps: Finegrained road segmentation by parsing ground and aerial images,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3611–3619, 2016.

[18] J. Kim and C. Park, “Endtoend ego lane estimation based on sequential transfer learning for selfdriving cars,” in IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1194–1202, 2017.

[19] A. Gurghian, T. Koduri, S. V. Bailur, K. J. Carey, and V. N. Murali, “Deeplanes: Endtoend lane position estimation using deep neural networks,” in IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 38–45, 2016.

[20] S. Lee, J. Kim, J. S. Yoon, S. Shin, O. Bailo, N. Kim, T. Lee, H. S. Hong, S. Han, and I. S. Kweon, “Vpgnet: Vanishing point guided network for lane and road marking detection and recognition,” in IEEE International Conference on Computer Vision (ICCV), pp. 1947–1955, 2017.

[21] Wikipedia, “Douglas peucker algorithm — Wikipedia, the free encyclopedia.” https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm. [Online; accessed 12May2020].

[22] Wikipedia, “Least squares — Wikipedia, the free encyclopedia.” https://en.
wikipedia.org/wiki/Least_squares. [Online; accessed 12May2020].

[23] Wikipedia, “Curve fitting — Wikipedia, the free encyclopedia.” https://en.
wikipedia.org/wiki/Curve_fitting. [Online; accessed 12May2020].

[24] 交通部, “道路交通標誌標線號誌設置規則.” https://law.moj.gov.tw/ LawClass/LawAll.aspx?pcode=K0040014. [Online; accessed 12May2020].