植物在世界上處處可見，而樹的重建於遊戲場景、虛擬世界、道路設計、景觀設計等等方面都應用的上。用實景影像建模出的3D 地景模型，常常因為拍攝時的資料不足或本身物體太過精細，造成建模結果會失真。地景模型如要進行後續的景觀設計等等應用，得人工慢慢選取失真的樹mesh 並刪除掉，再放上新的景觀樹模型。因此，我們希望能自動把樹換成不失真且參數化的相似樹模型。而且，如要進行後續的地景編輯或動畫輸出，一般樹模型的polygon 數太高，軟體渲染的時間長，常導致編輯畫面卡頓或輸出時間過久。

本論文提出一套對地景模型中的樹木自動辨識與簡化重建的流程：我們整合現有軟體提供之將照片集建模3D 世界的模組，建構了一個雲端自動建模系統，以生成我們實驗的資料來源。然後，把地景模型從top view 生成RGB 與height 影像，再用深度學習Cascade RCNNwith Height 來辨識樹木位置。將每個辨識出來的樹mesh 切割出來，除了可以自動清除樹之外，我們也對其進行逆程序化建模的重建。接著，用我們預訓練好的深度學習Parameter Prediction Model ，對樹預測程序化建模的參數，生成出與原樹相似的參數化樹模型。之後，將重建的樹模型進行不同程度的樹葉分群，以簡化樹的polygon ，生成有DLOD 的billboardingtree ，增加rendering 效能。

我們用50 組臺灣空拍照片集，自動建模出了50 個、總共約40 平方公里的地景模型來進行實驗。根據實驗結果，我們的樹木辨識對獨立單一棵樹跟樹欉，在一般的case 下分別有97.3% 和98.5% 的面積被辨識出來，且即使是在不好的case下也有不錯的表現。在樹重建結果的Likert scale 相似度評估中，重建前後的3D模型整體來說有93.5% 的相似度，證實了我們用深度學習來進行逆程序化建模是可行的；且相較使用Markov Chain Monte Carlo 的逆程序化建模方法，推估一棵樹的參數動輒幾分鐘到幾小時，使用深度學習只要幾毫秒的時間。最後，我們的billboarding tree with DLOD ，大幅降低了參數化樹模型的polygon 數，在百棵樹左右的場景中，減少了五成以上的渲染時間。

另外，我們辨識地景模型樹的資料集，也公開於網路上供其他人研究使用。

Plants are everywhere in the world, and the reconstruction of trees is used in game scenes, virtual worlds, road design, landscape design and so on. For landscape applications such as landscape design, the mesh of the distorted landscape tree model must be manipulated manually, and so consume considerable manpower. Therefore, we hope to automatically replace the tree with an undistorted and parametric tree model. Moreover, the number of polygons of a tree is too high to efficiently rendering if not simplified.

In this thesis, we propose a process of automatic detection and reconstruction of trees in the landscape model. We build a Cloud Automatic Modeling System to generate the data source, RGB and height images from the top view of the landscape model, for our experiments to detect the trees. Then we detect the trees using Cascade RCNN with height. By cutting out each detected tree mesh, we can not only automatically clear the tree, but also reconstruct it by the proposed inverse procedural modeling approach. Next, we train a Parameter Prediction Model to predict the parameters of the procedural modeling of the tree and generate the parametric tree model similar to the original tree. Finally, the generated tree model is divided into different levels of leaves group to simplify the polygon, and we generate a billboarding tree with DLOD to increase rendering efficiency.

We have used 50 sets of Taiwan aerial images to automatically reconstruct 50 landscape models about 40 square kilometers for experiments. According to the experimental results, our tree detection is with recall rates of 97.3% and 98.5% of single tree and grove for general cases respectively. Even in worse cases, our tree detection is with recall rate of 94.7%. Also, we have measured the similarity between inversed procedural generated trees and corresponding reconstructed input trees based on Likert scale approach, and have an overall similarity of 93.5%. Furthermore, our proposed billboarding tree with DLOD method greatly reduces the number of polygons of the parametric tree model, speeding up the rendering time higher than 50% on average. In addition, the dataset of our landscape model will publish on the Internet for others to study.

[1] J. Guo, S. Xu, D. Yan, Z. Cheng, M. Jaeger, and X. Zhang, “Realistic procedural plant modeling from multiple view images,” IEEE Transactions on Visualization and Computer Graphics, vol. 26, no. 2, pp. 1372–1384, 2020.
[2] O. Stava, S. Pirk, J. Kratt, B. Chen, R. Mech, O. Deussen, and B. Benes, “Inverse procedural modelling of trees,” Computer Graphics Forum, vol. 33, no. 6, pp. 118– 131, 2014.
[3] Z. Cai and N. Vasconcelos, “Cascade RCNN: High quality object detection and instance segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2019.
[4] G. Nishida, A. Bousseau, and D. G. Aliaga, “Procedural modeling of a building from a single image,” Computer Graphics Forum, vol. 37, no. 2, pp. 415–429, 2018.
[5] O. Deussen and B. Lintermann, Digital Design of Nature:Computer Generated Plants and Organics. SpringerVerlag Berlin Heidelberg, 2006.
[6] P. Prusinkiewicz, “Modeling plant growth and development,” Current Opinion in Plant Biology, vol. 7, no. 1, pp. 79 – 83, 2004.
[7] R. M. Smelik, T. Tutenel, R. Bidarra, and B. Benes, “A survey on procedural modelling for virtual worlds,” Comput. Graph. Forum, vol. 33, p. 31–50, Sept. 2014.
[8] P. Prusinkiewicz, M. Hammel, J. Hanan, and R. Mech, “Lsystems: From the theory to visual models of plants,” CSIRO Symposiumon on Computational Challanges in Life Sciences, pp. 1 – 32, 1996.
[9] R. Měch and P. Prusinkiewicz, “Visual models of plants interacting with their environment,” in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ＇96, (New York, NY, USA), p. 397–410, Association for Computing Machinery, 1996.
[10] S. Pirk, O. Stava, J. Kratt, M. A. Massih Said, B. Neubert, R. Mech, B. Benes, and O. Deussen, “Plastic trees : Interactive selfadapting botanical tree models,” ACM Transactions on Graphics, vol. 31, no. 4, pp. 1–10, 2012.
[11] B. Lintermann and O. Deussen, “Interactive modeling of plants,” IEEE Computer Graphics and Applications, vol. 19, no. 1, pp. 56–65, 1999.
[12] Interactive Data Visualization, Inc.:SpeedTree, “http://www.speedtree.com/,” Accessed March 2020.
[13] S. Longay, A. Runions, F. Boudon, and P. Prusinkiewicz, “TreeSketch: Interactive Procedural Modeling of Trees on a Tablet,” in Eurographics Workshop on SketchBased Interfaces and Modeling (K. Singh and L. B. Kara, eds.), The Eurographics Association, 2012.
[14] H. Xu, N. Gossett, and B. Chen, “Knowledge and heuristicbased modeling of laserscanned trees,” ACM Transactions on Graphics, vol. 26, p. 19–es, Oct. 2007.
[15] Y. Livny, F. Yan, M. Olson, B. Chen, H. Zhang, and J. ElSana, “Automatic reconstruction of tree skeletal structures from point clouds,” ACM Transactions on Graphics, vol. 29, Dec. 2010.
[16] Z. Wang, L. Zhang, T. Fang, P. T. Mathiopoulos, H. Qu, D. Chen, and Y. Wang, “A structureaware global optimization method for reconstructing 3d tree models from terrestrial laser scanning data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 9, pp. 5653–5669, 2014.
[17] N. Snavely, S. M. Seitz, and R. Szeliski, “Modeling the world from internet photo collections,” International Journal of Computer Vision, vol. 80, no. 2, pp. 189–210, 2008.
[18] J. Guo, D.M. Yan, L. Chen, X. Zhang, O. Deussen, and P. Wonka, “Tetrahedral meshing via maximal poissondisk sampling,” Computer Aided Geometric Design, vol. 43, pp. 186 – 199, 2016. Geometric Modeling and Processing 2016.
[19] W. Palubicki, K. Horel, S. Longay, A. Runions, B. Lane, R. Měch, and P. Prusinkiewicz, “Selforganizing tree models for image synthesis,” ACM Transactions on Graphics, vol. 28, July 2009.
[20] X. Zhang, H. Li, M. Dai, W. Ma, and L. Quan, “Datadriven synthetic modeling of trees,” IEEE Transactions on Visualization and Computer Graphics, vol. 20, no. 9, pp. 1214–1226, 2014.
[21] N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equationofstate calculations by fast computing machines,” Journal of Chemical Physics, vol. 21, 6, pp. 1087–1092, 1953.
[22] M. G. Cline and C. A. Harrington, “Apical dominance and apical control in multiple flushing of temperate woody species,” Canadian Journal of Forest Research, vol. 37, no. 1, pp. 74–83, 2007.
[23] O. K.C. Au, C.L. Tai, H.K. Chu, D. CohenOr, and T.Y. Lee, “Skeleton extraction by mesh contraction,” ACM Transactions on Graphics, vol. 27, p. 1–10, Aug. 2008.
[24] N. Greene, “Voxel space automata: Modeling with stochastic growth processes in voxel space,” SIGGRAPH Comput. Graph., vol. 23, p. 175–184, July 1989.
[25] S. Ren, K. He, R. Girshick, and J. Sun, “Faster rcnn: Towards realtime object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, 2017.
[26] Bently, Inc.:ContextCapture, “https:// www.bentley.com/ en/ products/ brands/ contextcapture,” Accessed April 2020.
[27] Blender Foundation.:Sapling Tree Gen, “https://docs.blender.org/manual/en/latest/ addons/add_curve/sapling.html,” Accessed April 2020.
[28] B. H. Tsai, “Quality assessment of large scaled UAV photogrammetry study–example of Small Kinmen (Lieyue islet),” Master’s thesis, National Taipei University of Technology, Taipei, 7 2015.
[29] S. M. Azimi, P. Fischer, M. Körner, and P. Reinartz, “Aerial lanenet: Lanemarking semantic segmentation in aerial imagery using waveletenhanced costsensitive symmetric fully convolutional neural networks,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 5, pp. 2920–2938, 2019.
[30] K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin, “MMDetection: Open mmlab detection toolbox and benchmark,” ArXiv, vol. abs/1906.07155, 2019.
[31] F.Alsaade, N. Zaman, M. Z. Dawood, and S. H. A. Musavi, “Effectiveness of score normalization in multimodal biometric fusion,” Journal of Information & Communication Technology, vol. 3, no. 1, pp. 29–35, 2009.
[32] S. Xie, R. B. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5987–5995, 2017.
[33] S. S. Wu, Urban Humanistic Transportation Planning and Design Manual. Construction and Planning Agency of the Ministry of the Interior, 2 ed., 11 2018.
[34] C. Y. Chang and Y. C. Pern, “The research on buttresses’ formation of six landscape trees in natural condition of Taiwan,” Tunghai Journal, vol. 46, pp. 165–175, 2005.
[35] C. Y. Chang, “The debated street trees in Taiwan,” Tunghai Journal, vol. 26, no. 3, pp. 83–102, 2004.
[36] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800–1807, 2017.
[37] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826, 2016.
[38] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4510–4520, 2018.
[39] K. Simonyan and A. Zisserman, “Very deep convolutional networks for largescale image recognition,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
[40] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proceedings of the 32nd International Conference on International Conference on Machine Learning Volume 37, ICML＇15, p. 448–456, JMLR.org, 2015.
[41] I. Garcia, M. Sbert, and L. SzirmayKalos, “Tree rendering with billboard clouds,” Third Hungarian Conference on Computer Graphics and Geometry, 01 2005.
[42] G. Salvendy and W. Karwowski, Advances in Human Factors and Ergonomics 2012, vol. 14. CRC Press, 1 ed., 8 2012.