簡易檢索 / 詳目顯示

研究生: 王辰軒
Chen-Hsuan Wang
論文名稱: SEMA: 工地車輛管控助理
SEMA: A Site Equipment Management Assistant for Progress Management
指導教授: 楊亦東
I-Tung Yang
蔡孟涵
Meng-Han Tsai
口試委員: 康仕仲
Shih-Chung Kang
楊亦東
I-Tung Yang
蔡孟涵
Meng-Han Tsai
學位類別: 碩士
Master
系所名稱: 工程學院 - 營建工程系
Department of Civil and Construction Engineering
論文出版年: 2020
畢業學年度: 108
語文別: 英文
論文頁數: 76
中文關鍵詞: YOLOv3聊天機器人深度學習人工智慧影像辨識即時通訊軟體光學字元識別
外文關鍵詞: YOLOv3, Chatbot, Deep-learning, Artificial Intelligence, Image recognition, Instant messaging application, Optical Character Recognition
相關次數: 點閱:340下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 在營建業進度管理中,搜集並記錄工程設備資訊是很重要的任務之一,大多數建築施工現場都使用紙本紀錄將工程設備的進出資訊記錄下來。然而,手動紀錄資訊是一件高勞力且耗時的任務。因此,如何自動化監視並且記錄成為進度管理中的關鍵挑戰。本研究開發了一種用於工程設備且具有影像辨識以及多目標追蹤的進度管理系統,名為Site Equipment Management Assistant (SEMA) 該系統可以從施工的監控設備中擷取與工程設備相關的資訊,並且轉為結構化資訊記錄下來。該系統包含七個模組:資訊採集模組,工程設備追蹤模組,濃縮影片擷取模組,主動通知模組,工程設備通行單轉換模組,用戶意圖分析模組以及用戶界面模組。我們訓練了一個可以即時自動檢測並追蹤工程設備的模型。使用此模型,我們可以立即辨識出經過施工現場監控系統的工程設備,並及時通知相關人員。該系統具有儲存工程設備相關資訊的資料庫,可以讓用戶通過具有直觀介面且容易上手的聊天機器人經由關鍵字獲得工程設備相關資訊。根據系統評估結果顯示,該系統的平均精度均值以及召回率分別為87.14%以及69%。該系統也通過使用者測試進行了可用性測試,通過可用性測試結果,驗證該系統於使用者查詢工程設備相關資訊時能夠比紙本傳統作業平均節省718.7秒的時間。通過上述評估以及測試結果,該系統可以有效地提高紀錄以及獲得工程設備相關資訊的效率,對於施工現場的進度管理有顯著的改善。


    Collecting construction equipment information such as the site equipment enter and exit date-time, driver's name, type, and quantity is one of the most essential tasks in project progress management. Most construction projects use paper to record the equipment access history. However, manual recording is always labor-intensive and time-consuming. Therefore, a critical challenge in site progress management is how to automate this manual processing. This study develops a construction site equipment management system with image recognition and multiple object tracking for construction equipment management. The system can extract the equipment-related information from construction site video and convert it into structured data. The system contains seven modules: data acquisition, construction equipment tracking, highlight video extraction, proactive notification, access form detection, user intent analysis, and user interface. We trained a model that can recognize construction equipment automatically in real-time. Using this model, we can instantly recognize the construction equipment passing by the site monitor and notify the relevant personnel. The system also has a construction equipment database to store data. Users can obtain data through a chatbot with an intuitive and easy-to-use interface. Evaluation results showed that the mean average precision and recall of the system for recognizing construction equipment achieves are 87% and 69%, respectively. Through usability testing, the system was validated to be able to save users 718.7 seconds on average for querying construction equipment related information. The results of the usability test showed that the system can effectively improve record efficiency for project progress management and save user time spent querying data.

    Table of contents V List of Figures VII List of Tables X 1. Introduction 1 2. Literature review 3 2.1 Videos in construction management 5 2.2 Construction resource recognition 7 2.3 Challenges of data delivery 10 2.4 Chatbot for data transmission 11 3. Objective 12 4. Methodology 13 4.1 System architecture 13 4.2 Construction equipment tracking module 15 4.2.1 Multiple object tracking 19 4.3 The access form detection module 24 4.4 Highlight video extraction module 24 4.5 User intent analysis module 27 4.6 User interface modules 28 5. Implementation 29 5.1 Dataset preparation and training 30 5.2 Video processing 33 5.3 Chatbot design 36 5.4 Key functions of SEMA 37 5.4.1 Daily report 38 5.4.2 Proactive notification 39 5.4.3 Live stream 40 5.4.4 Access form 40 5.4.5 Weekly report 43 5.4.6 Highlight video 44 5.4.7 Schedule setting 45ㄅ 6. Validation 47 6.1 System evaluation 47 6.2 Usability test 50 6.2.1 Test design 50 6.2.2 Results 53 7. Discussion 57 7.1. Contributions 57 7.2. Future work 59 7.3 Potential Applications of the SEMA 60 8. Conclusion 61 Reference 63

    Abeid, J., Allouche, E., Arditi, D., and Hayman, M. 2003. “PHOTO-NET II: a computer-based monitoring system applied to project management.” Autom. Constr., 12(5), 603–616. https://doi.org/10.1016/S0926-5805(03)00042-6
    Alavi, A. H., and Gandomi, A. H. 2017. “Big data in civil engineering.” Autom. Constr., 79, 1–2. https://doi.org/10.1016/j.autcon.2016.12.008
    Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. 2016. “Simple online and realtime tracking.” In proc., 2016 IEEE Int. Conf. on Image Processing, Phoenix, Arizona, 3464–3468, September 25 - September 8, https://doi.org/10.1109/ICIP.2016.7533003
    Brilakis, I. K., Soibelman, L., and Shinagawa, Y. 2006. “Construction site image retrieval based on material cluster recognition.” Adv. Eng. Inform. , 20(4), 443–452. https://doi.org/10.1016/j.aei.2006.03.001
    Brilakis, I., Park, M.-W., and Jog, G. 2011. “Automated vision tracking of project related entities.” Adv. Eng. Inform. , 25(4), 713–724. https://doi.org/10.1016/j.aei.2011.01.003
    Carranza, K. A. L. R., Manalili, J., Bugtai, N. T., and Baldovino, R. G. 2019. “Expression Tracking with OpenCV Deep Learning for a Development of Emotionally Aware Chatbots.” In proc., 7th Int. Conf. on Robot Intelligence Technology and Applications, Daejeon, Korea: Korea Advanced Institute of Science and Technology, 160–163, November 1 - November November 3, https://doi.org/ 10.1109/RITAPP.2019.8932852
    Chae, S., and Yoshida, T. 2010. “Application of RFID technology to prevention of collision accident with heavy equipment.” Autom. Constr., 19(3), 368–374. https://doi.org/10.1016/j.autcon.2009.12.008
    Chi, S., and Caldas, C. H. 2011. “Automated Object Identification Using Optical Video Cameras on Construction Sites.” Computer-Aided Civil and Infrastructure Engineering, 26(5), 368–380. https://doi.org/ 10.1111/j.1467-8667.2010.00690.x
    Chiaráin, N. N., and Chasaide, A. N. 2016. “Chatbot Technology with Synthetic Voices in the Acquisition of an Endangered Language: Motivation, Development and Evaluation of a Platform for Irish.” In Proc. of the 10th Int. Conf. on Language Resources and Evaluation, Portorož, Slovenia, 3429–3435, May 23 - May 28
    Deng, Z. ., Li, H., Tam, C. ., Shen, Q. ., and Love, P. E. 2001. “An application of the Internet-based project management system.” Autom. Constr., 10(2), 239–246. https://doi.org/10.1016/S0926-5805(99)00037-0
    Elghamrawy, T., and Boukamp, F. 2010. “Managing construction information using RFID-based semantic contexts.” Autom. Constr., 19(8), 1056–1066. https://doi.org/10.1016/j.autcon.2010.07.015
    Ergen, E., Akinci, B., and Sacks, R. 2007. “Tracking and locating components in a precast storage yard utilizing radio frequency identification technology and GPS.” Autom. Constr., 16(3), 354–367. https://doi.org/10.1016/j.autcon.2006.07.004
    Golparvar-Fard, M., Heydarian, A., and Niebles, J. C. 2013. “Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers.” Adv. Eng. Inform., 27(4), 652–663. https://doi.org/10.1016/j.aei.2013.09.001
    Gong, J., and Caldas, C. H. 2011. “An object recognition, tracking, and contextual reasoning-based video interpretation method for rapid productivity analysis of construction operations.” Autom. Constr., 20(8), 1211–1226. https://doi.org/10.1016/j.autcon.2011.05.005
    Goodrum, P. M., McLaren, M. A., and Durfee, A. 2006. “The application of active radio frequency identification technology for tool tracking on construction job sites.” Autom. Constr., 15(3), 292–302. https://doi.org/10.1016/ j.autcon.2005.06.004
    Griol, D., Molina, J. M., and Callejas, Z. 2015. “A proposal for the development of adaptive spoken interfaces to access the Web.” Neurocomputing, 163(2), 56–68. https://doi.org/10.1016/j.neucom.2014.09.087
    Han, S., and Lee, S. 2013. “A vision-based motion capture and recognition framework for behavior-based safety management.” Autom. Constr., 35, 131–141. https://doi.org/10.1016/j.autcon.2013.05.001
    Han, S., Lee, S., and Peña-Mora, F. 2014. “Comparative Study of Motion Features for Similarity-Based Modeling and Classification of Unsafe Actions in Construction.” J. Comput. Civ. Eng., 28(5), A4014005. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000339
    Heydarian, A., Golparvar-Fard, M., and Niebles, J. C. 2012. “Automated Visual Recognition of Construction Equipment Actions Using Spatio-Temporal Features and Multiple Binary Support Vector Machines.” Construction Research Cong. of American Society of Civil Engineers, Reston, VA, 889–898. May 21 - May 23, https://doi.org/10.1061 /9780784412329.090
    Hui, L., Park, M.-W., and Brilakis, I. 2015. “Automated Brick Counting for Façade Construction Progress Estimation.” J. Comput. Civ. Eng., 29(6), 04014091. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000423
    Jaselskis, E. J., and El-Misalami, T. 2003. “Implementing Radio Frequency Identification in the Construction Process.” J. Constr. Eng. Manag., American Society of Civil Engineers, 129(6), 680–688. https://doi.org/10.1061/(ASCE)0733-9364(2003)129:6(680)
    Kamal, R., Janaka, R., and Siri, F. 2013. “Automated Real-Time Monitoring System to Measure Shift Production of Tunnel Construction Projects.” J. Comput. Civ. Eng., 27(1), 68–77. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000199
    Katz, I., Saidi, K., and Lytle, A. 2008. “The role of camera networks in construction automation.” In Vol. 19 of Proc., 25th Int. Symp on Automation and Robotics in Construction., edited by E.K. Zavadskas, Vilnius Gediminas Technical University Publishing House Technika, Vilnius, Lithuania, 324–329, Jun 26 - Jun 29, https://doi.org/10.3846/isarc.20080626.324
    Kim, H., Kim, K., and Kim, H. 2016. “Vision-Based Object-Centric Safety Assessment Using Fuzzy Inference: Monitoring Struck-By Accidents with Moving Objects.” J. Comput. Civ. Eng., 30(4), 04015075. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000562
    Kosugi, M., and Uchida, O. 2019. “Chatbot Application for Sharing Disaster-information.” In 2019 Conf. on Information and Communication Technologies for Disaster Management, Paris, France, 1–2, December 18 - December 20, https://doi.org/10.1109/ICT-DM47966.2019.9032901
    Li, H., Chen, Z., and Wong, C. T. C. 2003. “Barcode Technology for an Incentive Reward Program to Reduce Construction Wastes.” Computer-Aided Civil and Infrastructure Engineering, 18(4), 313–324. https://doi.org/10.1111/1467-8667.00320
    Liu, C.-W., Wu, T.-H., Tsai, M.-H., and Kang, S.-C. 2018. “Image-based semantic construction reconstruction.” Autom. Constr. , 90, 67–78. https://doi.org/10.1016/j.autcon.2018.02.016
    Lu, M., Chen, W., Shen, X., Lam, H.-C., and Liu, J. 2007. “Positioning and tracking construction vehicles in highly dense urban areas and building construction sites.” Autom. Constr., 16(5), 647–656. https://doi.org/10.1016/j.autcon.2006.11.001
    Mani, G.-F., Feniosky, P.-M., and Silvio, S. 2015. “Automated Progress Monitoring Using Unordered Daily Construction Photographs and IFC-Based Building Information Models.” J. Comput. Civ. Eng., 29(1), 4014025. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000205
    Muslih, M., Somantri, Supardi, D., Multipi, E., Nyaman, Y. M., Rismawan, A., and Gunawansyah. 2018. “Developing Smart Workspace Based IOT with Artificial Intelligence Using Telegram Chatbot.” 2018 Int. Conf. on Computing, Engineering, and Design, Bangkok, Thailand, 230–234, September 6 - September 8
    Oh, S.-W., Chang, H.-J., Kim, Y.-S., Lee, J.-B., and Kim, H.-S. 2004. “An Application of PDA and Barcode Technology for the Improvement of Information Management in Construction Projects.” In Proc., 21st Int. Symp. on Automation and Robotics in Construction of International Association for Automation and Robotics in Construction, Jeju, South Korea, September 21 - Spetember 25, https://doi.org/10.22260/ISARC2004/0090
    Oloufa, A. A., Ikeda, M., and Oda, H. 2003. “Situational awareness of construction equipment using GPS, wireless and web technologies.” Autom. Constr., 12(6), 737–748. https://doi.org/10.1016/S0926-5805(03)00057-8
    Park, M.-W., and Brilakis, I. 2012. “Construction worker detection in video frames for initializing vision trackers.” Autom. Constr., 28, 15–25. https://doi.org/10.1016/j.autcon.2012.06.001
    Park, M.-W., and Brilakis, I. 2016. “Continuous localization of construction workers via integration of detection and tracking.” Autom. Constr., 72, 129–142. https://doi.org/10.1016/j.autcon.2016.08.039
    Park, M.-W., Elsafty, N., and Zhu, Z. 2015. “Hardhat-Wearing Detection for Enhancing On-Site Safety of Construction Workers.” J. Constr. Eng. Manag., 141(9), 04015024. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000974
    Park, M.-W., Koch, C., and Brilakis, I. 2012. “Three-Dimensional Tracking of Construction Resources Using an On-Site Camera System.” J. Comput. Civ. Eng., 26(4), 541–549. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000168
    Rashidi, A., Sigari, M. H., Maghiar, M., and Citrin, D. 2016. “An analogy between various machine-learning techniques for detecting construction materials in digital images.” KSCE J. Civ. Eng., 20(4), 1178–1188. https://doi.org/10.1007/s12205-015-0726-0
    Ren, X., Zhu, Z., Germain, C., Dean, B., and Chen, Z. 2015. “A Case Study of Construction Equipment Recognition from Time-Lapse Site Videos under Low Ambient Illuminations.” In proc. 2015 Computing in Civil Engineering of American Society of Civil Engineers, Austin, Texas , 82–89. June 21 - June 23, https://doi.org/10.1061/9780784479247.011
    Shawar, B. A., and Atwell, E. S. 2005. “Using corpora in machine-learning chatbot systems.” Int. J. Corpus Linguist., 10(4), 489–516. https://doi.org/10.1075/ijcl.10.4.06sha
    Slaton, T., Hernandez, C., and Akhavian, R. 2020. “Construction activity recognition with convolutional recurrent networks.” Autom. Constr., 113, 103138. https://doi.org/10.1016/j.autcon.2020.103138
    Son, H., Kim, C., Hwang, N., Kim, C., and Kang, Y. 2014. “Classification of major construction materials in construction environments using ensemble classifiers.” Adv. Eng. Inform., 28(1), 1–10. https://doi.org/10.1016/j.aei.2013.10.001
    Tajeen, H., and Zhu, Z. 2014. “Image dataset development for measuring construction equipment recognition performance.” Autom. Constr., 48, 1–10. https://doi.org/10.1016/j.aei.2013.10.001
    Teizer, J., Allread, B. S., Fullerton, C. E., and Hinze, J. 2010. “Autonomous pro-active real-time construction worker and equipment operator proximity safety alert system.” Autom. Constr., 19(5), 630–640. https://doi.org/10.1016/j.autcon.2010.02.009
    Teizer, J., and Vela, P. A. 2009. “Personnel tracking on construction sites using video cameras.” Adv. Eng. Inform., 23(4), 452–462. https://doi.org/10.1016/j.aei.2009.06.011
    Teizer, J., Venugopal, M., and Walia, A. 2008. “Ultrawideband for Automated Real-Time Three-Dimensional Location Sensing for Workforce, Equipment, and Material Positioning and Tracking.” Int. J. Corpus Linguist., 2081(1), 56–64. https://doi.org/10.3141/2081-06
    Tsai, M.-H., Chen, J., and Kang, S.-C. 2019. “Ask Diana: A Keyword-Based Chatbot System for Water-Related Disaster Management.” Water, 11(2), 234. https://doi.org/10.3390/w11020234
    TURING, A. M. 1950. “I.—COMPUTING MACHINERY AND INTELLIGENCE.” Mind, LIX(236), 433–460. https://doi.org/10.1093/mind/LIX.236.433
    Turkan, Y., Bosche, F., Haas, C. T., and Haas, R. 2012. “Automated progress tracking using 4D schedule and 3D sensing technologies.” Autom. Constr., 22, 414–421. https://doi.org/10.1016/j.autcon.2011.10.003
    Weizenbaum, J. 1966. “ELIZA---a computer program for the study of natural language communication between man and machine.” Commun. ACM., 9(1), 36–45. https://doi.org/10.1145/365153.365168
    Xie, Q., Tan, D., Zhu, T., Zhang, Q., Xiao, S., Wang, J., Li, B., Sun, L., and Yi, P. 2019. “Chatbot Application on Cryptocurrency.” In 2019 IEEE Conf. Computational Intelligence for Financial Engineering & Economics, Shenzhen, China, 1–8, May 4 - May 5, https://doi.org/10.1109/CIFEr.2019.8759121
    Yang, J., Cheng, T., Teizer, J., Vela, P. A., and Shi, Z. K. 2011. “A performance evaluation of vision and radio frequency tracking methods for interacting workforce.” Adv. Eng. Inform., 25(4), 736–747. https://doi.org/10.1016/j.aei.2011.04.001
    Yang, J., Shi, Z., and Wu, Z. 2016. “Vision-based action recognition of construction workers using dense trajectories.” Adv. Eng. Inform., 30(3), 327–336. https://doi.org/10.1016/j.aei.2016.04.009

    無法下載圖示 全文公開日期 2025/08/25 (校內網路)
    全文公開日期 2025/08/25 (校外網路)
    全文公開日期 2025/08/25 (國家圖書館:臺灣博碩士論文系統)
    QR CODE