Since the construction industry in developed countries has encountered a serious shortage of construction workforce, one of the solutions is to brought in foreign workers where the labor cost is relatively lower. However, productivity issues are still encountered due to the poor management system for foreign workers. Constant monitoring of foreign labor activities can obtain comprehensive information in determining root causes of inefficiency. Column formwork activity is used in this study since the activity is involved in nearly every building construction project and is easily identifiable. This study presents a construction worker productivity evaluation by comparing the productivity of the foreign worker to the local worker with the integration of a deep learning method, named You Only Watched Once, to recognize workers’ actions. Based on the result findings, the YOWO model recognized the worker’s actions with proper evaluation metrics of 78.3%. The recognized actions are then used to measure the productivity for each of the worker based on the duration and identify the ineffective action of the foreign worker’s activity. The result shown that the assemble action of foreign worker is identified as the ineffective action and has the highest priority to be enhanced through training and education.

Since the construction industry in developed countries has encountered a serious shortage of construction workforce, one of the solutions is to brought in foreign workers where the labor cost is relatively lower. However, productivity issues are still encountered due to the poor management system for foreign workers. Constant monitoring of foreign labor activities can obtain comprehensive information in determining root causes of inefficiency. Column formwork activity is used in this study since the activity is involved in nearly every building construction project and is easily identifiable. This study presents a construction worker productivity evaluation by comparing the productivity of the foreign worker to the local worker with the integration of a deep learning method, named You Only Watched Once, to recognize workers’ actions. Based on the result findings, the YOWO model recognized the worker’s actions with proper evaluation metrics of 78.3%. The recognized actions are then used to measure the productivity for each of the worker based on the duration and identify the ineffective action of the foreign worker’s activity. The result shown that the assemble action of foreign worker is identified as the ineffective action and has the highest priority to be enhanced through training and education.

Akhavian, R., & Behzadan, A. H. (2016). Productivity analysis of construction worker activities using smartphone sensors. Paper presented at the Proc., 16th Int. Conf. Comput. Civil Building Eng.
Carreira, J., & Zisserman, A. (2017). Quo vadis, action recognition? a new model and the kinetics dataset. Paper presented at the proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
Cheng, C.-W., & Wu, T.-C. (2013). An investigation and analysis of major accidents involving foreign workers in Taiwan’s manufacture and construction industries. Safety Science, 57, 223-235. doi:https://doi.org/10.1016/j.ssci.2013.02.008
Cheok, G. S., Stone, W. C., Lipman, R. R., & Witzgall, C. (2000). Ladars for construction assessment and update. Automation in Construction, 9(5), 463-477. doi:https://doi.org/10.1016/S0926-5805(00)00058-3
Chi, S., & Caldas, C. H. (2011). Automated Object Identification Using Optical Video Cameras on Construction Sites. Computer-Aided Civil and Infrastructure Engineering, 26(5), 368-380. doi:https://doi.org/10.1111/j.1467-8667.2010.00690.x
Donahue, J., Hendricks, L. A., Rohrbach, M., Venugopalan, S., Guadarrama, S., Saenko, K., & Darrell, T. (2017). Long-Term Recurrent Convolutional Networks for Visual Recognition and Description. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 677-691. doi:10.1109/TPAMI.2016.2599174
Gong, J., & Caldas, C. H. (2011). An object recognition, tracking, and contextual reasoning-based video interpretation method for rapid productivity analysis of construction operations. Automation in Construction, 20(8), 1211-1226. doi:https://doi.org/10.1016/j.autcon.2011.05.005
Gong, J., Caldas, C. H., & Gordon, C. (2011). Learning and classifying actions of construction workers and equipment using Bag-of-Video-Feature-Words and Bayesian network models. Advanced Engineering Informatics, 25(4), 771-782. doi:https://doi.org/10.1016/j.aei.2011.06.002
Goodrum, P. M. (2004). Hispanic and Non-Hispanic Wage Differentials: Implications for United States Construction Industry. Journal of Construction Engineering and Management, 130(4), 552-559. doi:doi:10.1061/(ASCE)0733-9364(2004)130:4(552)
Goodrum, P. M., & Haas, C. T. (2002). Partial Factor Productivity and Equipment Technology Change at Activity Level in U.S. Construction Industry. Journal of Construction Engineering and Management, 128(6), 463-472. doi:doi:10.1061/(ASCE)0733-9364(2002)128:6(463)
Hamid, A. R. A., Singh, B., Yusof, A. M., & Abdullah, N. A. M. (2011). The employment of foreign workers at construction sites. Paper presented at the 2nd International Conference on Construction and Project Management.
Han, S. H., Park, S. H., Jin, E. J., Kim, H., & Seong, Y. K. (2008). Critical Issues and Possible Solutions for Motivating Foreign Construction Workers. Journal of Management in Engineering, 24(4), 217-226. doi:doi:10.1061/(ASCE)0742-597X(2008)24:4(217)
Hara, K., Kataoka, H., & Satoh, Y. (2018). Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? Paper presented at the Proceedings of the IEEE conference on Computer Vision and Pattern Recognition.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.
Hwang, B.-G., Zhu, L., & Ming, J. T. T. (2017). Factors Affecting Productivity in Green Building Construction Projects: The Case of Singapore. Journal of Management in Engineering, 33(3), 04016052. doi:doi:10.1061/(ASCE)ME.1943-5479.0000499
Ibrahim, Y. M., Lukins, T. C., Zhang, X., Trucco, E., & Kaka, A. P. (2009). Towards automated progress assessment of workpackage components in construction projects using computer vision. Advanced Engineering Informatics, 23(1), 93-103. doi:https://doi.org/10.1016/j.aei.2008.07.002
Ji, S., Xu, W., Yang, M., & Yu, K. (2013). 3D Convolutional Neural Networks for Human Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 221-231. doi:10.1109/TPAMI.2012.59
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014, 23-28 June 2014). Large-Scale Video Classification with Convolutional Neural Networks. Paper presented at the 2014 IEEE Conference on Computer Vision and Pattern Recognition.
Kim, K., & Cho, Y. K. (2020). Effective inertial sensor quantity and locations on a body for deep learning-based worker's motion recognition. Automation in Construction, 113, 103126. doi:https://doi.org/10.1016/j.autcon.2020.103126
Kog, Y. C. (2019). Construction Delays in Indonesia, Malaysia, Thailand, and Vietnam. Practice Periodical on Structural Design and Construction, 24(3), 04019013. doi:doi:10.1061/(ASCE)SC.1943-5576.0000434
Köpüklü, O., Wei, X., & Rigoll, G. (2019). You only watch once: A unified cnn architecture for real-time spatiotemporal action localization. arXiv preprint arXiv:1911.06644.
Li, L., Huang, W., Gu, I. Y. H., & Tian, Q. (2003). Foreground object detection from videos containing complex background. Paper presented at the Proceedings of the eleventh ACM international conference on Multimedia, Berkeley, CA, USA. https://doi.org/10.1145/957013.957017
Memarzadeh, M., Golparvar-Fard, M., & Niebles, J. C. (2013). Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors. Automation in Construction, 32, 24-37. doi:https://doi.org/10.1016/j.autcon.2012.12.002
Nasir, H., Ahmed, H., Haas, C., & Goodrum, P. M. (2014). An analysis of construction productivity differences between Canada and the United States. Construction Management and Economics, 32(6), 595-607. doi:10.1080/01446193.2013.848995
Navon, R., & Sacks, R. (2007). Assessing research issues in Automated Project Performance Control (APPC). Automation in Construction, 16(4), 474-484. doi:https://doi.org/10.1016/j.autcon.2006.08.001
Panas, A. (2010). Evaluating Research Methodology in Construction Productivity Studies.
Peddi, A., Huan, L., Bai, Y., & Kim, S. (2009). Development of Human Pose Analyzing Algorithms for the Determination of Construction Productivity in Real-Time. In Building a Sustainable Future (pp. 11-20).
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.
Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.
Simonyan, K., & Zisserman, A. (2014). Two-stream convolutional networks for action recognition in videos. arXiv preprint arXiv:1406.2199.
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015). Learning spatiotemporal features with 3d convolutional networks. Paper presented at the Proceedings of the IEEE international conference on computer vision.
Wang, H., Kläser, A., Schmid, C., & Liu, C. (2011, 20-25 June 2011). Action recognition by dense trajectories. Paper presented at the CVPR 2011.
Xie, S., Girshick, R., Dollár, P., Tu, Z., & He, K. (2017). Aggregated residual transformations for deep neural networks. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.
Yang, J., Shi, Z., & Wu, Z. (2016). Vision-based action recognition of construction workers using dense trajectories. Advanced Engineering Informatics, 30(3), 327-336. doi:https://doi.org/10.1016/j.aei.2016.04.009
Zhu, Y., Li, X., Liu, C., Zolfaghari, M., Xiong, Y., Wu, C., . . . Li, M. (2020). A Comprehensive Study of Deep Video Action Recognition. arXiv preprint arXiv:2012.06567.