作業員在產線中扮演靈活彈性角色，而確實執行SOP步驟則成為產線維持品質的關鍵。因此本研究提出一種基於CNN深度學習的智慧彈性操作指引模型，模型核心包含三項任務程序：任務一為物件識別，作業員執行工作的SOP過程切割為拿取、放置、進行鎖附，此三個動作模組基於YOLO物件辨識，決定了六項類別分別為Screwdriver、GPU、Jig、HoldingScrewdriver、HoldingGPU、GPUonJig；任務二為計算物件訊息例如物件出現狀態、所在位置、方向性、總體數量，獲取以上資訊由已建立之SOP動作定義表判斷出當下動作，並集合多幀的結果進行多數決提升動作辨識結果；任務三為判斷當下是否符合依據彈性架構下的SOP操作流程，不符合且檢測出異常時，模型分別輸出順序、超時、異物檢測等不同聲音建議警示。此模型研究應用於GPU主機板側蓋之螺絲鎖附站點，站點的組成包含一位作業員、螺絲刀、治具、待鎖物主機板、機構與輸送帶。結果經過四位作業員實測評估後，均能成功導入模型且Accuracy 為98.89%，Precision 為99.86%，Recall值為99.02%，F1-score為99.43%，以驗證此擬議模型在現實世界中的可行性。

Workers play a flexible role in the production line, and accurately executing standard operation procedure (SOP) is crucial for maintaining quality on a production line. This study proposes an intelligent flexible operation guidance model based on CNN deep learning, with the core of the model consisting of three task procedures: Task 1 is object recognition, where the worker's SOP process is divided into pick-up, placement, and attachment actions based on YOLO object recognition, determining six categories: Screwdriver, GPU, Jig, Holding Screwdriver, Holding GPU, and GPU on Jig. Task 2 involves calculating object information such as object appearance status, position, direction, and overall quantity; the current action is determined based on the established SOP action definition table, and the results from multiple frames are combined to improve action recognition through majority voting. Task 3 assesses whether the current situation complies with the SOP operation process under a flexible framework, and when non-compliance and abnormalities are detected, the model outputs sound warnings for sequence, overtime, and foreign object detection. This model is applied to the screw attachment site of GPU motherboard side covers, with the site consisting of a worker, a screwdriver, a jig, a motherboard to be assembled, a mechanism, and a conveyor belt. After evaluation by four workers, the model was successfully integrated with high accuracy, with an Accuracy of 98.89%, a Precision of 99.86%, a Recall of 99.02%, an F1-score of 99.43%, demonstrating the feasibility of the proposed model.

Akbar, A. Z., Fatichah, C., & Dikairono, R. (2022, November). Autonomous Surface Vehicle in Search and Rescue Process of Marine Casualty using Computer Vision Based Victims Detection. In 2022 International Conference on Computer Engineering, Network, and Intelligent Multimedia (CENIM) (pp. 1-6). IEEE.
Areeb, Q. M., Nadeem, M., Alroobaea, R., & Anwer, F. (2022). Helping hearing-impaired in emergency situations: a deep learning-based approach. IEEE Access, 10, 8502-8517.
Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.
Bottani, E., Montanari, R., Volpi, A., Tebaldi, L., & Di Maria, G. (2021). Statistical Process Control of assembly lines in a manufacturing plant: Process Capability assessment. Procedia Computer Science, 180, 1024-1033.
Chen, Y., Luo, Y., Yang, C., Yerebakan, M. O., Hao, S., Grimaldi, N., ... & Hu, B. (2022). Human mobile robot interaction in the retail environment. Scientific Data, 9(1), 673.
Domingo, J. D., Gómez-García-Bermejo, J., & Zalama, E. (2022). Improving human activity recognition integrating lstm with different data sources: Features, object detection and skeleton tracking. IEEE Access, 10, 68213-68230.
Gellert, A., Precup, S. A., Pirvu, B. C., & Zamfirescu, C. B. (2020, September). Prediction-based assembly assistance system. In 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) (Vol. 1, pp. 1065-1068). IEEE.
Gorecky, D., Schmitt, M., Loskyll, M., & Zühlke, D. (2014, July). Human-machine-interaction in the industry 4.0 era. In 2014 12th IEEE international conference on industrial informatics (INDIN) (pp. 289-294). Ieee.
Ji, S. J., Ling, Q. H., & Han, F. (2023). An improved algorithm for small object detection based on YOLO v4 and multi-scale contextual information. Computers and Electrical Engineering, 105, 108490.
Jianhua, L., Qingchao, S., Hui, C., Xiaokang, L. I. U., Xiaoyu, D. I. N. G., Shaoli, L. I. U., & Hui, X. I. O. N. G. (2018). The state-of-the-art, connotation and developing trends of the products assembly technology. Journal of Mechanical Engineering, 54(11), 2-28.
Kassab, M. A., Ahmed, M., Maher, A., & Zhang, B. (2020). Real-time human-UAV interaction: New dataset and two novel gesture-based interacting systems. IEEE Access, 8, 195030-195045.
Kim, H., Kim, J., You, J. M., Lee, S. W., Kyung, K. U., & Kwon, D. S. (2021). A sigmoid-colon-straightening soft actuator with peristaltic motion for colonoscopy insertion assistance: Easycolon. IEEE Robotics and Automation Letters, 6(2), 3577-3584.
Kumar, A. (2023). SEAT-YOLO: A Squeeze-Excite and Spatial Attentive You Only Look Once Architecture for Shadow Detection. Optik, 170513.
Li, J., Pang, D., Zheng, Y., Guan, X., & Le, X. (2022). A flexible manufacturing assembly system with deep reinforcement learning. Control Engineering Practice, 118, 104957.
Li, Z., Xiong, J., & Chen, H. (2022, September). Based on improved YOLO_v3 for college students’ classroom behavior recognition. In 2022 International Conference on Artificial Intelligence and Computer Information Technology (AICIT) (pp. 1-4). IEEE.
Ling, S., Guo, D., Rong, Y., & Huang, G. Q. (2022). Real-time data-driven synchronous reconfiguration of human-centric smart assembly cell line under graduation intelligent manufacturing system. Journal of Manufacturing Systems, 65, 378-390.
Liu, C., Li, X., Li, Q., Xue, Y., Liu, H., & Gao, Y. (2021). Robot recognizing humans intention and interacting with humans based on a multi-task model combining ST-GCN-LSTM model and YOLO model. Neurocomputing, 430, 174-184.
Liu, T., Lyu, E., Wang, J., & Meng, M. Q. H. (2021). Unified Intention Inference and Learning for Human–Robot Cooperative Assembly. IEEE Transactions on Automation Science and Engineering, 19(3), 2256-2266.
Margherita, E. G., & Braccini, A. M. (2020). Industry 4.0 technologies in flexible manufacturing for sustainable organizational value: reflections from a multiple case study of Italian manufacturers. Information Systems Frontiers, 1-22.
Masehian, E., & Ghandi, S. (2021). Assembly sequence and path planning for monotone and nonmonotone assemblies with rigid and flexible parts. Robotics and Computer-Integrated Manufacturing, 72, 102180.
Mueller, R., Hoerauf, L., & Bashir, A. (2020, September). Assembly process prediction with digital assistance systems to ensure synchrony between digital and physical product state using recurrent neural networks. In 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) (Vol. 1, pp. 513-517). IEEE.
Nelles, J., Kuz, S., Mertens, A., & Schlick, C. M. (2016, March). Human-centered design of assistance systems for production planning and control: The role of the human in Industry 4.0. In 2016 IEEE International Conference on Industrial Technology (ICIT) (pp. 2099-2104). IEEE.
Peruzzini, M., Pellicciari, M., & Gadaleta, M. (2019). A comparative study on computer-integrated set-ups to design human-centred manufacturing systems. Robotics and Computer-Integrated Manufacturing, 55, 265-278.
Qian, C., Zhang, Y., Jiang, C., Pan, S., & Rong, Y. (2020). A real-time data-driven collaborative mechanism in fixed-position assembly systems for smart manufacturing. Robotics and Computer-Integrated Manufacturing, 61, 101841.
Rodríguez, I., Nottensteiner, K., Leidner, D., Durner, M., Stulp, F., & Albu-Schäffer, A. (2020). Pattern recognition for knowledge transfer in robotic assembly sequence planning. IEEE Robotics and Automation Letters, 5(2), 3666-3673.
Romero, D., Stahre, J., Wuest, T., Noran, O., Bernus, P., Fast-Berglund, Å., & Gorecky, D. (2016, October). Towards an operator 4.0 typology: a human-centric perspective on the fourth industrial revolution technologies. In proceedings of the international conference on computers and industrial engineering (CIE46), Tianjin, China (pp. 29-31).
Samant, A. P., Warhade, K., & Gunale, K. (2021, September). Pedestrian Intent Detection using Skeleton-based Prediction for Road Safety. In 2021 2nd International Conference on Advances in Computing, Communication, Embedded and Secure Systems (ACCESS) (pp. 238-242). IEEE.
Sgarbossa, F., Grosse, E. H., Neumann, W. P., Battini, D., & Glock, C. H. (2020). Human factors in production and logistics systems of the future. Annual Reviews in Control, 49, 295-305.
Shinde, S., Kothari, A., & Gupta, V. (2018). YOLO based human action recognition and localization. Procedia computer science, 133, 831-838.
Tong, J., Li, J., Zhang, M., & Zhang, B. (2022). Action Localization Using 2D-CNN and 3D-CNN Collaboration. IEEE Access, 10, 77658-77667.
Vinciarelli, A., Esposito, A., André, E., Bonin, F., Chetouani, M., Cohn, J. F., ... & Salah, A. A. (2015). Open challenges in modelling, analysis and synthesis of human behaviour in human–human and human–machine interactions. Cognitive Computation, 7, 397-413.
Wang, K. J., & Santoso, D. (2022). A smart operator advice model by deep learning for motion recognition in human–robot coexisting assembly line. The International Journal of Advanced Manufacturing Technology, 1-20.
Wang, K. J., & Yan, Y. J. (2021). A Smart Operator Assistance System Using Deep Learning for Angle Measurement. IEEE Transactions on Instrumentation and Measurement, 70, 1-14.
Yan, J., & Wang, Z. (2022). YOLO V3+ VGG16-based automatic operations monitoring and analysis in a manufacturing workshop under Industry 4.0. Journal of Manufacturing Systems, 63, 134-142.
Yang, J., Liu, F., Dong, Y., Cao, Y., & Cao, Y. (2022). Multiple-objective optimization of a reconfigurable assembly system via equipment selection and sequence planning. Computers & Industrial Engineering, 172, 108519.
Yoon, Y., Hwang, H., Choi, Y., Joo, M., Oh, H., Park, I., ... & Hwang, J. H. (2019). Analyzing basketball movements and pass relationships using realtime object tracking techniques based on deep learning. IEEE Access, 7, 56564-56576.