作業指引可以幫助資淺工作者遵循正確的作業程序，確保人機共存組裝線之作
業協調性。本研究針對 GPU 組裝線提出一種透過深度學習的智慧作業指引模型，
使用卷積神經網絡 YOLOv3 的物件檢測機制和決策樹分類器 CART 的動作識別機
制，作為模型核心。在此人機協作組裝線上，透過三個獨立的攝影機監控所有
物件（如：風扇、主機板、GPU、機器人、工作者頭部、工作者身體、螺絲鎖附
機、螺絲起子等）。動作檢測是由三個攝影機輸入目標物件的坐標和速度，並
執行三個平行且獨立的決策樹分類器來完成，最終輸出作業建議。本研究透過
GPU 組裝過程完成模型的評估，F1-score 達到 0.96。此智慧模型有助於裝配過
程中，向資淺工作者提供即時有效的工作指導。

Operations advice system can help junior workers to follow standard operations
procedure, which is critical to a human-robot coexisted assembly line to guarantee
harmonious tasks. In this study, a smart operator advice model by deep-learning is
proposed for a GPU assembly line. Two mechanisms are built as the core of the model,
which is an object detection mechanism using convolutional neural network YOLOv3
and a motion recognition mechanism using decision tree classifier CART. The object
detection is conducted by 3 parallel and independent cameras monitoring all the objects
(e.g., fan, motherboard, GPU, robot, head, body, screwing machine, screwdriver) in the
assembly line. The motion detection is done by three parallel and independent decision
tree classifiers by the three cameras where the input is the coordinates and speed of
objects. The final output isthe task advice given by the proposed operator advice model.
The evaluation of the model is done through a case study of a GPU final assembly
process. F1-score shows a value of 0.96 by the model. The smart model facilitates
informative instruction to the junior operator during the assembly process in real time.

Adnan, M. A., Sulaiman, N., Zainuddin, N. I., & Besar, T. B. H. T. (2013). Vehicle
speed measurement technique using various speed detection instrumentation.
BEIAC 2013 - 2013 IEEE Business Engineering and Industrial Applications
Colloquium, 668±672. https://doi.org/10.1109/BEIAC.2013.6560214
Ajit, A., Acharya, K., & Samanta, A. (2020). A Review of Convolutional Neural
Networks. International Conference on Emerging Trends in Information
Technology and Engineering, Ic-ETITE 2020, 1±5. https://doi.org/10.1109/icETITE47903.2020.049
Babaee, M., Dinh, D. T., & Rigoll, G. (2018). A deep convolutional neural network for
video sequence background subtraction. Pattern Recognition, 76, 635±649.
https://doi.org/10.1016/j.patcog.2017.09.040
Cao, Z., Simon, T., Wei, S. E., & Sheikh, Y. (2017). Realtime multi-person 2D pose
estimation using part affinity fields. Proceedings - 30th IEEE Conference on
Computer Vision and Pattern Recognition, CVPR 2017, 2017-Janua, 1302±1310.
https://doi.org/10.1109/CVPR.2017.143
Chen, C., Liu, M.-Y., Tuzel, O., & Xiao, J. (2017). R-CNN for Small Object Detection.
In S.-H. Lai, V. Lepetit, K. Nishino, & Y. Sato (Eds.), Computer Vision -- ACCV
2016 (pp. 214±230). Springer International Publishing.
Cristani, M., Raghavendra, R., Del Bue, A., & Murino, V. (2013). Human behavior
analysis in video surveillance: A Social Signal Processing perspective.
Neurocomputing, 100, 86±97. https://doi.org/10.1016/j.neucom.2011.12.038
Dey, A. (2016). Machine Learning Algorithms: A Review. International Journal of
Computer Science and Information Technologies, 7(3), 1174±1179.
www.ijcsit.com
Girshick, R. (2015). Fast R-CNN. Proceedings of the IEEE International Conference
on Computer Vision, 2015 Inter, 1440±1448.
https://doi.org/10.1109/ICCV.2015.169
Guo, K., Ishwar, P., & Konrad, J. (2013). Action recognition from video using feature
covariance matrices. IEEE Transactions on Image Processing, 22(6), 2479±2494.
https://doi.org/10.1109/TIP.2013.2252622
Hinrichsen, D., Riediger, D., & Unrau, A. (2016). Assistance systems in manual
assembly. Production Engineering and Management. 6th International 38
Conference, December, 3±14.
https://www.researchgate.net/publication/311535944_Assistance_Systems_in_M
anual_Assembly
Hinrichsen, S., & Bendzioch, S. (2019). How Digital Assistance Systems Improve
Work Productivity in Assembly. In International Conference on Applied Human
Factors and Ergonomics (pp. 332±342). Springer. https://doi.org/10.1007/978-3-
319-94334-3_33
Idrees, H., Zamir, A. R., Jiang, Y. G., Gorban, A., Laptev, I., Sukthankar, R., & Shah,
07KH7+8026FKDOOHQJHRQDFWLRQUHFRJQLWLRQIRUYLGHRV³LQWKHZLOG´
Computer Vision and Image Understanding, 155, 1±23.
https://doi.org/10.1016/j.cviu.2016.10.018
Janidarmian, M., Fekr, A. R., Radecka, K., & Zilic, Z. (2017). A comprehensive
analysis on wearable acceleration sensors in human activity recognition. Sensors
(Switzerland), 17(3). https://doi.org/10.3390/s17030529
Jianping, W., Zhaobin, L., Jinxiang, L., Caidong, G., Maoxin, S., & Fangyong, T.
(2009). An algorithm for automatic vehicle speed detection using video camera.
Proceedings of 2009 4th International Conference on Computer Science and
Education, ICCSE 2009, 193±196. https://doi.org/10.1109/ICCSE.2009.5228496
Kamate, S., & Yilmazer, N. (2015). Application of Object Detection and Tracking
Techniques for Unmanned Aerial Vehicles. Procedia Computer Science, 61, 436±
441. https://doi.org/10.1016/j.procs.2015.09.183
.UDZF]\N % :RĨQLDN 0 6FKDHIHU G. (2014). Cost-sensitive decision tree
ensembles for effective imbalanced classification. Applied Soft Computing
Journal, 14(PART C), 554±562. https://doi.org/10.1016/j.asoc.2013.08.014
Loh, W. (2011). Classification and regression trees. Wiley Interdisciplinary Reviews:
Data Mining and Knowledge Discovery, 1.
Poppe, R. (2010). A survey on vision-based human action recognition. Image and
Vision Computing, 28(6), 976±990. https://doi.org/10.1016/j.imavis.2009.11.014
Qi, S., Wu, X., Chen, W. H., Liu, J., Zhang, J., & Wang, J. (2020). sEMG-based
recognition of composite motion with convolutional neural network. Sensors and
Actuators, A: Physical, 311, 112046. https://doi.org/10.1016/j.sna.2020.112046
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once:
Unified, real-time object detection. Proceedings of the IEEE Computer Society 39
Conference on Computer Vision and Pattern Recognition, 2016-Decem, 779±788.
https://doi.org/10.1109/CVPR.2016.91
Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, faster, stronger. Proceedings -
30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017,
2017-Janua, 6517±6525. https://doi.org/10.1109/CVPR.2017.690
Redmon, J., & Farhadi, A. (2018). YOLOv3: An incremental improvement. ArXiv.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy,
A., Khosla, A., Bernstein, M., Berg, A. C., & Fei-Fei, L. (2015). ImageNet Large
Scale Visual Recognition Challenge. International Journal of Computer Vision,
115(3), 211±252. https://doi.org/10.1007/s11263-015-0816-y
Shinde, S., Kothari, A., & Gupta, V. (2018). YOLO based Human Action Recognition
and Localization. Procedia Computer Science, 133(2018), 831±838.
https://doi.org/10.1016/j.procs.2018.07.112
Vinciarelli, A., Esposito, A., André, E., Bonin, F., Chetouani, M., Cohn, J. F., Cristani,
M., Fuhrmann, F., Gilmartin, E., Hammal, Z., Heylen, D., Kaiser, R.,
Koutsombogera, M., Potamianos, A., Renals, S., Riccardi, G., & Salah, A. A.
(2015). Open Challenges in Modelling, Analysis and Synthesis of Human
Behaviour in Human±Human and Human±Machine Interactions. Cognitive
Computation, 7(4), 397±413. https://doi.org/10.1007/s12559-015-9326-z
Wang, D., Khosla, A., Gargeya, R., Irshad, H., & Beck, A. H. (2016). Deep Learning
for Identifying Metastatic Breast Cancer. 1±6. http://arxiv.org/abs/1606.05718
Yao, G., Lei, T., & Zhong, J. (2019). A review of Convolutional-Neural-Network-based
action recognition. Pattern Recognition Letters, 118, 14±22.
https://doi.org/10.1016/j.patrec.2018.05.018
Zhao, Z. Q., Zheng, P., Xu, S. T., & Wu, X. (2018). Object detection with deep learning:
A review. ArXiv, 30(11), 3212±3232