近年來，混合實境技術迅速發展，帶來比擴增實境更出色的視覺化體驗。然而，由於現存MR應用大多需要預先將模型等資料設置在現場環境中，不適合應用在變動較大的環境中，像是施工工地等人員、機具、物料不斷變化的情境中，導致MR技術的使用範圍和情境受限。因此，本研究利用混合實境(Mixed Reality，MR)結合AI影像辨識技術，應用在變動較大的施工現場中。施工現場中的機具檢核主要還是仰賴傳統紙本的方式，因此過程較為複雜而且需要具備相關經驗，因此本研究將藉由上述技術來實作施工機具安全檢核系統。本研究採用無須手持的頭戴式MR裝置，透過裝置上的攝影機擷取現場即時畫面，透過網路將其傳輸到遠端電腦進行影像辨識，偵測眼前的機具並辨識結果回傳，藉由MR虛實整合的特性將辨識的機具模型自動化顯示到檢核人員眼前，以立體機具模型取代傳統紙本表單，將需要檢核的部件以閃爍顯示，工檢核人員與實際機具比對，快速查找對應檢查點，並附上輔助圖文來以供參考，輔助檢核人員藉由虛擬手勢與機具模型互動，提升機具安全檢查之效率並降低錯誤率，實現自動化與視覺化之施工機具安全檢核系統。

In recent years, mixed reality (MR) technology has experienced rapid development, delivering a superior visual experience compared to augmented reality. However, due to the prevailing requirement of pre-positioning models and related data within the on-site environment, most existing MR applications are ill-suited for deployment in dynamically changing contexts, such as construction sites where personnel, machinery, and materials undergo constant fluctuations. This limitation constrains the applicability and scenarios of MR technology. Therefore, this study leverages mixed reality (MR) in conjunction with artificial intelligence (AI) image recognition techniques for application within highly dynamic construction sites.
The validation of construction machinery at the construction site predominantly relies on conventional paper-based methods, resulting in a complex and experience-dependent process. To address this, the present research proposes the implementation of a construction machinery safety verification system through the utilization of the aforementioned technologies. The study employs a hands-free, head-mounted MR device, which captures real-time on-site imagery through an integrated camera. This imagery is subsequently transmitted via a network to a remote computer for image recognition. The system detects machinery present in the field of view, performs recognition, and returns the identification results. Capitalizing on the augmented reality integration capabilities of MR, the recognized machinery models are automatically superimposed onto the visual field of verification personnel, replacing traditional paper-based forms. Components requiring verification are highlighted with flickering effects. Verification personnel can efficiently correlate the virtual model with the actual machinery, swiftly identifying corresponding inspection points. Supplementary graphics and textual information are also provided for reference. The system empowers verification personnel to interact with the virtual machinery model using gesture-based interactions, thereby enhancing the efficiency of machinery safety inspections and reducing error rates. This effectively achieves an automated and visualized construction machinery safety verification system.

[1] Mixed Reality Examples: 5 Uses in the Workplace,[Online],Available: https://virtualspeech.com/blog/mixed-reality-change-future-workplaces
[2] AR Machine Operator Hololens / AR instruction IoT ,[Online],Available: https://www.pdsol.com/product/vuforia-studio/
[3] HoloLens 2 hardware | Microsoft Docs. ,[Online],Available:https://doc s.microsoft.com/en-us/hololens/hololens2-hardware
[4] Milgram P., Kishino F.(1994). A Taxonomy of Mixed Reality Visual Displays.IEICE Transactions on Information and Systems,vol. E77-D.
[5] Yu k., Keisuke S., Masahide K., Kotaro T.,Akira I.,Ichiro W.,Akihiro N., Satoru M., Akira K.,(2023).Mixed reality for extraction of maxillary mesiodens. Maxillofacial Plastic and Reconstructive Surgery ,vol. 45, doi:10.1186/s40902-022-00370-6
[6] Liu S., Xie J., Wang X., Meng H.,(2023).Mixed Reality collaboration environment improves the efficiency of human-centered industrial system: A case study in the mining industry. Autom. Constr ,vol. 180, doi:10.1016/j.cie.2023.
[7] Rosenblatt F.,(1985).The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, vol. 65(6), doi:10.1037/h0042519
[8] Rumelhart D.E., Hinton G.E.,Williams R.J.,(1986). Learningrepresentations by back-propagating errors. Nature, vol. 323
[9] Yang X., Xie Y., Yang S.,(2023). Research on application of object detection based on yolov5 in construction site. International Conference on Advanced Computational Intelligence,vol. 15 ,doi:10.1109/ICACI58115.2023.10146151
[10] Lee K., Jeon C., Shin D.H., (2023).Small Tool Image Database and Object Detection Approach for Indoor Construction Site Safety. KSCE Journal of Civil Engineering volume,vol. 27 , doi:10.1007/s12205-023-1011-2
[11] 陳宛榆(2023).應用深度學習技術於施工現場影像之工項進度自動化偵測," 國立臺灣科技大學碩士學位論文
[12] 尹傳頌(2013).以營造工安意外案例探討營造安全衛生法規之缺失,” 朝陽科技大學碩士學位論文
[13] HoloLens 2 hardware | Microsoft Docs. ,[Online],Available: https://docs. microsoft.com/en-us/hololens/hololens2-hardware.
[14] Attach and detach the overhead strap ,[Online],Available:https:// learn.microsoft.com/en-us/hololens/hololens2-setup.
[15] HoloLens 2 interactive objects ,[Online],Available:https://learn.microsoft. com/zh-tw/windows/mixed-reality/design/interactable-object.
[16] HoloLens 2 Button ,[Online],Available:https://learn.microsoft.com/zh-cn/windows/mixed-reality/design/button.
[17] ultralytics/yolov5 ,[Online],Available:https://github.com/ultralytics/yolov5.
[18] Alberta Construction Image Dataset ,[Online],Available:https://www.acidb.ca/.
[19] Xiao B., Kang S., (2021).Development of an image data set of construction machines for deep learning object detection. Journal of Computing in Civil Engineering,vol. 35(2) , doi:10.1061/(ASCE)CP.1943-5487.0000945
[20] WebRTC API - Web APIs | MDN,[Online],Available: https://developer. mozilla.org/zh-TW/docs/Web/API/WebRTC_API.
[21] Locatable camera - Mixed Reality | Microsoft Docs,[Online],Available: https://docs.microsoft.com/en-us/windows/mixed-reality/develop/platform-capabilities-and-apis/locatable-camera.
[22] microsoft/MixedReality-WebRTC: MixedReality-WebRTC is a collection of components to help mixed reality app developers integrate audio and video real-time communication into their application and improve their collaborative experience ,[Online],Available:https://github.com/microsoft/MixedReality-WebRTC.