近年來，影像處理在解決醫療、工業、安全、汽車應用等領域發揮著關鍵作用。然而，大多數影像處理系統是基於具有大量運算與儲存資源的計算機系統所開發。此類系統更適用於影像處理應用程式的執行，而可能無法滿足對即時性有要求而運算資源相對缺乏的應用。本論文開發基於嵌入式平台之即時影像處理系統，使其擷取影像並進行數位影像訊號處理運算，對於工業應用具有重要意義。本論文在嵌入式平台中設計與實現基於移動工業用影像擷取介面 (Mobile Industry Processor Ineterface ; MIPI)所制定的CSI (Camera Serial Interface)影像傳輸協議。本論文設計的影像擷取介面可直接讀取透過外部設備初始化後的MIPI D-PHY/C-PHY相機所傳輸的原始影像資料，擷取後的原始影像資料則在嵌入式平台中進行數位訊號處理相關運算。本論文設計的系統能以嵌入式平台實現影像擷取卡的功能，使相機不受嵌入式平台的限制，直接接收影像資料。透過多工平行運算開發平台來加速影像擷取及數位訊號處理，在共用記憶體上直接進行讀取及運算，得以實現不受嵌入式平台限制、更快的影像處理速度及節省設備成本之目的。本篇論文研發過程包含應用軟體開發及最佳化、Linux系統架構的研究與驅動程式開發、MIPI D-PHY/C-PHY傳輸協定的研究與軟硬體整合、及研發完成後的驗證測試。

Image processing has played an essential role in industrial applications such as medical, industrial control, security, and automotive applications. However, most image processing systems are developed based on computer systems with large computing and storage resources. Such systems are more suitable for the execution of image processing applications, but may not be able to meet the requirements of real-time requirements with limited resources. This thesis presents the design and development of a real-time image processing system based on an embedded platform, so that it can capture images and perform digital image signal processing operations, which is of great significance for industrial applications. This thesis designs and implements the CSI (Camera Serial Interface) image transmission protocol based on the Mobile Industry Processor Ineterface (MIPI) in the embedded platform. The image capture interface designed in this theis can directly read the raw image data transmitted by the MIPI D-PHY/C-PHY camera that is initialized by the external device. The captured raw image data is processed in the embedded platform for digital signal processing related operations. The system designed in this thesis can realize the function of image capture card with embedded platform, so that the camera is not restricted by the embedded platform and can directly receive image data. This system directly reads and calculates raw image data in the way of shared memory, which can achieve the purpose of not being limited by embedded platform, faster image processing speed and saving equipment cost. Theis thesis presents the development and optimization of application software, the study and development of Linux system architecture and device driver, the study and development of MIPI D-PHY/C-PHY transmission protocol, and verification testing for the designed software.

[1] Ashraf Takla, C.K. Lee "MIPI C-PHYSM/D-PHYSM Dual Mode Subsystem Performance & Use Cases", MIPI Alliance Developers Conference 2017. Available: https://www.mipi.org/sites/default/files/Bangalore-Mixel-C-PHY-D-PHY-Combo-Implementation.pdf
[2] NVIDIA Corporation. Jetson_AGX_Xavier_Series_Camera_Module_Hardware_Design_Guide_DG-09364-001_v1.2, May 2020. Available: https://developer.nvidia.com/embedded/downloads
[3] The Linux Kernel’s documentation, 2016. Available: https://www.kernel.org/doc/html/v4.9/index.html
[4] Gstreamer Introduction. Available: https://gstreamer.freedesktop.org/documentation/plugin-development/introduction/index.html?gi-language=c
[5] Bayer Format Introduction. Available: https://blog.csdn.net/qq_42261630/article/details/102922737
[6] Color Format Introduction. Available:
https://blog.csdn.net/tao475824827/article/details/119141811
[7] Bayer Filter Introduction. Available: https://en.wikipedia.org/wiki/Bayer_filter
[8] D. Jinghong, D. Yaling and L. Kun, "Development of Image Processing System Based on DSP and FPGA," 2007 8th International Conference on Electronic Measurement and Instruments, 2007, pp. 2-791-2-794, doi: 10.1109/ICEMI.2007.4350799.
[9] M. V. G. Rao, P. R. Kumar and A. M. Prasad, "Implementation of real time image processing system with FPGA and DSP," 2016 International Conference on Microelectronics, Computing and Communications (MicroCom), 2016, pp. 1-4, doi: 10.1109/MicroCom.2016.7522496.
[10] Zhanchao Wang, Min Huang, Guifeng Zhang, and Lulu Qian "The design of MIPI image processing based on FPGA", Proc. SPIE 11028, Optical Sensors 2019, 110282A (13 April 2019) Available : https://doi.org/10.1117/12.2521433
[11] F. Liu, L. Wang and Y. Yang, "A UHD MIPI CSI-2 image acquisition system based on FPGA," 2021 40th Chinese Control Conference (CCC), 2021, pp. 5668-5673, doi: 10.23919/CCC52363.2021.9550077.
[12] NVIDIA Corporation. Sensor Software Driver Programming Guide, July 2020.
Available: https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3243/index.html
[13] NVIDIA Corporation. P3326_A01_Concept_schematics, 2015. Available: https://developer.nvidia.com/embedded/downloads
[14] NVIDIA Corporation. ACCELERATED GSTREAMER USER GUIDE, March 2019.
Available: https://developer.download.nvidia.com/embedded/L4T/r32_Release_v1.0/Docs/Accelerated_GStreamer_User_Guide.pdf
[15] NVIDIA Corporation. Jetson_Xavier_TX2_Camera_Module_Driver_Guide_Release_JP_4.4, 2017. Available: https://developer.nvidia.com/embedded/downloads
[16] Sony Corporation. IMX290LQR-C DATASHEET, 2015.
Available: http://static6.arrow.com/aropdfconversion/c0c7efde6571c768020a72f59b226308b9669e45/sony_imx290lqr-c_datasheet.pdf