行車數據加值應用廣泛，目前除了自動駕駛外，車險場域、耗油分析、公共運輸司機派遣、計程車調度、行車路徑最佳化規劃等。當前如要取得汽車的行車資訊，以安裝行車記錄器紀錄行車訊息最為便利。為了能更好地實現於各式場景，本論文提出一項基於行車記錄器影片進行車道線偵測、追蹤與分類之系統。

本論文先對輸入影片進行鏡頭扭曲校正（Lens Distortion）還原影像，再利用Faster R-CNN模型辨識車前蓋以及LaneAF模型偵測車道線。由於經由LaneAF模型輸出的車道線會有一條被辨識成二條之現象，因此本論文先透過演算法將其合併後，再藉由追蹤器（Tracker）完成前後幀的車道線之追蹤。後取最靠近自身之左右兩條車道線計算消失點並進行透視變換取得鳥瞰圖，再將車道線逐一輸入至Resnet101模型取得車道線分類結果，最後以判斷駕駛是否有違規切換車道作為應用。就平均而言，本系統處理一部10秒，共100幀，解析度為1920*1080像素之影片約須花費95秒。

在進行車道線研究時我們發現，目前尚未有提供經透視變換後之車道線分類資料集。故本論文提供一種生成方法，藉由此方法可取得經透視變換後之車道線影像，並透過影像處理方法增加樣本的多樣性，以利後續研究。

Driving data are widely used in multiple scenarios, such as autonomous driving, usage-based insurance, and even the dispatchment of public transportation, etc. currently, installing a dashcam is probably the most convenient way to obtain driving data. Therefore, we propose a lane detection, tracking, and classification system based on dashcam videos to process the data to be used.

First, lens distortion correction is performed on the dashcam videos to reduce lens distortion, the Faster R-CNN model is adopted to identify the hood, and the LaneAF model is used to detect lanes. Since the outputs by the LaneAF model could be divided into two lanes, we design an algorithm to combine them. Then, we design a tracker to track the lanes in videos. Next, we convert the image to a bird's-eye view by calculating the vanishing point and perform a perspective transformation. After that, obtain lane classification results through the Resnet101 model. Finally, we combine the above information to determine if a lane switching is legal not.

As for as we know currently there is no lane classification dataset after the perspective transformation. As a result, this paper provides a method to obtain the lane image using a perspective transformation matrix and increase the diversity of samples through image processing for future research.

[1] S. O.-R. A. V. S. Committee et al., “Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems,” SAE Standard J, vol. 3016, pp. 1–16, 2014.
[2] I. Hasan, S. Liao, J. Li, S. U. Akram, and L. Shao, “Generalizable pedestrian detection: The elephant in the room,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11328–11337, 2021.
[3] A. H. Khan, M. Munir, L. van Elst, and A. Dengel, “F2dnet: Fast focal detection network for pedestrian detection,” arXiv preprint arXiv:2203.02331, 2022.
[4] D. Tabernik and D. Skočaj, “Deep learning for large-scale traffic-sign detection and recognition,” IEEE transactions on intelligent transportation systems, vol. 21, no. 4, pp. 1427–1440, 2019.
[5] J. Chen, K. Jia, W. Chen, Z. Lv, and R. Zhang, “A real-time and high-precision method for small traffic-signs recognition,” Neural Computing and Applications, vol. 34, no. 3, pp. 2233–2245, 2022.
[6] J. Son, H. Yoo, S. Kim, and K. Sohn, “Real-time illumination invariant lane detection for lane departure warning system,” Expert Systems with Applications, vol. 42, no. 4, pp. 1816–1824, 2015.
[7] B. Reddy, Y.-H. Kim, S. Yun, C. Seo, and J. Jang, “Real-time driver drowsiness detection for embedded system using model compression of deep neural networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 121–128, 2017.
[8] “Improving global road safety : resolution /,” p. 9 p., Sep 2020. Issued in GAOR, 74th sess., Suppl. no. 49.
[9] “道 安 資 訊 網.” https://roadsafety.tw/AccAgeI26CauseOrder?type=èĆĞåŻăåĹĘæđŘ. Accessed on 2022
[10] S. Arumugam and R. Bhargavi, “A survey on driving behavior analysis in usage based insurance using big data,” Journal of Big Data, vol. 6, no. 1, pp. 1–21, 2019.
[11] J. Bordoff and P. Noel, “Pay-as-you-drive auto insurance,” in Issues of the Day, pp. 160–161, Routledge, 2010.
[12] B. Yu and A. K. Jain, “Lane boundary detection using a multiresolution hough transform,” in Proceedings of International Conference on Image Processing, vol. 2, pp. 748–751, IEEE, 1997.
[13] Y. Wang, D. Shen, and E. K. Teoh, “Lane detection using spline model,” Pattern Recognition Letters, vol. 21, no. 8, pp. 677–689, 2000.
[14] J. Kim and M. Lee, “Robust lane detection based on convolutional neural network and random sample consensus,” in International conference on neural information processing, pp. 454–461, Springer, 2014.
[15] L. Tabelini, R. Berriel, T. M. Paixao, C. Badue, A. F. De Souza, and T. OliveiraSantos, “Keep your eyes on the lane: Real-time attention-guided lane detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 294–302, 2021.
[16] T. Zheng, Y. Huang, Y. Liu, W. Tang, Z. Yang, D. Cai, and X. He, “Clrnet: Cross layer refinement network for lane detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 898–907, 2022.
[17] Z. Chen, Q. Liu, and C. Lian, “Pointlanenet: Efficient end-to-end cnns for accurate real-time lane detection,” in 2019 IEEE intelligent vehicles symposium (IV), pp. 2563–2568, IEEE, 2019.
[18] D. Jin, W. Park, S.-G. Jeong, H. Kwon, and C.-S. Kim, “Eigenlanes: Data-driven lane descriptors for structurally diverse lanes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17163–17171, 2022
[19] S. Lee, J. Kim, J. Shin Yoon, S. Shin, O. Bailo, N. Kim, T.-H. Lee, H. Seok Hong, S.-H. Han, and I. So Kweon, “Vpgnet: Vanishing point guided network for lane and road marking detection and recognition,” in Proceedings of the IEEE international conference on computer vision, pp. 1947–1955, 2017.
[20] A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. Emer, S. W. Keckler, and W. J. Dally, “Scnn: An accelerator for compressed- sparse convolutional neural networks,” ACM SIGARCH computer architecture news, vol. 45, no. 2, pp. 27–40, 2017.
[21] T. Zheng, H. Fang, Y. Zhang, W. Tang, Z. Yang, H. Liu, and D. Cai, “Resa: Recurrent feature-shift aggregator for lane detection,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 3547–3554, 2021.
[22] J. Xie, J. Han, D. Qi, F. Chen, K. Huang, and J. Shuai, “Lane detection with position embedding,” arXiv preprint arXiv:2203.12301, 2022.
[23] V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481–2495, 2017.
[24] Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7291–7299, 2017.
[25] Y. Ko, Y. Lee, S. Azam, F. Munir, M. Jeon, and W. Pedrycz, “Key points estimation and point instance segmentation approach for lane detection,” IEEE Transactions on Intelligent Transportation Systems, 2021.
[26] Z. Qu, H. Jin, Y. Zhou, Z. Yang, and W. Zhang, “Focus on local: Detecting lane marker from bottom up via key point,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14122–14130, 2021.
[27] H. Abualsaud, S. Liu, D. B. Lu, K. Situ, A. Rangesh, and M. M. Trivedi, “Laneaf: Robust multi-lane detection with affinity fields,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 7477–7484, 2021.
[28] T. M. Hoang, H. G. Hong, H. Vokhidov, and K. R. Park, “Road lane detection by discriminating dashed and solid road lanes using a visible light camera sensor,” Sensors, vol. 16, no. 8, p. 1313, 2016.
[29] M. B. de Paula and C. R. Jung, “Automatic detection and classification of road lane markings using onboard vehicular cameras,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 6, pp. 3160–3169, 2015.
[30] W. Song, Y. Yang, M. Fu, Y. Li, and M. Wang, “Lane detection and classification for forward collision warning system based on stereo vision,” IEEE Sensors Journal, vol. 18, no. 12, pp. 5151–5163, 2018.
[31] N. Khairdoost, S. S. Beauchemin, and M. A. Bauer, “Road lane detection and classification in urban and suburban areas based on cnns,” in VISIGRAPP,2021.
[32] F. Pizzati, M. Allodi, A. Barrera, and F. García, “Lane detection and classification using cascaded cnns,” in International Conference on Computer Aided Systems Theory, pp. 95–103, Springer, 2019.
[33] L. Zhang, F. Jiang, J. Yang, B. Kong, A. Hussain, M. Gogate, and K. Dashtipour, “Dnet-cnet: a novel cascaded deep network for real-time lane detection and classification,” Journal of Ambient Intelligence and Humanized Computing, pp. 1–16, 2022.
[34] L. Bravi, L. Kubin, S. Caprasecca, D. C. de Andrade, M. Simoncini, L. Taccari, and F. Sambo, “Detection of stop sign violations from dashcam data,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 5411–5420, 2022.
[35] M. G. Li, B. Jiang, Z. Che, X. Shi, M. Liu, Y. Meng, J. Ye, and Y. Liu, “Dbus: Human driving behavior understanding system,” in 2019 IEEE/CVFnternational Conference on Computer Vision Workshop (ICCVW), pp. 2436–2444, 2019.
[36] K. Bito, I. Siio, Y. Ishiguro, and K. Takeda, “Automatic generation of road trip summary video for reminiscence and entertainment using dashcam video,” in 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’21, (New York, NY, USA), p. 181–190, Association for Computing Machinery, 2021.
[37] D. C. Brown, “Decentering distortion of lenses,” Photogrammetric Engineering and Remote Sensing, 1966.
[38] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems, vol. 28, 2015.
[39] F. Yu, D. Wang, E. Shelhamer, and T. Darrell, “Deep layer aggregation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2403–2412, 2018.
[40] Y. Zhang, L. Zhu, W. Feng, H. Fu, M. Wang, Q. Li, C. Li, and S. Wang, “Vil100: A new dataset and a baseline model for video instance lane detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15681–15690, 2021.
[41] F. Yu, H. Chen, X. Wang, W. Xian, Y. Chen, F. Liu, V. Madhavan, and T. Darrell, “Bdd100k: A diverse driving dataset for heterogeneous multitask learning,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[42] X. Pan, J. Shi, P. Luo, X. Wang, and X. Tang, “Spatial as deep: Spatial cnn for traffic scene understanding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, 2018.
[43] H. Xu, S. Wang, X. Cai, W. Zhang, X. Liang, and Z. Li, “Curvelane-nas: Unifying lane-sensitive architecture search and adaptive point blending,” in European Conference on Computer Vision, pp. 689–704, Springer, 2020.