本文介紹了一種使用學習車輛 (LAV) 系統進行自動駕駛的新方法，利用鳥瞰圖 (BEV) 圖像作為感知的主要來源，並根據信息訓練周圍車輛以生成其運動預測，使它們能夠預測並對附近車輛的潛在動作做出反應。此外，我們建議對 LAV 模型進行改進，通過在模型中添加一個控制分支來進行直接預測控制，從而解決高曲率道路和陡峭道路等關鍵道路條件，而無需 PID 控製過程。這項研究的最終目標是開發一種先進的 AI 智能代理，能夠以無與倫比的效率和可靠性在複雜和動態的環境中導航，使用CARLA 模擬器作為主要環境。

This thesis presents a novel approach to autonomous driving using the
Learning From Vehicle (LAV) system which utilizes bird’s eye view (BEV)
images as the primary perception source and trains on surrounding vehicle
information to generate predictions of their movements, enabling it to an-
ticipate and respond to potential actions of nearby vehicles. Additionally,
we propose an improvement to the LAV model to tackle critical road con-
ditions such as high curvature roads and steep roads, by adding a control
branch to the model to predict control directly without the need for a PID
control process. The Trajectory Guided Control Prediction (TCP) approach
also utilizes RGB camera data and incorporates elements of Reinforcement
Learning to train an intelligent model. The ultimate goal of this research is
to develop a cutting-edge AI-powered agent that can navigate complex and
dynamic environments with unparalleled efficiency and reliability using
the CARLA simulator as the main environment.

[1] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “CARLA: An open urban driving
simulator,” in Proceedings of the 1st Annual Conference on Robot Learning, 2017, pp. 1–16.
[2] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization
algorithms,” arXiv preprint arXiv:1707.06347, 2017.
[3] D. Frossard, S. Da Suo, S. Casas, J. Tu, and R. Urtasun, “Strobe: Streaming object detection from
lidar packets,” in Conference on Robot Learning. PMLR, 2021, pp. 1174–1183.
[4] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for
object detection from point clouds,” in Proceedings of the IEEE/CVF conference on computer vision
and pattern recognition, 2019, pp. 12 697–12 705.
[5] T. Yin, X. Zhou, and P. Krahenbuhl, “Center-based 3d object detection and tracking,” in Proceedings
of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 11 784–11 793.
[6] J. Levinson, M. Montemerlo, and S. Thrun, “Map-based precision vehicle localization in urban envi-
ronments.” in Robotics: science and systems, vol. 4, no. Citeseer. Atlanta, GA, USA, 2007, p. 1.
[7] W. Luo, B. Yang, and R. Urtasun, “Fast and furious: Real time end-to-end 3d detection, tracking
and motion forecasting with a single convolutional net,” in Proceedings of the IEEE conference on
Computer Vision and Pattern Recognition, 2018, pp. 3569–3577.
[8] W. Zeng, W. Luo, S. Suo, A. Sadat, B. Yang, S. Casas, and R. Urtasun, “End-to-end interpretable
neural motion planner,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition, 2019, pp. 8660–8669.
[9] S. Casas, W. Luo, and R. Urtasun, “Intentnet: Learning to predict intention from raw sensor data,” in
Conference on Robot Learning. PMLR, 2018, pp. 947–956.
[10] Y. Chai, B. Sapp, M. Bansal, and D. Anguelov, “Multipath: Multiple probabilistic anchor trajectory
hypotheses for behavior prediction,” arXiv preprint arXiv:1910.05449, 2019.
[11] A. Cui, S. Casas, A. Sadat, R. Liao, and R. Urtasun, “Lookout: Diverse multi-future prediction and
planning for self-driving,” in Proceedings of the IEEE/CVF International Conference on Computer
Vision, 2021, pp. 16 107–16 116.
[12] A. Kamenev, L. Wang, O. B. Bohan, I. Kulkarni, B. Kartal, A. Molchanov, S. Birchfield, D. Nistér, and
N. Smolyanskiy, “Predictionnet: Real-time joint probabilistic traffic prediction for planning, control,
and simulation,” in 2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022,
pp. 8936–8942.
[13] D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” Advances in neural
information processing systems, vol. 1, 1988.
[14] F. Codevilla, M. Müller, A. López, V. Koltun, and A. Dosovitskiy, “End-to-end driving via condi-
tional imitation learning,” in 2018 IEEE international conference on robotics and automation (ICRA).
IEEE, 2018, pp. 4693–4700.
[15] A. Kendall, J. Hawke, D. Janz, P. Mazur, D. Reda, J.-M. Allen, V.-D. Lam, A. Bewley, and A. Shah,
“Learning to drive in a day,” in 2019 International Conference on Robotics and Automation (ICRA).
IEEE, 2019, pp. 8248–8254.
[16] M. Toromanoff, E. Wirbel, and F. Moutarde, “End-to-end model-free reinforcement learning for urban
driving using implicit affordances,” in Proceedings of the IEEE/CVF conference on computer vision
and pattern recognition, 2020, pp. 7153–7162.
[17] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Re-
liable reinforcement learning implementations,” Journal of Machine Learning Research, 2021.
[18] D. Chen, V. Koltun, and P. Krähenbühl, “Learning to drive from a world on rails,” in Proceedings of
the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15 590–15 599.
[19] D. Chen and P. Krähenbühl, “Learning from all vehicles,” in Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 17 222–17 231.
[20] S. Vora, A. H. Lang, B. Helou, and O. Beijbom, “Pointpainting: Sequential fusion for 3d object
detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition,
2020, pp. 4604–4612.
[21] P. Wu, X. Jia, L. Chen, J. Yan, H. Li, and Y. Qiao, “Trajectory-guided control prediction for end-to-end
autonomous driving: A simple yet strong baseline,” arXiv preprint arXiv:2206.08129, 2022.