Wu Penghao ; Jia Xiaosong ; Chen Li ; Yan Junchi ; Li Hongyang ; Qiao Yu

Abstract

Current end-to-end autonomous driving methods either run a controller basedon a planned trajectory or perform control prediction directly, which havespanned two separately studied lines of research. Seeing their potential mutualbenefits to each other, this paper takes the initiative to explore thecombination of these two well-developed worlds. Specifically, our integratedapproach has two branches for trajectory planning and direct control,respectively. The trajectory branch predicts the future trajectory, while thecontrol branch involves a novel multi-step prediction scheme such that therelationship between current actions and future states can be reasoned. The twobranches are connected so that the control branch receives correspondingguidance from the trajectory branch at each time step. The outputs from twobranches are then fused to achieve complementary advantages. Our results areevaluated in the closed-loop urban driving setting with challenging scenariosusing the CARLA simulator. Even with a monocular camera input, the proposedapproach ranks first on the official CARLA Leaderboard, outperforming othercomplex candidates with multiple sensors or fusion mechanisms by a largemargin. The source code is publicly available athttps://github.com/OpenPerceptionX/TCP