Wu Dong ; Liao Manwen ; Zhang Weitian ; Wang Xinggang ; Bai Xiang ; Cheng Wenqing ; Liu Wenyu

Abstract

A panoptic driving perception system is an essential part of autonomousdriving. A high-precision and real-time perception system can assist thevehicle in making the reasonable decision while driving. We present a panopticdriving perception network (YOLOP) to perform traffic object detection,drivable area segmentation and lane detection simultaneously. It is composed ofone encoder for feature extraction and three decoders to handle the specifictasks. Our model performs extremely well on the challenging BDD100K dataset,achieving state-of-the-art on all three tasks in terms of accuracy and speed.Besides, we verify the effectiveness of our multi-task learning model for jointtraining via ablative studies. To our best knowledge, this is the first workthat can process these three visual perception tasks simultaneously inreal-time on an embedded device Jetson TX2(23 FPS) and maintain excellentaccuracy. To facilitate further research, the source codes and pre-trainedmodels are released at https://github.com/hustvl/YOLOP.