Yu Fisher ; Chen Haofeng ; Wang Xin ; Xian Wenqi ; Chen Yingying ; Liu Fangchen ; Madhavan Vashisht ; Darrell Trevor

Abstract

Datasets drive vision progress, yet existing driving datasets areimpoverished in terms of visual content and supported tasks to study multitasklearning for autonomous driving. Researchers are usually constrained to study asmall set of problems on one dataset, while real-world computer visionapplications require performing tasks of various complexities. We constructBDD100K, the largest driving video dataset with 100K videos and 10 tasks toevaluate the exciting progress of image recognition algorithms on autonomousdriving. The dataset possesses geographic, environmental, and weatherdiversity, which is useful for training models that are less likely to besurprised by new conditions. Based on this diverse dataset, we build abenchmark for heterogeneous multitask learning and study how to solve the taskstogether. Our experiments show that special training strategies are needed forexisting models to perform such heterogeneous tasks. BDD100K opens the door forfuture studies in this important venue.