Aygün Mehmet ; Ošep Aljoša ; Weber Mark ; Maximov Maxim ; Stachniss Cyrill ; Behley Jens ; Leal-Taixé Laura

Abstract

Temporal semantic scene understanding is critical for self-driving cars orrobots operating in dynamic environments. In this paper, we propose 4D panopticLiDAR segmentation to assign a semantic class and a temporally-consistentinstance ID to a sequence of 3D points. To this end, we present an approach anda point-centric evaluation metric. Our approach determines a semantic class forevery point while modeling object instances as probability distributions in the4D spatio-temporal domain. We process multiple point clouds in parallel andresolve point-to-instance associations, effectively alleviating the need forexplicit temporal data association. Inspired by recent advances in benchmarkingof multi-object tracking, we propose to adopt a new evaluation metric thatseparates the semantic and point-to-instance association aspects of the task.With this work, we aim at paving the road for future developments of temporalLiDAR panoptic perception.