Engelmann Francis ; Bokeloh Martin ; Fathi Alireza ; Leibe Bastian ; Nießner Matthias

Abstract

We present 3D-MPA, a method for instance segmentation on 3D point clouds.Given an input point cloud, we propose an object-centric approach where eachpoint votes for its object center. We sample object proposals from thepredicted object centers. Then, we learn proposal features from grouped pointfeatures that voted for the same object center. A graph convolutional networkintroduces inter-proposal relations, providing higher-level feature learning inaddition to the lower-level point features. Each proposal comprises a semanticlabel, a set of associated points over which we define a foreground-backgroundmask, an objectness score and aggregation features. Previous works usuallyperform non-maximum-suppression (NMS) over proposals to obtain the final objectdetections or semantic instances. However, NMS can discard potentially correctpredictions. Instead, our approach keeps all proposals and groups them togetherbased on the learned aggregation features. We show that grouping proposalsimproves over NMS and outperforms previous state-of-the-art methods on thetasks of 3D object detection and semantic instance segmentation on theScanNetV2 benchmark and the S3DIS dataset.