Xiaoyang Wu Daniel DeTone Duncan Frost Tianwei Shen Chris Xie Nan Yang Jakob Engel Richard Newcombe Hengshuang Zhao Julian Straub

Abstract

In this paper, we question whether we have a reliable self-supervised pointcloud model that can be used for diverse 3D tasks via simple linear probing,even with limited data and minimal computation. We find that existing 3Dself-supervised learning approaches fall short when evaluated on representationquality through linear probing. We hypothesize that this is due to what we termthe "geometric shortcut", which causes representations to collapse to low-levelspatial features. This challenge is unique to 3D and arises from the sparsenature of point cloud data. We address it through two key strategies: obscuringspatial information and enhancing the reliance on input features, ultimatelycomposing a Sonata of 140k point clouds through self-distillation. Sonata issimple and intuitive, yet its learned representations are strong and reliable:zero-shot visualizations demonstrate semantic grouping, alongside strongspatial reasoning through nearest-neighbor relationships. Sonata demonstratesexceptional parameter and data efficiency, tripling linear probing accuracy(from 21.8% to 72.5%) on ScanNet and nearly doubling performance with only 1%of the data compared to previous approaches. Full fine-tuning further advancesSOTA across both 3D indoor and outdoor perception tasks.