Wang Changshuo ; Wu Meiqing ; Lam Siew-Kei ; Ning Xin ; Yu Shangshu ; Wang Ruiping ; Li Weijun ; Srikanthan Thambipillai

Abstract

Despite the significant advancements in pre-training methods for point cloudunderstanding, directly capturing intricate shape information from irregularpoint clouds without reliance on external data remains a formidable challenge.To address this problem, we propose GPSFormer, an innovative Global Perceptionand Local Structure Fitting-based Transformer, which learns detailed shapeinformation from point clouds with remarkable precision. The core of GPSFormeris the Global Perception Module (GPM) and the Local Structure FittingConvolution (LSFConv). Specifically, GPM utilizes Adaptive Deformable GraphConvolution (ADGConv) to identify short-range dependencies among similarfeatures in the feature space and employs Multi-Head Attention (MHA) to learnlong-range dependencies across all positions within the feature space,ultimately enabling flexible learning of contextual representations. Inspiredby Taylor series, we design LSFConv, which learns both low-order fundamentaland high-order refinement information from explicitly encoded local geometricstructures. Integrating the GPM and LSFConv as fundamental components, weconstruct GPSFormer, a cutting-edge Transformer that effectively capturesglobal and local structures of point clouds. Extensive experiments validateGPSFormer's effectiveness in three point cloud tasks: shape classification,part segmentation, and few-shot learning. The code of GPSFormer is available at\url{https://github.com/changshuowang/GPSFormer}.