Li Guanglin ; Li Yifeng ; Ye Zhichao ; Zhang Qihang ; Kong Tao ; Cui Zhaopeng ; Zhang Guofeng

Abstract

Empowering autonomous agents with 3D understanding for daily objects is agrand challenge in robotics applications. When exploring in an unknownenvironment, existing methods for object pose estimation are still notsatisfactory due to the diversity of object shapes. In this paper, we propose anovel framework for category-level object shape and pose estimation from asingle RGB-D image. To handle the intra-category variation, we adopt a semanticprimitive representation that encodes diverse shapes into a unified latentspace, which is the key to establish reliable correspondences between observedpoint clouds and estimated shapes. Then, by using a SIM(3)-invariant shapedescriptor, we gracefully decouple the shape and pose of an object, thussupporting latent shape optimization of target objects in arbitrary poses.Extensive experiments show that the proposed method achieves SOTA poseestimation performance and better generalization in the real-world dataset.Code and video are available at https://zju3dv.github.io/gCasp.