Xu Lumin ; Jin Sheng ; Zeng Wang ; Liu Wentao ; Qian Chen ; Ouyang Wanli ; Luo Ping ; Wang Xiaogang

Abstract

Existing works on 2D pose estimation mainly focus on a certain category, e.g.human, animal, and vehicle. However, there are lots of application scenariosthat require detecting the poses/keypoints of the unseen class of objects. Inthis paper, we introduce the task of Category-Agnostic Pose Estimation (CAPE),which aims to create a pose estimation model capable of detecting the pose ofany class of object given only a few samples with keypoint definition. Toachieve this goal, we formulate the pose estimation problem as a keypointmatching problem and design a novel CAPE framework, termed POse MatchingNetwork (POMNet). A transformer-based Keypoint Interaction Module (KIM) isproposed to capture both the interactions among different keypoints and therelationship between the support and query images. We also introduceMulti-category Pose (MP-100) dataset, which is a 2D pose dataset of 100 objectcategories containing over 20K instances and is well-designed for developingCAPE algorithms. Experiments show that our method outperforms other baselineapproaches by a large margin. Codes and data are available athttps://github.com/luminxu/Pose-for-Everything.