Zhou Mu ; Stoffl Lucas ; Mathis Mackenzie Weygandt ; Mathis Alexander

Abstract

Frequent interactions between individuals are a fundamental challenge forpose estimation algorithms. Current pipelines either use an object detectortogether with a pose estimator (top-down approach), or localize all body partsfirst and then link them to predict the pose of individuals (bottom-up). Yet,when individuals closely interact, top-down methods are ill-defined due tooverlapping individuals, and bottom-up methods often falsely infer connectionsto distant bodyparts. Thus, we propose a novel pipeline called bottom-upconditioned top-down pose estimation (BUCTD) that combines the strengths ofbottom-up and top-down methods. Specifically, we propose to use a bottom-upmodel as the detector, which in addition to an estimated bounding box providesa pose proposal that is fed as condition to an attention-based top-down model.We demonstrate the performance and efficiency of our approach on animal andhuman pose estimation benchmarks. On CrowdPose and OCHuman, we outperformprevious state-of-the-art models by a significant margin. We achieve 78.5 AP onCrowdPose and 48.5 AP on OCHuman, an improvement of 8.6% and 7.8% over theprior art, respectively. Furthermore, we show that our method strongly improvesthe performance on multi-animal benchmarks involving fish and monkeys. The codeis available at https://github.com/amathislab/BUCTD