Zhu Yue ; Samet Nermin ; Picard David

Abstract

We present a benchmark for 3D human whole-body pose estimation, whichinvolves identifying accurate 3D keypoints on the entire human body, includingface, hands, body, and feet. Currently, the lack of a fully annotated andaccurate 3D whole-body dataset results in deep networks being trainedseparately on specific body parts, which are combined during inference. Or theyrely on pseudo-groundtruth provided by parametric body models which are not asaccurate as detection based methods. To overcome these issues, we introduce theHuman3.6M 3D WholeBody (H3WB) dataset, which provides whole-body annotationsfor the Human3.6M dataset using the COCO Wholebody layout. H3WB comprises 133whole-body keypoint annotations on 100K images, made possible by our newmulti-view pipeline. We also propose three tasks: i) 3D whole-body pose liftingfrom 2D complete whole-body pose, ii) 3D whole-body pose lifting from 2Dincomplete whole-body pose, and iii) 3D whole-body pose estimation from asingle RGB image. Additionally, we report several baselines from popularmethods for these tasks. Furthermore, we also provide automated 3D whole-bodyannotations of TotalCapture and experimentally show that when used with H3WB ithelps to improve the performance. Code and dataset is available athttps://github.com/wholebody3d/wholebody3d