Sun Yu ; Bao Qian ; Liu Wu ; Fu Yili ; Black Michael J. ; Mei Tao

Abstract

This paper focuses on the regression of multiple 3D people from a single RGBimage. Existing approaches predominantly follow a multi-stage pipeline thatfirst detects people in bounding boxes and then independently regresses their3D body meshes. In contrast, we propose to Regress all meshes in a One-stagefashion for Multiple 3D People (termed ROMP). The approach is conceptuallysimple, bounding box-free, and able to learn a per-pixel representation in anend-to-end manner. Our method simultaneously predicts a Body Center heatmap anda Mesh Parameter map, which can jointly describe the 3D body mesh on the pixellevel. Through a body-center-guided sampling process, the body mesh parametersof all people in the image are easily extracted from the Mesh Parameter map.Equipped with such a fine-grained representation, our one-stage framework isfree of the complex multi-stage process and more robust to occlusion. Comparedwith state-of-the-art methods, ROMP achieves superior performance on thechallenging multi-person benchmarks, including 3DPW and CMU Panoptic.Experiments on crowded/occluded datasets demonstrate the robustness undervarious types of occlusion. The released code is the first real-timeimplementation of monocular multi-person 3D mesh regression.