Sengupta Akash ; Budvytis Ignas ; Cipolla Roberto

Abstract

This paper addresses the problem of 3D human body shape and pose estimationfrom an RGB image. This is often an ill-posed problem, since multiple plausible3D bodies may match the visual evidence present in the input - particularlywhen the subject is occluded. Thus, it is desirable to estimate a distributionover 3D body shape and pose conditioned on the input image instead of a single3D reconstruction. We train a deep neural network to estimate a hierarchicalmatrix-Fisher distribution over relative 3D joint rotation matrices (i.e. bodypose), which exploits the human body's kinematic tree structure, as well as aGaussian distribution over SMPL body shape parameters. To further ensure thatthe predicted shape and pose distributions match the visual evidence in theinput image, we implement a differentiable rejection sampler to impose areprojection loss between ground-truth 2D joint coordinates and samples fromthe predicted distributions, projected onto the image plane. We show that ourmethod is competitive with the state-of-the-art in terms of 3D shape and posemetrics on the SSP-3D and 3DPW datasets, while also yielding a structuredprobability distribution over 3D body shape and pose, with which we canmeaningfully quantify prediction uncertainty and sample multiple plausible 3Dreconstructions to explain a given input image. Code is available athttps://github.com/akashsengupta1997/HierarchicalProbabilistic3DHuman .