Biggs Benjamin ; Ehrhadt Sébastien ; Joo Hanbyul ; Graham Benjamin ; Vedaldi Andrea ; Novotny David

Abstract

We consider the problem of obtaining dense 3D reconstructions of humans fromsingle and partially occluded views. In such cases, the visual evidence isusually insufficient to identify a 3D reconstruction uniquely, so we aim atrecovering several plausible reconstructions compatible with the input data. Wesuggest that ambiguities can be modelled more effectively by parametrizing thepossible body shapes and poses via a suitable 3D model, such as SMPL forhumans. We propose to learn a multi-hypothesis neural network regressor using abest-of-M loss, where each of the M hypotheses is constrained to lie on amanifold of plausible human poses by means of a generative model. We show thatour method outperforms alternative approaches in ambiguous pose recovery onstandard benchmarks for 3D humans, and in heavily occluded versions of thesebenchmarks.