Zheng Zerong ; Yu Tao ; Liu Yebin ; Dai Qionghai

Abstract

Modeling 3D humans accurately and robustly from a single image is verychallenging, and the key for such an ill-posed problem is the 3D representationof the human models. To overcome the limitations of regular 3D representations,we propose Parametric Model-Conditioned Implicit Representation (PaMIR), whichcombines the parametric body model with the free-form deep implicit function.In our PaMIR-based reconstruction framework, a novel deep neural network isproposed to regularize the free-form deep implicit function using the semanticfeatures of the parametric model, which improves the generalization abilityunder the scenarios of challenging poses and various clothing topologies.Moreover, a novel depth-ambiguity-aware training loss is further integrated toresolve depth ambiguities and enable successful surface detail reconstructionwith imperfect body reference. Finally, we propose a body referenceoptimization method to improve the parametric model estimation accuracy and toenhance the consistency between the parametric model and the implicit function.With the PaMIR representation, our framework can be easily extended tomulti-image input scenarios without the need of multi-camera calibration andpose synchronization. Experimental results demonstrate that our method achievesstate-of-the-art performance for image-based 3D human reconstruction in thecases of challenging poses and clothing types.