Shen Kaiyue ; Guo Chen ; Kaufmann Manuel ; Zarate Juan Jose ; Valentin Julien ; Song Jie ; Hilliges Otmar

Abstract

We present X-Avatar, a novel avatar model that captures the fullexpressiveness of digital humans to bring about life-like experiences intelepresence, AR/VR and beyond. Our method models bodies, hands, facialexpressions and appearance in a holistic fashion and can be learned from eitherfull 3D scans or RGB-D data. To achieve this, we propose a part-aware learnedforward skinning module that can be driven by the parameter space of SMPL-X,allowing for expressive animation of X-Avatars. To efficiently learn the neuralshape and deformation fields, we propose novel part-aware sampling andinitialization strategies. This leads to higher fidelity results, especiallyfor smaller body parts while maintaining efficient training despite increasednumber of articulated bones. To capture the appearance of the avatar withhigh-frequency details, we extend the geometry and deformation fields with atexture network that is conditioned on pose, facial expression, geometry andthe normals of the deformed surface. We show experimentally that our methodoutperforms strong baselines in both data domains both quantitatively andqualitatively on the animation task. To facilitate future research onexpressive avatars we contribute a new dataset, called X-Humans, containing 233sequences of high-quality textured scans from 20 participants, totalling 35,500data frames.