Saito Shunsuke ; Huang Zeng ; Natsume Ryota ; Morishima Shigeo ; Kanazawa Angjoo ; Li Hao

Abstract

We introduce Pixel-aligned Implicit Function (PIFu), a highly effectiveimplicit representation that locally aligns pixels of 2D images with the globalcontext of their corresponding 3D object. Using PIFu, we propose an end-to-enddeep learning method for digitizing highly detailed clothed humans that caninfer both 3D surface and texture from a single image, and optionally, multipleinput images. Highly intricate shapes, such as hairstyles, clothing, as well astheir variations and deformations can be digitized in a unified way. Comparedto existing representations used for 3D deep learning, PIFu can producehigh-resolution surfaces including largely unseen regions such as the back of aperson. In particular, it is memory efficient unlike the voxel representation,can handle arbitrary topology, and the resulting surface is spatially alignedwith the input image. Furthermore, while previous techniques are designed toprocess either a single image or multiple views, PIFu extends naturally toarbitrary number of views. We demonstrate high-resolution and robustreconstructions on real world images from the DeepFashion dataset, whichcontains a variety of challenging clothing types. Our method achievesstate-of-the-art performance on a public benchmark and outperforms the priorwork for clothed human digitization from a single image.