Papantoniou Foivos Paraperas ; Lattas Alexandros ; Moschoglou Stylianos ; Deng Jiankang ; Kainz Bernhard ; Zafeiriou Stefanos

Abstract

This paper presents Arc2Face, an identity-conditioned face foundation model,which, given the ArcFace embedding of a person, can generate diversephoto-realistic images with an unparalleled degree of face similarity thanexisting models. Despite previous attempts to decode face recognition featuresinto detailed images, we find that common high-resolution datasets (e.g. FFHQ)lack sufficient identities to reconstruct any subject. To that end, wemeticulously upsample a significant portion of the WebFace42M database, thelargest public dataset for face recognition (FR). Arc2Face builds upon apretrained Stable Diffusion model, yet adapts it to the task of ID-to-facegeneration, conditioned solely on ID vectors. Deviating from recent works thatcombine ID with text embeddings for zero-shot personalization of text-to-imagemodels, we emphasize on the compactness of FR features, which can fully capturethe essence of the human face, as opposed to hand-crafted prompts. Crucially,text-augmented models struggle to decouple identity and text, usuallynecessitating some description of the given face to achieve satisfactorysimilarity. Arc2Face, however, only needs the discriminative features ofArcFace to guide the generation, offering a robust prior for a plethora oftasks where ID consistency is of paramount importance. As an example, we traina FR model on synthetic images from our model and achieve superior performanceto existing synthetic datasets.