Abstract

In this work, we present Digital Life Project, a framework utilizing languageas the universal medium to build autonomous 3D characters, who are capable ofengaging in social interactions and expressing with articulated body motions,thereby simulating life in a digital environment. Our framework comprises twoprimary components: 1) SocioMind: a meticulously crafted digital brain thatmodels personalities with systematic few-shot exemplars, incorporates areflection process based on psychology principles, and emulates autonomy byinitiating dialogue topics; 2) MoMat-MoGen: a text-driven motion synthesisparadigm for controlling the character's digital body. It integrates motionmatching, a proven industry technique to ensure motion quality, withcutting-edge advancements in motion generation for diversity. Extensiveexperiments demonstrate that each module achieves state-of-the-art performancein its respective domain. Collectively, they enable virtual characters toinitiate and sustain dialogues autonomously, while evolving theirsocio-psychological states. Concurrently, these characters can performcontextually relevant bodily movements. Additionally, a motion captioningmodule further allows the virtual character to recognize and appropriatelyrespond to human players' actions. Homepage: https://digital-life-project.com/