Abstract

The field of 4D world modeling - aiming to jointly capture spatial geometryand temporal dynamics - has witnessed remarkable progress in recent years,driven by advances in large-scale generative models and multimodal learning.However, the development of truly general 4D world models remains fundamentallyconstrained by the availability of high-quality data. Existing datasets andbenchmarks often lack the dynamic complexity, multi-domain diversity, andspatial-temporal annotations required to support key tasks such as 4D geometricreconstruction, future prediction, and camera-control video generation. Toaddress this gap, we introduce OmniWorld, a large-scale, multi-domain,multi-modal dataset specifically designed for 4D world modeling. OmniWorldconsists of a newly collected OmniWorld-Game dataset and several curated publicdatasets spanning diverse domains. Compared with existing synthetic datasets,OmniWorld-Game provides richer modality coverage, larger scale, and morerealistic dynamic interactions. Based on this dataset, we establish achallenging benchmark that exposes the limitations of current state-of-the-art(SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuningexisting SOTA methods on OmniWorld leads to significant performance gainsacross 4D reconstruction and video generation tasks, strongly validatingOmniWorld as a powerful resource for training and evaluation. We envisionOmniWorld as a catalyst for accelerating the development of general-purpose 4Dworld models, ultimately advancing machines' holistic understanding of thephysical world.