Dataset Summary: From Operation Trajectories to 3D Scenes, 8 Robot Datasets Cover Key Tasks for Embodied AI

Robot data is becoming a key variable in the competition for embodied intelligence. As model architectures gradually converge, the source, scale, and format of data begin to determine robots' actual performance in tasks such as manipulation, grasping, and locomotion. Unlike traditional visual and textual data, robot data emphasizes "process"—the complete alignment of trajectories, actions, observations, and task instructions directly determines whether a model can execute stably in real-world environments.

Recent data releases show that robot data production is splitting into two clear routes: one is real-world collection, relying on human teleoperation and optical motion capture to accumulate high-quality trajectories; the other is synthetic simulation, relying on simulators like Isaac Sim and dm_control to batch generate grasping and locomotion samples. Each route handles a different segment, while formats like LeRobot, GR00T, and OpenUSD are enabling them to interlock.

The 8 datasets curated in this article cover several core directions in current robot data research: real-world manipulation demonstrations, video evaluation benchmarks, motion capture, humanoid robot videos, as well as simulated grasping, locomotion, and 3D scene assets. These datasets come from NVIDIA, Microsoft Research, optical motion capture vendor ChingMu, and Macrodata Labs.

The following datasets are now available on HyperAI, helping researchers and developers rapidly explore training, evaluation, and applications.

More high-quality datasets: https://hyper.ai/datasets

1. PhysicalAI-Robotics Robot Kitchen Manipulation Demo Dataset

Run Online: https://go.hyper.ai/EAIP6

PhysicalAI-Robotics-Manipulation-Kitchen-Demos is a large-scale robot manipulation demonstration dataset released by NVIDIA in 2026. It includes approximately 600 hours of human teleoperation demonstrations, covering 316 different tasks and totaling about 55,000 trajectories. The data was collected by Franka Panda robots equipped with Omron mobile bases, aiming to provide real-world manipulation data for robot learning and embodied intelligence research.

This dataset follows the LeRobot standard. Low-dimensional trajectories are stored in parquet format, and videos include three views: left third-person, right third-person, and eye-in-hand. The meta directory provides info.json, tasks.jsonl, episodes.jsonl, stats.json, modality.json, and embodiment.json. The extras directory includes MuJoCo environment parameters, MJCF models, and raw state files for playback.

2. WGO-Bench Robot Video Evaluation Benchmark Dataset

Run Online: https://go.hyper.ai/o3Fze

WGO-Bench is a robot video benchmark dataset released by Macrodata Labs, designed to evaluate the ability of vision-language models to convert robot and first-person manipulation videos into timestamped subtask annotations. It primarily targets boundary detection and subtask annotation tasks.

The dataset contains 100 video clips, covering 743 gold-standard subtask segments and 63 distinct task instructions. Data sources include three types: HomER first-person videos, RoboInter DROID robotic arm videos, and RoboCOIN Galaxea R1 Lite head-mounted camera videos. The data is stored in Parquet format, with MP4 video bytes directly embedded in each row. Fields include id, video, instruction, and segments.

3. MotionDecode Robot Motion Capture Dataset

Run Online: https://go.hyper.ai/CVoFS

MotionDecode is an optical robot motion capture dataset released by ChingMu, focusing on high-precision motion capture for embodied intelligence and humanoid robots. It includes over 1,000 hours of robot motion trajectories, covering more than 15 real-world scenarios such as industrial, home, retail, medical, logistics, agriculture, and performance, involving over 200 trackable objects and more than 500 standardized tasks.

The data includes raw motion data (BVH, 120 frames/second, 4767 skeletal joints), robot retargeting trajectories (CSV), object 6D poses (CSV, 120 Hz), multi-view videos from 48 synchronized cameras, and JSONL semantic labels. All data has undergone automated cleaning and manual quality inspection.

4. GR1-100 Humanoid Robot Manipulation Video Dataset

Run Online: https://go.hyper.ai/zEMXF

GR1-100 is a humanoid robot manipulation video dataset released by NVIDIA in 2025. It contains 92 video clips recording third-person views of the Fourier GR1-T2 humanoid robot performing various manipulation tasks in a laboratory environment. The total size is approximately 142.3 MB. The related paper is "DreamGen: Unlocking Generalization in Robot Learning through Video World Models," suitable for computer vision and robot learning fields.

5. GraspGen Large-Scale Simulated Grasping Dataset

Run Online: https://go.hyper.ai/AhNid

GraspGen is a large-scale simulated grasping dataset released by NVIDIA in 2025. It contains over 57 million grasp data points, generated based on 8,515 objects from Objaverse XL (LVIS). It is designed for three end-effector types: Franka Panda, Robotiq-2f-140 industrial gripper, and a 30 mm radius single-point contact suction cup, aiming to improve the transfer of grasping skills from simulation to reality.

The data uses the WebDataset format, organized into three directories: franka, robotiq2f140, and suction, corresponding to the three gripper configurations.

6. X-Mobility Robot Locomotion Simulation Dataset

Run Online: https://go.hyper.ai/2Omzo

X-Mobility is a dataset released by NVIDIA in 2024 for robot locomotion training, aiming to train world models and action networks through simulated data.

This dataset provides training, validation, and test data specifically for the X-Mobility project, including policy datasets generated based on the Isaac Sim simulator. The dataset covers teacher policy datasets and random action datasets, supporting pre-training of world models and joint training of world models with action networks, providing high-quality simulation data support for robot locomotion research.

7. NVIDIA PhysicalAI SimReady Warehouse OpenUSD Robot Simulation Scene Dataset

Run Online: https://go.hyper.ai/LdadM

NVIDIA Physical AI SimReady Warehouse OpenUSD Dataset is a high-precision 3D dataset released by NVIDIA in 2025 for robot simulation and physical AI research, aiming to provide realistic industrial warehouse scenes and interactive objects for simulation platforms like IsaacSim.

The dataset contains 753 OpenUSD 3D assets compatible with NVIDIA IsaacSim applications, with a compressed package size of approximately 15 GB, covering warehouse environments, graspable objects, and various components. The data provides rich semantic labels (such as WikiData Q-Codes), physical properties (such as mass), and classification information, supporting tasks like image segmentation, object detection, and depth estimation. It is suitable for research in robot grasping, navigation, and scene reconstruction.

8. MoCapAct Humanoid Robot Motion Control Dataset

Run Online: https://go.hyper.ai/UsXf9

MoCapAct is a dataset released by Microsoft Research in 2024 for simulated humanoid robot motion control, aiming to provide pre-trained expert policies and their demonstration data for humanoid robot motion control research, reducing the computational cost of training low-level control policies from motion capture data.

The dataset includes pre-trained expert agents capable of tracking over 3 hours of motion capture data in the dm_control physics simulation environment, as well as replay data generated by these expert agents. The replay data includes proprioceptive observations and actions, covering humanoid motion skills such as standing, walking, and running, and can be used for tasks like imitation learning, reinforcement learning, and high-level motor behavior learning.