PhysicalAI-Robotics-Manipulation-Kitchen-Demos is a large-scale robot manipulation demonstration dataset released by NVIDIA in 2026. It contains approximately 600 hours of human teleoperation demonstrations, covering 316 different tasks and totaling around 55,000 trajectories. The dataset was collected using Franka Panda robots equipped with Omron mobile bases, aiming to provide high-quality real-world manipulation data for research in robotics learning and embodied intelligence, supporting learning and training from basic manipulation to complex multi-step tasks.

Dataset Composition

The dataset structure follows the LeRobot standard and mainly includes the following parts:

Main Data Files (meta/data/videos):

• info.json: Dataset information, including robot type, number of episodes, frame count, frame rate, and features.
• tasks.jsonl: Language instructions with task indices.
• episodes.jsonl: Metadata for each episode, including index, instruction, and length.
• episodes_stats.jsonl: Statistics for actions and proprioception for each episode.
• stats.json: Aggregate statistics for all episodes.
• modality.json: Information contained in observation and action vectors.
• embodiment.json: Embodiment information.

Trajectory Data (data):

• Contains low-dimensional trajectory data, stored in parquet file format.
• Each chunk contains multiple episode files, recording proprioception, actions, completion status, and timestamps.

Video Data (videos):

• Contains MP4 video files for each camera view.
• Includes three views: left third-person camera, right third-person camera, and eye-in-hand camera.

Extra Data (extras):

• MuJoCo-specific metadata, including environment parameters and controller configurations.
• Additional metadata for each episode, including kitchen layout, style, fixtures, and object information.
• Compressed MJCF MuJoCo model XML files and raw MuJoCo state files (for replay, not for training).