Arena-GR1-Manipulation-PlaceItemCloseDoor-Task is a multimodal dataset released by NVIDIA in 2025 for humanoid robot manipulation tasks, designed to support the training and evaluation of general robotic policies.

The dataset includes trajectory data generated in the Isaac Lab environment, covering state, visual, linguistic, and action modalities, making it suitable for behavior cloning, policy learning, and sim-to-real transfer research. With a size of approximately 9.44 GB, the data was automatically generated using NVIDIA's MimicGen framework and supplemented with a small number of human teleoperation demonstrations for reference.

Dataset Composition

This dataset primarily consists of the following subsets:

• Human Teleoperation Demonstrations: 10 demos stored in an HDF5 file ([OBJECT]_into_fridge_recorded.hdf5).
• Manually Annotated Demonstrations: 10 demos stored in an HDF5 file ([OBJECT]_into_fridge_annotated.hdf5).
• MimicGen Generated Demonstrations: 100 demos stored in an HDF5 file ([OBJECT]_into_fridg_generated_100.hdf5).
• GR00T-Lerobot Format: Converted from MimicGen-generated HDF5 files, compliant with the GR00T-Lerobot standard.

Data Fields

Each demonstration comprises time-indexed sequences containing the following modalities:

• Actions: action (FP64), representing desired positions for all body joints (36 degrees of freedom).

• Observations: observation.state (FP64), including positional states for all body joints (54 degrees of freedom).

• Task-Specific Information:

  • timestamp (FP64): Simulation time in seconds.
  • annotation.human.action.task_description (INT64): Index of the textual instruction.
  • annotation.human.action.valid (INT64): Validity index of the annotations.
  • episode_index (INT64): Sequential demo identifier.
  • task_index (INT64): Multi-task dataloader index (always 0).

• Videos: First-person perspective camera recordings in RGB MP4 format at 512x512 resolution.

• episodes.jsonl: Contains lists of all episodes along with their durations.

• tasks.jsonl: Lists all available tasks.

• modality.json: Modality configuration details.

• info.json: General dataset information.

• stats.json: Statistical summaries of the dataset.

• relative_stats.json: Relative statistical metrics.

Citation

@inproceedings{mandlekar2023mimicgen,
    title={MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations},
    author={Mandlekar, Ajay and Nasiriany, Soroush and Wen, Bowen and Akinola, Iretiayo and Narang, Yashraj and Fan, Linxi and Zhu, Yuke and Fox, Dieter},
    booktitle={7th Annual Conference on Robot Learning},
    year={2023}
    }