tags:

• robot
• maniskill

Robot ManiSkill Image–State Dataset

0. Update

12.22

pickcube_rgb_random_v2: having a broader distribution of viewpoints and qpos values

12.14

pickcube_rgb_traj_v3: official PickCube-v1 scene RGBD trajectories (Type-1 scaling baseline)

12.2

delete pickcube_rgb_traj_v1

11.30

pickcube_rgb_traj_v2: rendering images with cube in pickcube task.

11.28

pickcube_rgb_traj_v1: Trajectory subset

pickcube_rgb_random_v1: Random joint sampling subset

1. Overview

This dataset is generated in the ManiSkill simulation environment.
It contains two types of data pairs:

1. Random joint sampling subset

   • Randomly sampled robot joint configurations (qpos).
   • For each sampled configuration, multi-view RGBD images are rendered.

2. Trajectory subset

   • Full trajectories from ManiSkill's demonstrations (.h5).
   • For each time step, one-view RGBD images are rendered.

2. Dataset Structure

On the Hub, the dataset is organized as follows:

surgingTu/robot-maniskill-image-state-dataset-v1
├── pickcube_rgb_traj_v1/
│   └── PickCube-v1/
│       ├── traj_0/
│       ├── traj_1/
│       ├── ...
│       └── traj_K/
└── pickcube_rgb_random_v1/
    └── random_robot_rendering/
        ├── sample_00000/
        ├── sample_00001/
        ├── ...
        └── sample_NNNNN/

2.1 Trajectory subset

Each trajectory directory (e.g. traj_0) has the structure:

pickcube_rgb_traj_v1/PickCube-v1/traj_0/
  ├── images/
  │     ├── {CAM_NAME}_step0000.png
  |     ├── {CAM_NAME}_step0000_depth.npy
  │     ├── {CAM_NAME}_step0001.png
  |     ├── {CAM_NAME}_step0001_depth.npy
  │     ├── ...
  │     ├── {CAM_NAME}_step{T-1:04d}.png
  │     └── {CAM_NAME}_step{T-1:04d}_depth.npy
  └── actions.npy          # shape (T, A), low-level actions per time step
  ├── qpos.npy    # shape (T, D_qpos), joint positions per time step
  ├── camera_params.json
  ├── ...

CAM_NAME encodes the camera configuration (e.g. y_angle_0_z_angle_0).
*_stepXXXX.png is the image at time step XXXX.

All cameras share the same step index.

2.2 Random joint sampling subset

Each random sample directory (e.g. sample_0000) has the structure:

pickcube_rgb_random_v1/random_robot_rendering/sample_00000/
  ├── images/
  │     ├── {CAM_NAME}.png
  |     ├── {CAM_NAME}_depth.npy
  │     ├── {CAM_NAME}.png
  |     ├── {CAM_NAME}_depth.npy
  │     ├── ...
  │     ├── {CAM_NAME}.png
  │     └── {CAM_NAME}_depth.npy
  └── qpos.npy             # shape (1, D_qpos), random joint configurations
  ├── camera_params.json
  ├── ...