license: cc-by-nc-sa-4.0

LET:Full-Size Humanoid Robot Real-World Dataset

中文| [English]

📋 Table of Contents

• Key Features
• Hardware Platform
• Usage Guide
  • Tool Repository
• Tasks and Data Overview
  • Semantic Labels
  • Data Statistics
• Dataset
  • Dataset Directory Structure
  • Data Format
  • Label Format
• Data Access
• Data Communication Group
• Citation
• License

✨ Key Features

• Large-scale, real-world, full-size humanoid robot multi-view, multi-modal data, continuously updated
• Covers multiple domains including industry, home, medical, and service, with 31 sub-task scenarios
• Includes 117 atomic skills such as grasping, bimanual operation, tool use, with a total duration of over 1000 hours
• Expert-labeled and human-verified data to ensure high quality
• Provides a complete toolchain from data conversion, model training to inference and validation

Assembly line sorting	Daily table cleaning	Assembly line sorting (dexterous hand)
Left hand camera view	Head camera view	Right hand camera view

🤖 Hardware Platform

The main hardware platform is Kuavo 4 Pro and its wheeled version, with the following features:

• Robot parameters: Height 1.66 m, weight 55 kg, supports hot-swappable batteries
• Motion control: 40 degrees of freedom, max walking speed 7 km/h, supports bipedal autonomous SLAM
• Generalization: Supports multi-modal large models (e.g., Pangu, DeepSeek, ChatGPT), with 20+ atomic skills

🚀 Usage Guide

Tool Repository

We provide a complete tool repository, including:

• Data conversion tool (rosbag2lerobot): Convert rosbag files to formats suitable for model training
• Two imitation learning models: Diffusion Policy and ACT
• Model training scripts
• Code and deployment instructions for both real robots and simulation environments

For details, see the open-source repository: kuavo_data_challenge 🔥

🎬 Tasks and Data Overview

This dataset covers various scenarios such as automobile factories, FMCG, hotel services, 3C factories, life services, logistics, etc., including multi-modal observations (RGB, Depth, joints, etc.) and a rich set of atomic skills (grasping, bimanual operation, tool use, etc.).

Consumer goods sorting

Simulation data demonstration

Assembly feeding

Semantic Labels

The LET dataset decomposes complex tasks into a series of atomic action steps with clear semantics, using standardized annotation methods to provide sub-task level timelines and natural language annotations for each task.

Each data entry is accompanied by multi-dimensional semantic label information, including:

• Object labels: industrial parts, tableware, daily utensils, medicines, etc.
• Skill labels: grasp, place, rotate, push, pull, press, etc.
• Task and scene identifiers: unified task name coding, scene dimension distinguishes operation context semantics
• End effector type: records actions performed by gripper and dexterous hand separately
• Language description: e.g., "Pick up the medicine box from the conveyor belt and place it on the designated tray", supporting natural language and action alignment modeling

Data Statistics

LET dataset statistics are as follows:

Data type & Scene distribution

Data type distribution	Scene distribution

Task distribution

Task duration distribution

Distribution of atomic skills

📦 Dataset

Dataset Directory Structure

.
├── hdf5
│   ├── real
│   │   ├── Labelled
│   │   │   ├── customer_check_in-P4-dex_hand
│   │   │   ├── deliver_room_card-P4-dex_hand
│   │   │   ├── deliver_water_bottle-P4-dex_hand
│   │   │   ├── loading_of_large_tooling-P4-dex_hand
│   │   │   ├── loading_of_small_tooling-P4-dex_hand
│   │   │   ├── more_coil_sorting-P4-dex_hand
│   │   │   ├── more_FMCG_loading-P4-dex_hand
│   │   │   ├── more_goods_orders-P4-dex_hand
│   │   │   ├── more_scan_code_for_weighing-P4-dex_hand
│   │   │   ├── parts_offline-P4-dex_hand
│   │   │   ├── quick_sort-P4-leju_claw
│   │   │   ├── rubbish_sorting-P4-leju_claw
│   │   │   ├── shop_oversale-P4-leju_claw
│   │   │   ├── single_coil_sorting-P4-dex_hand
│   │   │   ├── single_FMCG_loading-P4-dex_hand
│   │   │   ├── single_goods_orders-P4-dex_hand
│   │   │   ├── single_scan_code_for_weighing-P4-dex_hand
│   │   │   ├── SPS_parts_grab-P4-leju_claw
│   │   │   ├── SPS_parts_sorting-P4-dex_hand
│   │   │   └── task_mass_check-P4-leju_claw
│   │   └── Unlabelled
│   │       ├── assembly_line_sorting-P4-leju_claw
│   │       ├── clothing_storage-P4-leju_claw
│   │       ├── countertop_cleaning-P4-leju_claw
│   │       ├── deliver_room_card-P4-dex_hand
│   │       ├── desktop_decluttering-P4-leju_claw
│   │       ├── drug_finishing-P4-leju_claw
│   │       ├── express_delivery_sorting-P4-leju_claw
│   │       ├── express_logistics_scenario-P4-leju_claw
│   │       ├── loading_of_large_tooling-P4-dex_hand
│   │       ├── loading_of_small_tooling-P4-dex_hand
│   │       ├── loading_of_small_tooling-P4-leju_claw
│   │       ├── more_coil_sorting-P4-dex_hand
│   │       ├── more_FMCG_loading-P4-dex_hand
│   │       ├── more_goods_orders-P4-dex_hand
│   │       ├── more_goods_orders-P4-leju_claw
│   │       ├── more_scan_code_for_weighing-P4-dex_hand
│   │       ├── parts_offline-P4-dex_hand
│   │       ├── parts_off_line-P4-leju_claw
│   │       ├── quick_sort-P4-leju_claw
│   │       ├── rubbish_sorting-P4-leju_claw
│   │       ├── shop_oversale-P4-leju_claw
│   │       ├── single_coil_sorting-P4-dex_hand
│   │       ├── single_FMCG_loading-P4-leju_claw
│   │       ├── single_goods_orders-P4-dex_hand
│   │       ├── SMT_tray_rack_blanking-P4-leju_claw
│   │       ├── SPS_parts_grab-P4-leju_claw
│   │       ├── SPS_parts_sorting-P4-dex_hand
│   │       ├── SPS_parts_sorting-P4-leju_claw
│   │       ├── standardized_feeding_for_FMCG-P4-dex_hand
│   │       └── task_mass_check-P4-leju_claw
│   └── sim
│       └── Unlabelled
│           ├── bottle_flip-P4-claw(Rq2f85)
│           ├── package_weighing-P4-claw(Rq2f85)
│           ├── SPS_parts_sorting-P4-claw(Rq2f85)
│           └── target_placement-P4-claw(Rq2f85)
└── rosbag
    ├── real
    │   ├── Labelled  // Same task structure as HDF5.
    │   └── Unlabelled // Same task structure as HDF5.
    └── sim
        └── Unlabelled // Same task structure as HDF5.

Data Format

ROSbag Data Format

HDF5 Data Format

<task_root>
├── cameras
│   ├── hand_left  // Left hand camera
│   │   ├── color     // RGB image info
│   │   │   └── data   // RGB image data (by timestamp)
│   │   └── depth/     // Depth image info
│   │       └── data   // Depth data
│   ├── hand_right // Right hand camera
│   │   ├── color // RGB image info
│   │   │   └── data // RGB data
│   │   └── depth  // Depth image info
│   │       └── data // Depth data
│   └── head // Head camera
│       ├── color // RGB image info
│       │   └── data // RGB image data
│       └── depth // Depth image info
│           └── data // Depth data
├── joints  // Joint data
│   ├── action  // Desired joint values
│   │   ├── arm // Arm
│   │   │   ├── position   // N(rows)*14(cols); N=frames, 14=DoF for both arms (7 per arm)
│   │   │   └── velocity   // Desired joint velocity
│   │   ├── effector // End effector
│   │   │   └── position   //  N(rows)*2(cols); N=frames, 2=left/right gripper open/close
│   │   ├── head // Head
│   │   │   ├── position   //  N(rows)*2(cols); N=frames, 2=2 DoF (pitch/yaw)
│   │   │   └── velocity  // Joint velocity
│   │   └── leg // Leg 
│   │       ├── position   //  N(rows)*12(cols)
│   │       └── velocity  // Joint velocity 
│   └── state  // Actual joint values
│       ├── arm // Arm
│       │   ├── position // N(rows)*14(cols); N=frames, 14=DoF for both arms (7 per arm)
│       │   └── velocity // Joint velocity
│       ├── effector // End effector
│       │   └── position //  N(rows)*2(cols); N=frames, 2=left/right gripper open/close
│       ├── head // Head
│       │   ├── position  // N(rows)*2(cols); N=frames, 2=2 DoF (pitch/yaw)
│       │   └── velocity   // Joint velocity
│       └── leg // Leg
│           ├── position //  N(rows)*12(cols)
│           └── velocity // Joint velocity
├── parameters  // Sensor extrinsics
│   └── camera
│       ├── hand_left.json   # Left hand camera intrinsics/extrinsics
│       ├── hand_right.json  # Right hand camera intrinsics/extrinsics
│       └── head.json        # Head camera intrinsics/extrinsics
└── metadata.json            # Collection metadata: device, end effector type, camera frame rate, joint info, etc.

Label Format

Label information is stored in a JSON file with the same name as the data file. Example:

{
  "loaction": "Yangtze River Delta Integrated Demonstration Zone Intelligent Robot Training Center",
  "primaryScene": "Default primary scene",
  "primarySceneCode": "default_level_one_scene",
  "secondaryScene": "3C factory scene",
  "secondarySceneCode": "3C factory manufacturing",
  "tertiaryScene": "Coil sorting",
  "tertiarySceneCode": "Coil sorting",
  "initSceneText": "Coils of various colors are placed in the middle of the table, material boxes are placed on both sides of the table, and the robot is located at the back of the table",
  "englishInitSceneText": "Coils of various colors are placed in the middle of the table, material boxes are placed on both sides of the table, and the robot is located at the back of the table",
  "taskGroupName": "Single coil sorting",
  "taskGroupCode": "single_coil_sorting",
  "taskName": "7-22-Coil classification",
  "taskCode": "XQFL_11",
  "deviceSn": "P4-209",
  "taskPrompt": "",
  "marks": [
    {
      "taskId": "1947326026455584768",
      "markStart": "2025-07-22 9:18:39.640",
      "markEnd": "2025-07-22 9:18:39.814",
      "duration": 0.233,
      "startPosition": 0.7363737795977026,
      "endPosition": 0.769568869806783,
      "skillAtomic": "pick",
      "skillDetail": "Pick up the coil from the table",
      "enSkillDetail": "pick coil from table",
      "markType": "step"
    }
  ]
}

📥Data Access

• Official request: You can request access by contacting the official email [email protected].
• Public platforms: The LET dataset will be publicly released on major platforms such as Openloong, ModelScope, and Hugging Face to provide convenience for developers and researchers worldwide.

📋 Data Communication Group

• Data communication QQ group: 1043359345
  

📝 Citation

If you use this dataset in your research, please cite it according to the platform from which you accessed it:

Citation for Hugging Face

@misc{LET2025,
    title={LET:Full-Size Humanoid Robot Real-World Dataset},
    author={LejuRobotics},
    year={2025},
    howpublished={\url{https://huggingface.co/datasets/LejuRobotics/let_dataset}}
}

Citation for ModelScope

@misc{LET2025,
    title={LET:Full-Size Humanoid Robot Real-World Dataset},
    author={LejuRobotics},
    year={2025},
    howpublished={\url{https://www.modelscope.cn/datasets/lejurobot/let_dataset}}
}

Citation for Atomgit AI

@misc{LET2025,
    title={LET:Full-Size Humanoid Robot Real-World Dataset},
    author={LejuRobotics},
    year={2025},
    howpublished={\url{https://ai.atomgit.com/lejurobot/let_dataset}}
}

📄 License

All the data and code within this repo are under CC BY-NC-SA-4.0.