license: mit
task_categories:
- image-to-3d
tags:
- 3d-editing
- text-to-3d

Steer3D Training Data

This repository contains the training data for the paper Feedforward 3D Editing via Text-Steerable Image-to-3D.

Project Page: https://glab-caltech.github.io/steer3d/
Code: https://github.com/ziqi-ma/Steer3D

The metadata directory contains training metadata used for texture, removal, and addition. data/images contains the input images, identified by a unique uuid for each object. We release TRELLIS latents of all source and target assets in data/latents, also identified by the uuid.

Sample Usage

This dataset is used to train the Steer3D model. Below are examples of how to train Stage 1 and Stage 2 of the model using this dataset, adapted from the GitHub repository.

First, ensure your PYTHONPATH is set to the root directory of the Steer3D repository:

export PYTHONPATH=[path to Steer3D]

Flow-matching training of Stage 1:

python trainers/train_stage1.py --config configs/stage1_controlnet.json --output_dir outputs/stage1

Flow-matching training of Stage 2:

python trainers/train_stage2.py --config configs/stage2_controlnet.json --output_dir outputs/stage2

DPO of Stage 2:

python trainers/train_stage2.py --config configs/stage2_dpo.json --output_dir outputs/stage2dpo

Finetuning Stage 1 backbone:

python trainers/train_stage1_backbone.py \
        --config configs/stage1_base_sft.json \
        --output_dir outputs/stage1_sft