Creating Datasets

Collecting human demonstrations

You can use the collect_demos.py script to collect demonstrations:

python robocasa/scripts/collect_demos.py --env <env-name>

Attention Mac users

Mac users must prepend this script with mj, ie. mjpython

You can either control the robot with a spacemouse (--device spacemouse) or the keyboard (--device keyboard). A spacemouse is recommended.

This will save a raw dataset. Follow the steps in the next section to extract image observations for this dataset.

Extracting observations

To extract observations from an existing dataset, you can run:

OMP_NUM_THREADS=1 MPI_NUM_THREADS=1 MKL_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 python robocasa/scripts/dataset_states_to_obs.py --dataset <ds-path>

This script will generate a new dataset with the suffix _im128.hdf5 in the same directory as --dataset.

Image resolution

You can override the default image resolution by setting the flags --camera_height and --camera_width.

Visual randomization

You can add the flag --generative_textures to render images with AI-generated environment textures, and --randomize_cameras to randomize camera viewpoints for rendering.

Warning!

This observation extraction script may drop a few demonstrations due to subprocesses failing. To minimize this issue you can run the script with the flag --num_procs 1