From aec4fd5713846f1244549ba658e4877cd60c36d1 Mon Sep 17 00:00:00 2001 From: Ajay Mandlekar Date: Mon, 8 Jul 2024 16:26:14 -0700 Subject: [PATCH] some docs updates --- docs/conf.py | 2 +- docs/introduction/installation.md | 16 +++++++++++---- docs/miscellaneous/troubleshooting.md | 13 +++++++++---- docs/tutorials/getting_started.md | 28 ++++++++++++++++++++++++--- requirements-docs.txt | 1 - 5 files changed, 47 insertions(+), 13 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 55a381e..74c2c52 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -33,7 +33,7 @@ # ones. extensions = [ 'sphinx.ext.napoleon', - 'sphinx_markdown_tables', + # 'sphinx_markdown_tables', 'sphinx.ext.mathjax', 'sphinx.ext.githubpages', 'sphinx.ext.autodoc', diff --git a/docs/introduction/installation.md b/docs/introduction/installation.md index 215799f..4601cea 100644 --- a/docs/introduction/installation.md +++ b/docs/introduction/installation.md @@ -43,8 +43,8 @@ Most of these additional dependencies are best installed from source. ```sh $ cd $ git clone https://github.com/ARISE-Initiative/robosuite.git -$ git checkout b9d8d3de5e3dfd1724f4a0e6555246c460407daa $ cd robosuite +$ git checkout b9d8d3de5e3dfd1724f4a0e6555246c460407daa $ pip install -e . ``` @@ -64,8 +64,8 @@ The git checkout command corresponds to the commit we used for testing our polic ```sh $ cd $ git clone https://github.com/ARISE-Initiative/robomimic.git -$ git checkout d0b37cf214bd24fb590d182edb6384333f67b661 $ cd robomimic +$ git checkout d0b37cf214bd24fb590d182edb6384333f67b661 $ pip install -e . ``` @@ -87,11 +87,19 @@ The git checkout command corresponds to the commit we used for testing our polic +
+

Warning

+ +We recommend removing the dependencies in the `setup.py` file (the `install_requires` list) before installation, as it uses deprecated dependencies (such as mujoco-py). + +
+ ```sh $ cd $ git clone https://github.com/ARISE-Initiative/robosuite-task-zoo +$ cd robosuite-task-zoo $ git checkout 74eab7f88214c21ca1ae8617c2b2f8d19718a9ed -$ cd robosuite_task_zoo +# NOTE: should remove dependencies in setup.py in the "install_requires" before the last step $ pip install -e . ``` @@ -129,7 +137,7 @@ To test data generation please move on to the [Getting Started](https://mimicgen -## Install documentation dependencies +## Install Documentation Dependencies If you plan to contribute to the repository and add new features, you must install the additional requirements required to build the documentation locally: diff --git a/docs/miscellaneous/troubleshooting.md b/docs/miscellaneous/troubleshooting.md index 1cd5a97..475e7a4 100644 --- a/docs/miscellaneous/troubleshooting.md +++ b/docs/miscellaneous/troubleshooting.md @@ -1,9 +1,14 @@ # Troubleshooting and Known Issues -- If your robomimic training seems to be proceeding slowly (especially for image-based agents), it might be a problem with robomimic and more modern versions of PyTorch. We recommend PyTorch 1.12.1 (on Ubuntu, we used `conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch`). It is also a good idea to verify that the GPU is being utilized during training. -- In our testing on M1 macbook we ran into the following error when using `imageio-ffmpeg` installed through pip: `RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.` Using `conda install imageio-ffmpeg` fixed this issue on our end. +If you run into an error not documented below, please search through the [GitHub issues](https://github.com/NVlabs/mimicgen/issues), and create a new one if you cannot find a fix. + +## Installation + +- In our testing on M1 macbook we ran into the following error: `RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.` Using `conda install ffmpeg` fixed this issue on our end. - If you run into trouble with installing [egl_probe](https://github.com/StanfordVL/egl_probe) during robomimic installation (e.g. `ERROR: Failed building wheel for egl_probe`) you may need to make sure `cmake` is installed. A simple `pip install cmake` should work. - If you run into other strange installation issues, one potential fix is to launch a new terminal, activate your conda environment, and try the install commands that are failing once again. One clue that the current terminal state is corrupt and this fix will help is if you see installations going into a different conda environment than the one you have active. -- If you run into rendering issues with the Sawyer robot arm, or have trouble reproducing our results, your MuJoCo version might be the issue. As noted in the [Installation](https://mimicgen.github.io/docs/introduction/installation.html) section, please use MuJoCo 2.3.2 (`pip install mujoco==2.3.2`). -If you run into an error not documented above, please search through the [GitHub issues](https://github.com/NVlabs/mimicgen/issues), and create a new one if you cannot find a fix. \ No newline at end of file +## Policy Learning + +- If your robomimic training seems to be proceeding slowly (especially for image-based agents), it might be a problem with robomimic and more modern versions of PyTorch. We recommend PyTorch 1.12.1 (on Ubuntu, we used `conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch`). It is also a good idea to verify that the GPU is being utilized during training. +- If you run into rendering issues with the Sawyer robot arm, or have trouble reproducing our results, your MuJoCo version might be the issue. As noted in the [Installation](https://mimicgen.github.io/docs/introduction/installation.html) section, please use MuJoCo 2.3.2 (`pip install mujoco==2.3.2`). diff --git a/docs/tutorials/getting_started.md b/docs/tutorials/getting_started.md index d7f23fe..0300873 100644 --- a/docs/tutorials/getting_started.md +++ b/docs/tutorials/getting_started.md @@ -28,7 +28,10 @@ $ python mimicgen/scripts/download_datasets.py --dataset_type source --tasks squ This is a basic robomimic dataset collected via teleoperation in robosuite (see [here](https://robomimic.github.io/docs/datasets/robosuite.html)). We need to add in extra information to the hdf5 to make it compatible with MimicGen: ```sh -$ python mimicgen/scripts/prepare_src_dataset.py --dataset datasets/source/square.hdf5 --env_interface MG_Square --env_interface_type robosuite +$ python mimicgen/scripts/prepare_src_dataset.py \ +--dataset datasets/source/square.hdf5 \ +--env_interface MG_Square \ +--env_interface_type robosuite ``` The `--env_interface` and `--env_interface_type` arguments allow the script to find the correct [Environment Interface](https://mimicgen.github.io/docs/modules/env_interfaces.html) class to extract [DatagenInfo objects](https://mimicgen.github.io/docs/modules/datagen.html#datagen-info) at each timestep. In general, each task needs to have an environment interface class to tell MimicGen how to retrieve object poses and other information needed during data generation. @@ -37,7 +40,17 @@ The `--env_interface` and `--env_interface_type` arguments allow the script to f Each data generation run requires a config json (similar to robomimic [Configs](https://robomimic.github.io/docs/modules/configs.html)) that allows us to configure different settings. Template configs for each task are at `mimicgen/exps/templates` and are auto-generated (with `scripts/generate_config_templates.py`). The repository has easy ways to modify these templates to generate new config jsons. -For now, we will use a script to produce experiment configs consistent with the MimicGen paper. Open `scripts/generate_core_configs.py` and set `NUM_TRAJ = 10` and `GUARANTEE = False` -- this means we will attempt to generate 10 new trajectories. Next, run the script: +For now, we will use a script to produce experiment configs consistent with the MimicGen paper. Open `scripts/generate_core_configs.py` and set `NUM_TRAJ = 10` and `GUARANTEE = False` -- this means we will attempt to generate 10 new trajectories. + +
+

Warning

+ +If you do not edit `scripts/generate_core_configs.py` the default settings will run data generation until 1000 success trajectories have been collected. This is why it is important to set `NUM_TRAJ = 10` and `GUARANTEE = False` for a quick run. Alternatively, pass the `--debug` flag to the command in Step 3, which will run an even smaller data generation run. + +
+ +Next, run the script: + ```sh $ python mimicgen/scripts/generate_core_configs.py ``` @@ -48,9 +61,18 @@ It generates a set of configs (and prints their paths) and also prints lines tha Next, we run data generation on the Square D1 task (this will take a couple minutes): ```sh -$ python mimicgen/scripts/generate_dataset.py --config /tmp/core_configs/demo_src_square_task_D1.json --auto-remove-exp +$ python mimicgen/scripts/generate_dataset.py \ +--config /tmp/core_configs/demo_src_square_task_D1.json \ +--auto-remove-exp ``` +
+

Note

+ +If you run into a `RuntimeError: No ffmpeg exe could be found.` at the end of the script, this means rendering the dataset to video failed. We found that a simple `conda install ffmpeg` fixed the problem on our end (as documented on the [troubleshooting page](https://mimicgen.github.io/docs/miscellaneous/troubleshooting.html)). + +
+ By default, the data generation folder can be found at `/tmp/core_datasets/square/demo_src_square_task_D1`. The contents of this folder are as follows: ``` demo.hdf5 # generated hdf5 containing successful demonstrations diff --git a/requirements-docs.txt b/requirements-docs.txt index c3e63e5..3381b70 100644 --- a/requirements-docs.txt +++ b/requirements-docs.txt @@ -1,4 +1,3 @@ # requirements for building sphinx docs sphinx_book_theme==0.3.3 -sphinx_markdown_tables==0.0.15 recommonmark==0.7.1