IsaacGym Bimanual Franka task

This repo is based on Nvidia's repo IsaacGymEnvs, edited by me for the ICIRA conference and RAS conference experiment.

In this work, we propose new training methods for multi-agent RL. And we will provide Isaac Gym's users a test environment to help you monitor your training process. Moreover, some multi-agent APIs will be provided for multi-agent training in Isaac Gym.

A part of our work will be published at the ICIRA conference; the other part is still working and expected to be accepted by RAS.

• IsaacGym Bimanual Franka task
  • Handbook about our work
    • Installation
    • Test environment during the training
    • Test environment's keyboard shortcut
  • Work achievement
    • About ICIRA
    • About RAS

Handbook about our work

This is a handbook about our work's difference with Nvidia's work.

1. We provide a test platform where you can view the rewards, obs, or any information you want to know during the training process. Moreover, it can record offline data in Simulate environment to apply the offline algorithm without a real environment.

   It provides you more choices to train your tasks at a low cost.

2. We will provide a multi-agent train API with multi-PPO, multi-sac algorithms to help you train your multi-agent tasks.

Installation

1. Create the conda environment

   conda env create -f config/environment.yml
   

2. Install pytorch with the correct CUDA version from, 1.12.1 was tested.

3. Install IssacGym

4. Install this repo

   pip3 install -e .
   

5. sudo cp /home/ubuntu/anaconda3/envs/pytorch/lib/libpython3.8.so.1.0 /usr/lib

6. Install Ninja

   wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
   sudo unzip ninja-linux.zip -d /usr/local/bin/
   sudo update-alternatives --install /usr/bin/ninja ninja /usr/local/bin/ninja 1 --force
   

Pack this repo to replace the official isaacgymenvs package

python3 setup.py bdist_wheel 
python3 setup.py sdist
pip3 install -e .

Test environment during the training

1. Place test_new.py under the same folder with train.py, should be /IsaacGymEnvs/isaacgymenvs

2. Check the test config file in test_config.ini

3. Running in the environment

   python test.py
   

4. Save pose data under folder ./test_save, hdf5 under ./test_save/hdf5

Test environment's keyboard shortcut

Keyboard shortcut	Description
R	Reset the environment. Need to disable pose override in the viewer first.
T	Print the obs/reward once in terminal.
P	Turn on/off the print info.
C	Switch the view angle
S	Save the pose status in file.
0	Enable/disable target tracking
N	Force jump to next stage in auto tracking
D	Enable/disable debug info
V	Check task stage in certain obs
9	Pause tracking
L	Override Franka DOFs
UP, DOWN, LEFT, RIGHT	Drive franka position manually in xz plane
[, ]	Drive franka position manually in y axis
LEFT_SHIFT	Switch driving between left/right Franka

Work achievement

We try to show you our efforts from two papers; the ICIRA one is already accepted as soon as the RAS one is still working!

About ICIRA

Title: 'Mixline: A Hybrid Reinforcement Learning Framework for Long-horizon Bimanual Coffee Stirring Task'

The camera-ready version:

https://mycuhk-my.sharepoint.com/:b:/g/personal/1155169521_link_cuhk_edu_hk/ETAQRHX-UcRJr2E4gKzROZgBYWgVX_5-VrLBOjP8WaIq0w?e=l3X3hg

The powerpoint about our ICIRA work:

https://mycuhk-my.sharepoint.com/:p:/g/personal/1155169521_link_cuhk_edu_hk/EaD-nU8Dc2hHreP4dB2QQ-oB5EnMBSCWO4AFzSCv7wUfpQ?e=D1nHnh

Mixline diagram:

About RAS

The paper for RAS is still working.