Cleaned experiment files

.gitignore

@@ -0,0 +1,8 @@
+useless/
+storage/
+slurm_logs/
+plots/
+outputs/
+notebooks/*
+!notebooks/*.ipynb
+/old_slurms/

README.md

@@ -0,0 +1,31 @@
+This repository is currently under active cleaning.
+
+# Installation steps
+1. Create conda env
+```
+conda create -n dlp python=3.10.8; conda activate dlp
+```
+2. Install PyTorch
+```
+conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
+```
+3. Install packages required by our package
+```
+pip install -r requirements.txt
+```
+4. Install BabyAI
+```
+pip install blosc; cd babyai; pip install -e .; cd ..
+```
+5. Install gym-minigrid
+```
+cd gym-minigrid; pip install -e.; cd ..
+```
+6. Install Accelerate
+```
+cd v0.13.2/accelerate-0.13.2; pip install -e .; cd ../..
+```
+7. Install Lamorel
+```
+cd language-models-for-rl/lamorel; pip install -e .; cd ../..
+```

babyai/.gitignore

@@ -0,0 +1,112 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# dotenv
+.env
+
+# virtualenv
+.venv
+venv/
+ENV/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# pycharm
+.idea/
+
+# storage
+storage/
+
+# pytorch weights, training history
+*.csv
+*.pt
+*.json

babyai/.travis.yml

@@ -0,0 +1,49 @@
+language: python
+cache: pip
+python:
+    - "3.5"
+
+before_install:
+    - pip3 install --upgrade pip
+
+# command to install dependencies
+install:
+    - pip3 install http://download.pytorch.org/whl/cpu/torch-0.4.1-cp35-cp35m-linux_x86_64.whl
+    - pip3 install flake8
+    - pip3 install scikit-build
+    - pip3 install --editable .
+
+# command to run tests
+script:
+    # Check the source code for obvious errors
+    - python3 -m flake8 . --count --show-source --statistics --select=E901,E999,F821,F822,F823
+
+    # Test the BabyAI levels
+    - ./run_tests.py
+
+    # Quickly exercise the RL training code
+    - time python3 -m scripts.train_rl --env BabyAI-GoToObj-v0 --algo ppo --procs 4 --batch-size 80 --log-interval 1 --save-interval 2 --val-episodes 10 --frames 300 --arch cnn1 --instr-dim 16 --image-dim 16 --memory-dim 16
+
+    # Check that the bot works on a few episodes of Boss Level
+    - python3 -m scripts.eval_bot --level BossLevel --num_runs 50
+    - python3 -m scripts.eval_bot --level BossLevel --num_runs 50 --advise_mode --non_optimal_steps 100 --bad_action_proba .3
+    # Check that the bot works on a single episode from each level
+    - python3 -m scripts.eval_bot --num_runs 1
+
+    # Quickly test the generation of bot demos
+    - python3 -m scripts.make_agent_demos --env BabyAI-GoToRedBallGrey-v0 --episodes 100 --valid-episodes 32
+
+    # Quickly test the evaluation of bot demos
+    - python3 -m scripts.evaluate --env BabyAI-GoToRedBallGrey-v0 --demos BabyAI-GoToRedBallGrey-v0_agent
+
+    # Quick test for imitation learning
+    - python3 -m scripts.train_il --env BabyAI-GoToRedBallGrey-v0 --demos BabyAI-GoToRedBallGrey-v0_agent --model GoToRedBallGrey-il --val-interval 1 --patience 0 --episodes 100 --val-episodes 50
+
+    # Quickly test the evaluation of models
+    - python3 -m scripts.evaluate --env BabyAI-GoToRedBallGrey-v0 --model GoToRedBallGrey-il
+
+    # Quick test for imitation learning with multi env
+    - python3 -m scripts.train_il --multi-env BabyAI-GoToRedBall-v0 BabyAI-GoToRedBallGrey-v0 --multi-demos BabyAI-GoToRedBallGrey-v0_agent BabyAI-GoToRedBallGrey-v0_agent --val-interval 1 --patience 0 --multi-episodes 100 100 --val-episodes 50
+
+    # Quick test for train_intelligent_expert
+    - python3 -m scripts.train_intelligent_expert --env BabyAI-GoToRedBallGrey-v0 --demos BabyAI-GoToRedBallGrey-v0_agent --val-interval 1 --patience 0 --val-episodes 50 --start-demos 10 --num-eval-demos 5 --phases 2

babyai/CONTRIBUTING.md

@@ -0,0 +1,22 @@
+# Instructions for Contributors
+
+To contribute to this project, you should first create your own fork, and remember to periodically [sync changes from this repository](https://stackoverflow.com/questions/7244321/how-do-i-update-a-github-forked-repository). You can then create [pull requests](https://yangsu.github.io/pull-request-tutorial/) for modifications you have made. Your changes will be tested and reviewed before they are merged into this repository. If you are not familiar with forks and pull requests, we recommend doing a Google or YouTube search to find many useful tutorials on the topic.
+
+Also, you can have a look at the [codebase structure](docs/codebase.md) before getting started.
+
+A suggested flow for contributing would be:
+First, open up a new feature branch to solve an existing bug/issue
+```bash
+$ git checkout -b <feature-branch> upstream/master
+```
+This ensures that the branch is up-to-date with the `master` branch of the main repository, irrespective of the status of your forked repository.
+
+Once you are done making commits of your changes / adding the feature, you can:
+(In case this is the first set of commits from this _new_ local branch)
+```bash
+git push --set-upstream origin 
+```
+(Assuming the name of your forked repository remote is `origin`), which will create a new branch `<feature-branch>`
+tracking your local `<feature-branch>`, in case it hasn't been created already.
+
+Then, create a [pull request](https://help.github.com/en/articles/about-pull-requests) in this repository.

babyai/LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2017, Maxime Chevalier-Boisvert
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name of the copyright holder nor the names of its
+  contributors may be used to endorse or promote products derived from
+  this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.