Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Setting up MARL Benchmark with SMARTS 0.6.1 #1126

Closed
wants to merge 25 commits into from
Closed

Setting up MARL Benchmark with SMARTS 0.6.1 #1126

wants to merge 25 commits into from

Conversation

RutvikGupta
Copy link
Contributor

@RutvikGupta RutvikGupta commented Nov 18, 2021 •
edited by Gamenot
Loading

Restructured baselines/marl_benchmark folder to allow setting up with smarts==0.6.1 and updated package requirements in setup.py and README.md to resolve training procedure errors.

@RutvikGupta RutvikGupta added the bug Something isn't working label Nov 18, 2021
@RutvikGupta RutvikGupta marked this pull request as ready for review November 18, 2021 20:34
@RutvikGupta RutvikGupta marked this pull request as draft November 22, 2021 19:45
@RutvikGupta RutvikGupta marked this pull request as ready for review November 22, 2021 21:20
@RutvikGupta
Copy link
Contributor Author

The changes mentioned by @zyk516 in Issue #1131 have been made here. Now one can follow the instructions of README.md and get the training and evaluate procedures to run for MARL benchmark.

@RutvikGupta RutvikGupta linked an issue Nov 23, 2021 that may be closed by this pull request
how to run benchmark in the CoRL20 paper #1131
Closed
@RutvikGupta RutvikGupta changed the title MARL Benchmark path fixes Setting up MARL Benchmark with SMARTS 0.4.18 Nov 23, 2021
@Gamenot Gamenot mentioned this pull request Dec 28, 2021
Closed
1 task
@Gamenot Gamenot linked an issue Dec 28, 2021 that may be closed by this pull request
Restore MARL Benchmark #1213
Closed
1 task
@Gamenot
Copy link
Collaborator

Gamenot commented Jan 6, 2022

This should now be unblocked since #1138 is merged.

@Gamenot
Copy link
Collaborator

Gamenot commented Jan 7, 2022

It looks like #1230 will possibly block this.

@RutvikGupta RutvikGupta changed the title Setting up MARL Benchmark with SMARTS 0.4.18 Setting up MARL Benchmark with SMARTS 0.5.0 Jan 10, 2022
@RutvikGupta RutvikGupta mentioned this pull request Jan 10, 2022
Closed
@Gamenot
Copy link
Collaborator

Gamenot commented Jan 11, 2022

#1235 has merged in to address a secondary exception can block training.

@RutvikGupta RutvikGupta marked this pull request as draft January 11, 2022 20:08
@Gamenot Gamenot force-pushed the marl_benchmark_path_fixes branch from 2335a22 to 5f4842f Compare May 3, 2022 23:46
@Gamenot Gamenot marked this pull request as ready for review May 4, 2022 00:00
@Gamenot Gamenot changed the title Setting up MARL Benchmark with SMARTS 0.5.1 Setting up MARL Benchmark with SMARTS 0.6.1 May 4, 2022
@Gamenot Gamenot mentioned this pull request May 4, 2022
Closed
@Gamenot
Copy link
Collaborator

Gamenot commented May 4, 2022

Checkpoints are magically working for some reason on my environment...

@Gamenot Gamenot requested review from saulfield, Adaickalavan and sah-huawei and removed request for Gamenot, Adaickalavan, sah-huawei and saulfield May 9, 2022 18:00
saulfield
saulfield approved these changes May 17, 2022
View reviewed changes
Copy link
Contributor

@saulfield saulfield left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look fine to me. I'm just wondering why the directory restructuring was needed. Would relative imports accomplish the same thing?

baselines/marl_benchmark/README.md
- `marl_benchmark/agents/`: YAML files and some RLlib-based policy implementations
- `marl_benchmark/metrics/`: Class definition of metrics (default by a basic Metric class)
- `marl_benchmark/networks/`: Custom network implementations
- `marl_benchmark/communicate.py`: Used for Networked agent learning
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `marl_benchmark/communicate.py`: Used for Networked agent learning
- `communicate.py`: Used for Networked agent learning

baselines/marl_benchmark/README.md
- `marl_benchmark/scenarios/`: Contains three types of scenarios tested in the paper
- `marl_benchmark/wrappers/`: Environment wrappers
- `marl_benchmark/evaluate.py`: The evaluation program
- `marl_benchmark/run.py`: Executes multi-agent training
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like prepending the new folder name just adds noise to the documentation.

@Gamenot
Copy link
Collaborator

Gamenot commented May 17, 2022

@saulfield

Changes look fine to me. I'm just wondering why the directory restructuring was needed. Would relative imports accomplish the same thing?

I am not entirely certain why the directory was restructured. However, it does make the benchmark obviously packageable.

@Adaickalavan
Copy link
Member

This pull request is being closed as the marl_benchmark has been archived and is not expected to receive any further updates. See #2025.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@saulfield saulfield saulfield approved these changes

@Adaickalavan Adaickalavan Awaiting requested review from Adaickalavan

@sah-huawei sah-huawei Awaiting requested review from sah-huawei

Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
4 participants
@RutvikGupta @Gamenot @Adaickalavan @saulfield