Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge ElectricityMarketEnv 2 #7

Open
wants to merge 200 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
200 commits
Select commit Hold shift + click to select a range
c698514
Add files via upload
rajeev-datta Jun 25, 2022
1629d26
updated battery storage env
Jun 25, 2022
a3e0b18
Merge branch 'electricitymarket' of https://github.com/chrisyeh96/sus…
Jun 25, 2022
c2111bf
small changes and data analysis code
Jun 25, 2022
b946cfd
updated battery storage env
Jun 25, 2022
cdfd474
Add files via upload
rajeev-datta Jun 25, 2022
b6af985
small changes and data analysis code
Jun 25, 2022
a5a9a09
Merge branch 'electricitymarket' of https://github.com/chrisyeh96/sus…
Jun 25, 2022
9ec520f
added data folder
Jun 25, 2022
a33dced
updated data reading code
Jun 25, 2022
d783cbf
testing environment
Jun 28, 2022
1f44472
replacing data file
Jun 29, 2022
2332e89
fixed battery env
Jun 29, 2022
b14e04d
cleared outputs
Jun 29, 2022
6b11f12
fixed env and wrote ddpg code
Jul 4, 2022
c9f5d39
ran ddpg code
Jul 6, 2022
39d36f9
bar graph code
Jul 7, 2022
bf9694f
more data analysis
Jul 10, 2022
7fd4d7d
remove ignored model files
Jul 12, 2022
9ab2065
remove ignored model files
Jul 12, 2022
68f7951
remove ignored model files
Jul 12, 2022
b824721
remove ignored model files
Jul 12, 2022
2d6e82f
renaming files
Jul 12, 2022
33be77f
wrotee constructors
Jul 13, 2022
c2eef6c
adding MO problem to env
Jul 13, 2022
091e46f
first draft of env
Jul 13, 2022
701c45c
debugging
Jul 15, 2022
affaa0f
debugging env
Jul 15, 2022
8a09b01
temp edits by Chris
chrisyeh96 Jul 15, 2022
5eba907
check env runs
Jul 16, 2022
6f36db3
random actions
Jul 16, 2022
26d959d
debugging single agent
Jul 18, 2022
84d0f78
infeasible constraints
Jul 19, 2022
e86bea7
Simplify battery gym
chrisyeh96 Jul 19, 2022
624096b
debugging tests
Jul 19, 2022
dd40d0e
fixed tests
Jul 20, 2022
4e7d5ae
changing reset
Jul 20, 2022
5397b9f
added costs assurances
Jul 20, 2022
fe44dc6
running PPO on environment
Jul 21, 2022
b2b9218
including load data
Jul 24, 2022
f4a56fd
random actions work with load data
Jul 25, 2022
a6b584c
analyzing carbon data and using net demand
Jul 27, 2022
9d5f34c
add requests to env.yml
chrisyeh96 Jul 27, 2022
81202ff
retrieving moer values
Jul 28, 2022
b3a033e
Merge branch 'electricitymarket' of https://github.com/chrisyeh96/sus…
Jul 28, 2022
d0c1052
calculated offline optimal
Jul 28, 2022
9667b09
making env more realistic
Jul 29, 2022
28d5546
random actions running on more realistic environment w/ forecasts
Jul 30, 2022
5762e88
made data plots
Aug 1, 2022
b48a6ae
trying to find dist shift
Aug 3, 2022
71fb16a
trying to find dist shift
Aug 3, 2022
85aeb85
Clean up CAISO demand + forecast data
chrisyeh96 Aug 3, 2022
54a2f8a
Chris cleanup
chrisyeh96 Aug 3, 2022
7b25d67
Cleaner variables
chrisyeh96 Aug 4, 2022
07e212c
cleaned up code... will not verify seed logic and run PPO on May 2020
Aug 4, 2022
d979a63
debugging out of dist evaluation
Aug 5, 2022
30f3dcd
ran PPO and A2C
Aug 6, 2022
ecd0f59
vectorize battery charge update
chrisyeh96 Aug 9, 2022
935c801
minor updates
chrisyeh96 Aug 9, 2022
1d774a6
use self.obs
chrisyeh96 Aug 9, 2022
9da4b25
fix mypy errors
chrisyeh96 Aug 9, 2022
eeceb1c
fixed environment
Aug 10, 2022
c58dcc2
fixed environment
Aug 10, 2022
e48b067
fixed environment
Aug 10, 2022
ca05d1e
made some plots
Aug 10, 2022
7fc111d
added demo
Aug 10, 2022
4c94d9b
Merge remote-tracking branch 'origin/main' into electricitymarket
Aug 11, 2022
0e7d2f8
changed solver to mosek
Aug 11, 2022
c750f73
Create battery folder
chrisyeh96 Aug 12, 2022
30cfff0
break out reward components
chrisyeh96 Aug 12, 2022
83f9c72
fixed environment
Aug 12, 2022
22712dc
fixed environment
Aug 12, 2022
bcc78b4
added terminal reward
Aug 12, 2022
90d9e09
Merge remote-tracking branch 'origin/main' into electricitymarket
Aug 12, 2022
af39dac
use MOER loader
Aug 13, 2022
cdda90f
fixed random actions w/ terminal reward
Aug 13, 2022
c5010e4
changed training scripts
Aug 14, 2022
cde0973
making plots
Aug 15, 2022
5de9265
finished presentation plots
Aug 16, 2022
c50580a
no change
Aug 17, 2022
ca882d6
moved files and plotted prices and state of charge
Aug 20, 2022
0dc2819
added panda power to RL env
Aug 23, 2022
d3f8cde
adding plot helper fxn
Aug 25, 2022
8e42946
factoring out plotting code
Aug 26, 2022
b73edd3
added reward plots
Aug 26, 2022
5e13659
changed reward plots and printed soc to debug
Aug 27, 2022
6c429d8
debugged env and retraining models
Aug 28, 2022
a8fe4f5
debugged env and retraining models
Aug 28, 2022
d17f8c5
debugged env and retraining models
Aug 28, 2022
0db23f7
trained models
Aug 28, 2022
d000fbd
remade graphs
Aug 28, 2022
b202764
fixed per step offline reward
Aug 31, 2022
6b9dc75
fixing plots
Sep 1, 2022
71774d6
fixed plots
Sep 1, 2022
1160cac
retraining with added penalty
Sep 2, 2022
5a16a3a
retraining with added penalty
Sep 2, 2022
d58eb20
Merge
chrisyeh96 Sep 2, 2022
901d1e0
updated offline optimal code
Sep 2, 2022
75853bb
refactor offline optimal
chrisyeh96 Sep 2, 2022
d7b020a
trained models with terminal cost
Sep 3, 2022
94c991d
updated results with terminal cost
Sep 4, 2022
969f9eb
trained with increased gamma
Sep 5, 2022
1144438
rescaling action space
Sep 6, 2022
6463b20
tracked actions taken during training
Sep 7, 2022
3b1582e
tracked actions taken during training
Sep 8, 2022
e1a939e
added logging action callback
Sep 8, 2022
20b4182
Add discrete action space to electricity market
chrisyeh96 Sep 8, 2022
33f05b6
Cleanup
chrisyeh96 Sep 8, 2022
20f6119
running 250k training steps
Sep 9, 2022
fe1fbc0
ran models on discrete action space
Sep 10, 2022
d9ddeab
running optimal vs discrete
Sep 11, 2022
96bd5f5
Merge branch 'main' into electricitymarket
chrisyeh96 Sep 12, 2022
2e7cdce
Document generators + battery system parameters
chrisyeh96 Sep 13, 2022
e16d26f
ran dqn on discrete action space
Sep 13, 2022
78f9cd4
Merge branch 'electricitymarket' of https://github.com/chrisyeh96/sus…
Sep 13, 2022
f80725c
Update actions to be (charge cost, discharge cost)
chrisyeh96 Sep 14, 2022
ed1f641
Update charging and discharging efficiency
chrisyeh96 Sep 14, 2022
f4e60ab
Fix approx offline optimal to account for charging efficiency
chrisyeh96 Sep 14, 2022
a302f94
Use 3-action discretization
chrisyeh96 Sep 14, 2022
afb8cfe
added bar graph and debugged
Sep 14, 2022
67f55d2
updated offline optimal plotting code
Sep 14, 2022
5014450
Fix offline optimal
chrisyeh96 Sep 14, 2022
0d651aa
Update plot utils
chrisyeh96 Sep 14, 2022
ceb2cde
Updates
chrisyeh96 Sep 15, 2022
96011b4
Move price to state
chrisyeh96 Sep 15, 2022
3818abd
Return offline opt results via dict
chrisyeh96 Sep 15, 2022
a5ab0c9
Typo
chrisyeh96 Sep 15, 2022
f3766a2
update typing annotations
chrisyeh96 Sep 15, 2022
78e9997
Enable setting bats_init_energy
chrisyeh96 Sep 16, 2022
077f320
added follow optimal plotting code
Sep 17, 2022
d041ad4
Fix plot_utils
chrisyeh96 Sep 17, 2022
0c189d2
Unify training code
chrisyeh96 Sep 17, 2022
a57edb4
Update data plots
chrisyeh96 Sep 17, 2022
4472a17
ran models... debugging plotting code
Sep 17, 2022
e0b12ac
Many updates
chrisyeh96 Sep 20, 2022
9896b8f
More significant changes
chrisyeh96 Sep 20, 2022
037b7bc
analysis notebooks
chrisyeh96 Sep 20, 2022
727890e
Figures
chrisyeh96 Sep 21, 2022
fe5a535
cleaning up...
Nov 21, 2022
0697dd2
cleaning up
Nov 21, 2022
33f3526
refactoring test code and cleaning up
Nov 28, 2022
29d2de2
altered training script and revisted plotting script
Dec 5, 2022
fd5f109
updated plot code and notebooks
Jan 6, 2023
8e80e60
ignoring model_analysis.ipynb file
Jan 6, 2023
24d037b
wrote draft for PTDF matrix generation
Jan 11, 2023
92a2d74
current draft of congestion
Feb 17, 2023
ffd19da
debugged reset method
Mar 22, 2023
094442c
debugging dispatch problem
Mar 27, 2023
1f06460
Debugged infeasible optimization problem
chrisyeh96 Mar 27, 2023
44f7eeb
cleaned up code and debugged step function
Apr 2, 2023
0695b5d
congested network plotting and analysis
Apr 14, 2023
a732f5f
Use MILP to limit battery to either charging or discharging (but not …
chrisyeh96 Apr 14, 2023
51a3c48
Merge remote-tracking branch 'origin' into electricitymarket2
rajeev-datta May 17, 2023
65ee59b
Update .gitignore
rajeev-datta May 17, 2023
c666c26
RLLib training code + MPC
rajeev-datta May 31, 2023
6aef37c
Better docs
chrisyeh96 May 31, 2023
dceae59
updated training script code
rajeev-datta Jun 5, 2023
7f3135f
updated training script code
rajeev-datta Jun 5, 2023
b825636
Merge branch 'electricitymarket2' of https://github.com/chrisyeh96/su…
rajeev-datta Jun 5, 2023
c35253c
added bash training script and updated rllib training script
rajeev-datta Jun 6, 2023
fb83d4c
updated bash training script
rajeev-datta Jun 6, 2023
f601909
updated wrapped code and training script
rajeev-datta Jun 6, 2023
bee6a1a
updated rllib training
rajeev-datta Jun 6, 2023
9ceca85
Set prices 0 in case dual_value is None
rajeev-datta Jun 7, 2023
f416e85
Merge branch 'main' into electricitymarket2
chrisyeh96 Jun 7, 2023
d14c416
Merge branch 'main' into electricitymarket2
chrisyeh96 Jun 7, 2023
8e9f2fe
Fix rescale action
rajeev-datta Jun 7, 2023
13300c7
Merge branch 'electricitymarket2' of https://github.com/chrisyeh96/su…
rajeev-datta Jun 7, 2023
0ff46e6
Fix scaling actions, vf_clip_param
Jun 7, 2023
0f1883f
working env_em.yml
Jun 7, 2023
1d7216e
Restructure electricitynarketenv
rajeev-datta Jun 9, 2023
d5677fb
fixed congestion environment
rajeev-datta Jun 14, 2023
81e42eb
fixed scaling for congestion environment
rajeev-datta Jun 14, 2023
0d922a5
Merge branch 'main' into electricitymarket2
chrisyeh96 Aug 15, 2023
e560c0e
Update documentation + LICENSE
chrisyeh96 Aug 18, 2023
5a67800
Separate out network and market operator
chrisyeh96 Sep 2, 2023
4478291
Update noncongested env to gymnasium
chrisyeh96 Sep 2, 2023
f2d08a0
Merge remote-tracking branch 'origin/main' into electricitymarket2
chrisyeh96 Sep 2, 2023
906ff67
Merge remote-tracking branch 'origin/main' into electricitymarket2
chrisyeh96 Sep 2, 2023
e0ec849
Remove case24_ieee_rts.py
chrisyeh96 Sep 2, 2023
8284317
Initial package build attempt. Add contributing guide
chrisyeh96 Sep 9, 2023
523a095
Merge remote-tracking branch 'origin/main' into electricitymarket2
chrisyeh96 Sep 9, 2023
5a30484
Merge remote-tracking branch 'origin/main' into electricitymarket2
chrisyeh96 Sep 9, 2023
95d74c0
Merge remote-tracking branch 'origin/main' into electricitymarket2
chrisyeh96 Sep 9, 2023
8570466
Merge remote-tracking branch 'origin/main' into electricitymarket2
chrisyeh96 Sep 13, 2023
0404316
Add electricitymarket to pyproject.toml
chrisyeh96 Oct 12, 2023
3f12ada
Rename electricitymarket data folder
chrisyeh96 Oct 12, 2023
9fe7a5a
Add forecast_steps param to MOERLoader.retrieve()
chrisyeh96 Oct 12, 2023
eff8e1d
Rename electricitymarket wrappers
chrisyeh96 Oct 12, 2023
dbe9399
Separate IEEE24 case file into two scripts
chrisyeh96 Oct 12, 2023
6298e9f
Update __init__.py files
chrisyeh96 Oct 12, 2023
e0bd9be
Update market_operator.py and networks/base.py
chrisyeh96 Oct 12, 2023
0e25a76
Updated ElectricityMarketEnv
chrisyeh96 Oct 12, 2023
4e55c71
Add ElectricityMarketPlot
chrisyeh96 Oct 12, 2023
9fbee14
Refactor
chrisyeh96 Oct 12, 2023
ff7b0b4
Merge remote-tracking branch 'origin/main' into electricitymarket2
chrisyeh96 Nov 2, 2023
8ab0828
Fix pyproject.toml
chrisyeh96 Nov 2, 2023
90a60e1
Merge remote-tracking branch 'origin/main' into electricitymarket2
chrisyeh96 Nov 25, 2023
37f5ad8
Remove test notebook
chrisyeh96 Nov 25, 2023
e51ef33
Delete unused notebooks
chrisyeh96 Nov 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@
.vscode/
sandbox/
logs/
*.zip
*.code-workspace
examples/DQN*
examples/SAC*
examples/PPO*
examples/A2C*
examples/old
examples/train_DQN.py
examples/test.ipynb
examples/intermreward
examples/congested_intermreward/
examples/model_analysis.ipynb
*.png
sustaingym/tests*
oldlogs/
log/
plots/
Expand Down
38 changes: 38 additions & 0 deletions env_em.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Install or update environment
# conda env update --file env_em.yml --prune
#
# Remove environment
# conda remove --name sustaingym_em --all
#
# Last updated: 2023-08-16

name: sustaingym_em
channels:
- pytorch # for pytorch
- nvidia # for pytorch
- mosek # for mosek
- conda-forge
dependencies:
- python=3.11
- cvxpy=1.3.2
- flake8
- gymnasium=0.26.3
- ipympl=0.9.3 # for Jupyter / VSCode notebooks
- ipykernel # for Jupyter / VSCode notebooks
- matplotlib=3.7.2
- mosek=10.1.8
- mypy
- numpy=1.24.3 # TensorFlow 2.13 requires numpy >= 1.22, <= 1.24.3
- pandas
- pip
- pytorch=2.0.1
- pytorch-cuda=11.8
- pytz=2023.3
- seaborn=0.12.2
- tqdm=4.66.1

- pip:
# - stable_baselines3>=2.0.0 # stablebaselines3>=2.0.0 requires gymnasium>=0.28.1 (https://github.com/DLR-RM/stable-baselines3/blob/master/setup.py)
- pettingzoo==1.22.4 # latest version compatible with gymnasium==0.26.3 (https://github.com/Farama-Foundation/PettingZoo/blob/1.22.4/pyproject.toml)
- "ray[rllib]==2.6.3" # requires gymnasium==0.26.3 (https://github.com/ray-project/ray/blob/ray-2.6.3/python/requirements.txt)
- tensorflow==2.13.0
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading