Modify Industry Sector Ratios #293

toniseibold · 2024-12-02T19:54:09Z

This PR allows to adjust the industry sector ratios for Germany.
A new config config[industry][sector_ratios_fraction_future_DE] is introduced to specify the German industrial transition which will be completed in 2045.
The rule modify_sector_ratios reads in the existing sector ratios for Europe and calculates a faster transition for Germany. The adjusted sector ratios are then written into the file industry_sector_ratios_{planning_horizons}-modified.csv.
This modified file is fed into the pypsa-eur rule build_industrial_energy_demand_per_node so the new sector ratios propagate through the entire industrial sector rules.
As a result, Germany reaches net zero in the industrial sector which means that there is no more coal and coke demand by 2045.

Before asking for a review for this PR make sure to complete the following checklist:

Workflow with target rule ariadne_all completes without errors
The logic of export_ariadne_variables has been adapted to the changes
not applicable
One or several figures that validate the changes in the PR have been posted as a comment
A brief description of the changes has been added to Changelog.md
The latest main has been merged into the PR
The config has a new prefix of the format YYYYMMDDdescriptive_title

for more information, see https://pre-commit.ci

github-actions · 2024-12-02T19:56:26Z

Validator Report

I am the Validator. Download all artifacts here.
I'll be back and edit this comment for each new commit.

⚠️ Config changes detected!

Results may differ due to these changes:

diff --git a/config/config.yaml b/config/config.yaml
index 713449c..dd37582 100644
--- a/config/config.yaml
+++ b/config/config.yaml
@@ -4,7 +4,7 @@
 
 # docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#run
 run:
-  prefix: 20241121-fix-missing-gas-capa
+  prefix: 20241202-modify-sector-ratios
 
   name:
   # - CurrentPolicies
@@ -62,7 +62,7 @@ scenario:
   ll:
   - vopt
   clusters:
-  - 27 #current options: 27, 49
+  - 49 #current options: 27, 49
   opts:
   - ''
   sector_opts:
@@ -206,38 +206,38 @@ clustering:
   # print(fw.div(fw.sum()).subtract(5e-5).round(4).to_dict().__repr__().replace(",","\n"))
   focus_weights:
   # 27 nodes: 8 for Germany, 3 for Italy, 2 each for Denmark, UK and Spain, 1 per each of other 10 "Stromnachbarn"
-    'DE': 0.2966
-    'AT': 0.0370
-    'BE': 0.0370
-    'CH': 0.0370
-    'CZ': 0.0370
-    'DK': 0.0741
-    'FR': 0.0370
-    'GB': 0.0741
-    'LU': 0.0370
-    'NL': 0.0370
-    'NO': 0.0370
-    'PL': 0.0370
-    'SE': 0.0370
-    'ES': 0.0741
-    'IT': 0.1111
+    # 'DE': 0.2966
+    # 'AT': 0.0370
+    # 'BE': 0.0370
+    # 'CH': 0.0370
+    # 'CZ': 0.0370
+    # 'DK': 0.0741
+    # 'FR': 0.0370
+    # 'GB': 0.0741
+    # 'LU': 0.0370
+    # 'NL': 0.0370
+    # 'NO': 0.0370
+    # 'PL': 0.0370
+    # 'SE': 0.0370
+    # 'ES': 0.0741
+    # 'IT': 0.1111
   # high spatial resolution: change clusters to 49
   # 49 nodes: 30 for Germany, 3 for Italy, 2 each for Denmark, UK and Spain, 1 per each of other 10 "Stromnachbarn"
-    # 'DE': 0.6124
-    # 'AT': 0.0204
-    # 'BE': 0.0204
-    # 'CH': 0.0204
-    # 'CZ': 0.0204
-    # 'DK': 0.0408
-    # 'FR': 0.0204
-    # 'GB': 0.0408
-    # 'LU': 0.0204
-    # 'NL': 0.0204
-    # 'NO': 0.0204
-    # 'PL': 0.0204
-    # 'SE': 0.0204
-    # 'ES': 0.0408
-    # 'IT': 0.0612
+    'DE': 0.6124
+    'AT': 0.0204
+    'BE': 0.0204
+    'CH': 0.0204
+    'CZ': 0.0204
+    'DK': 0.0408
+    'FR': 0.0204
+    'GB': 0.0408
+    'LU': 0.0204
+    'NL': 0.0204
+    'NO': 0.0204
+    'PL': 0.0204
+    'SE': 0.0204
+    'ES': 0.0408
+    'IT': 0.0612
   temporal:
     resolution_sector: 365H
 
@@ -349,6 +349,13 @@ sector:
 
 # docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#industry
 industry:
+  sector_ratios_fraction_future_DE:
+    2020: 0.0
+    2025: 0.1
+    2030: 0.3
+    2035: 0.6
+    2040: 0.9
+    2045: 1.0
   ammonia: false
   St_primary_fraction:
     2020: 0.6

Ariadne Variables

Comparison

	NRMSE	Pearson
Capacity\|Electricity\|Storage Reservoir\|Pump Hydro	1.000	0.000
Capacity\|Hydrogen\|Gas	0.807	0.975
Capacity\|Hydrogen\|Gas\|w/o CCS	0.806	0.975
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Kernnetz\|NOT-PCI+IPCEI	0.577	0.000
Investment\|Energy Supply\|Electricity\|Biomass\|Solids	0.544	1.000
Investment\|Energy Supply\|Electricity\|Biomass\|w/o CCS	0.544	1.000
Investment\|Energy Supply\|Heat\|Biomass	0.544	1.000
Investment\|Energy Supply\|Heat\|Biomass\|w/o CCS	0.544	1.000
Investment\|Energy Supply\|Electricity\|Biomass	0.544	1.000
Secondary Energy\|Electricity\|Hydro	0.458	0.502
Capacity\|Electricity\|Transmission\|AC	0.431	0.967
Investment\|Energy Supply\|Electricity\|Gas	0.408	0.000
Investment\|Energy Supply\|Electricity\|Gas\|w/o CCS	0.408	0.000
Capacity Additions\|Electricity\|Gas	0.408	0.388
Capacity Additions\|Electricity\|Gas\|w/o CCS	0.408	0.387
Investment\|Energy Supply\|Electricity\|Transmission\|AC\|Offshore	0.407	0.353
Investment\|Energy Supply\|Heat\|Gas	0.406	0.217
Capacity Additions\|Heat\|Gas	0.404	0.210
Secondary Energy\|Electricity\|Oil	0.386	0.687
Capacity Additions\|Heat\|Biomass\|w/o CCS	0.385	1.000
Capacity Additions\|Electricity\|Biomass\|Solids	0.385	1.000
Capacity Additions\|Electricity\|Biomass	0.385	1.000
Capacity Additions\|Electricity\|Biomass\|w/o CCS	0.385	1.000
Capacity Additions\|Heat\|Biomass	0.385	1.000
Price\|Primary Energy\|Oil	0.381	0.000
Price\|Secondary Energy\|Liquids\|Hydrogen	0.375	0.342
Secondary Energy\|Heat\|Oil	0.371	0.663
Price\|Final Energy\|Industry\|Solids\|Coal	0.370	0.000
Primary Energy\|Oil\|Heat	0.368	0.466
Capacity\|Electricity\|Storage Converter	0.362	0.558
Investment\|Energy Supply\|Electricity\|Transmission\|AC\|NEP\|Onshore	0.343	0.852
Length Additions\|Electricity\|Transmission\|AC\|NEP	0.337	0.735
Capacity Additions\|Electricity\|Transmission\|AC\|NEP	0.337	0.735
Length Additions\|Electricity\|Transmission\|AC	0.337	0.606
Capacity Additions\|Electricity\|Transmission\|AC	0.337	0.606
Price\|Secondary Energy\|Gases\|Hydrogen	0.335	0.000
Capacity\|Electricity\|Transmission\|AC\|NEP	0.331	0.749
Capacity\|Electricity\|Gas\|OC	0.329	0.908
Investment\|Energy Supply\|Electricity\|Transmission\|AC\|Onshore	0.327	0.880
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Endogen\|New-build	0.317	0.849
Investment\|Energy Supply\|Electricity\|Solar\|PV\|Open Field	0.275	0.760
Capacity\|Electricity\|Transmission	0.272	0.995
Final Energy\|Carbon Dioxide Removal\|Electricity	0.271	1.000
Carbon Sequestration\|DACCS	0.270	1.000
Capacity Additions\|Methanol	0.258	0.716
Final Energy\|Industry excl Non-Energy Use\|Liquids\|Efuel	0.251	0.947
Capacity Additions\|Electricity\|Solar\|PV\|Open Field	0.250	0.805
Capacity\|Heat\|Non-Renewable Waste	0.249	0.999
Capacity\|Electricity\|Non-Renewable Waste	0.248	0.999
Investment\|Energy Supply\|Electricity\|Transmission\|DC\|Offshore	0.235	0.811
Capacity\|Heat\|Gas	0.233	0.986
Final Energy\|Carbon Dioxide Removal	0.232	1.000
Secondary Energy\|Hydrogen	0.229	0.900
Investment\|Energy Supply\|Electricity\|Wind\|Offshore	0.228	0.810
Investment\|Energy Supply\|Electricity\|Wind Offshore	0.228	0.810
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Endogen	0.223	0.938
Investment\|Energy Supply\|Electricity\|Transmission\|AC\|Übernahme\|Startnetz Delta	0.223	1.000
Capacity\|Hydrogen\|Electricity	0.220	0.981
Secondary Energy\|Heat\|Hydrogen	0.212	0.997
Secondary Energy Input\|Hydrogen\|Heat	0.212	0.997
Secondary Energy\|Electricity\|Hydrogen	0.212	0.997
Secondary Energy Input\|Hydrogen\|Electricity	0.212	0.997
Secondary Energy\|Hydrogen\|Electricity	0.210	0.955
Capacity Additions\|Electricity\|Transmission	0.210	0.875
Secondary Energy Input\|Electricity\|Hydrogen	0.209	0.960
Investment\|Energy Supply\|Electricity\|Transmission\|Offshore	0.208	0.838
Capacity Additions\|Electricity\|Wind\|Offshore	0.204	0.834
Capacity\|Hydrogen	0.203	0.978
Investment\|Energy Supply\|Methanol	0.201	0.863
Capacity Additions\|Liquids	0.200	0.826
Capacity Additions\|Liquids\|Hydrogen	0.200	0.843
Primary Energy\|Hydro	0.200	0.901
Investment\|Energy Supply\|Hydrogen\|Electrolysis	0.199	0.976
Investment\|Energy Supply\|Hydrogen\|Electricity	0.199	0.976
Trade\|Secondary Energy\|Electricity\|Gross Import\|Volume	0.199	0.975
Capacity\|Heat\|Hydrogen	0.197	1.000
Capacity\|Electricity\|Hydrogen\|CC	0.197	1.000
Capacity\|Electricity\|Hydrogen	0.197	1.000
Investment\|Energy Supply\|Heat\|Hydrogen	0.196	1.000
Investment\|Energy Supply\|Electricity\|Hydrogen\|CC	0.196	1.000
Investment\|Energy Supply\|Electricity\|Hydrogen	0.196	1.000
Secondary Energy\|Heat	0.188	0.837
Capacity\|Hydrogen\|Transmission\|Kernnetz	0.188	0.998
Trade\|Secondary Energy\|Methanol\|Volume	0.184	0.936
Final Energy\|Carbon Dioxide Removal\|Heat	0.184	1.000
Secondary Energy\|Electricity\|Curtailment	0.183	0.929
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Kernnetz\|Retrofitted	0.180	0.996
Secondary Energy\|Heat\|Other	0.180	0.981
Capacity\|Heat\|Processes	0.180	0.988
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Retrofitted	0.174	0.993
Investment\|Energy Supply\|Electricity\|Non-Renewable Waste	0.174	0.998
Investment\|Energy Supply\|Heat\|Non-Renewable Waste	0.174	0.998
Investment\|Energy Supply\|Heat\|Processes	0.172	0.986
Final Energy\|Gases\|Biomass	0.171	0.971
Final Energy\|Residential and Commercial\|Gases\|Biomass	0.171	0.940
Emissions\|CO2\|Efuels\|Gases	0.169	0.930
Capacity\|Electricity\|Solar\|PV\|Open Field	0.167	0.963
Secondary Energy\|Electricity\|Storage Losses	0.167	0.940
Capacity\|Hydrogen\|Transmission	0.163	0.997
Primary Energy\|Biomass\|Solids	0.163	0.850
Secondary Energy\|Solids\|Biomass	0.163	0.850
Capacity\|Electricity\|Transmission\|NEP	0.160	0.973
Capacity\|Hydrogen\|Transmission\|Endogenous	0.159	0.995
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Endogen\|Retrofitted	0.158	0.967
Capacity Additions\|Heat	0.157	0.954
Capacity Additions\|Hydrogen	0.156	0.943
Capacity Additions\|Electricity\|Transmission\|NEP	0.155	0.976
Secondary Energy\|Electricity\|Waste	0.155	0.999
Primary Energy\|Waste\|Electricity	0.155	0.999
Final Energy\|Waste	0.155	0.999
Primary Energy\|Waste	0.155	0.999
Secondary Energy\|Heat\|Waste	0.155	0.999
Primary Energy\|Waste\|Heat	0.155	0.999
Capacity Additions\|Hydrogen\|Electricity	0.154	0.952
Capacity Additions\|Hydrogen\|Transmission\|Endogenous	0.154	0.968
Final Energy\|Industry excl Non-Energy Use\|Gases\|Biomass	0.152	0.992
Capacity\|Heat	0.152	0.988
Secondary Energy\|Electricity\|Biomass\|Solid	0.150	0.999
Secondary Energy\|Electricity\|Biomass	0.149	0.984
Secondary Energy\|Electricity\|Biomass\|w/o CCS	0.149	0.984
Capacity Additions\|Heat\|Processes	0.148	0.969
Secondary Energy Input\|Electricity\|Liquids	0.147	0.956
Secondary Energy\|Methanol	0.147	0.956
Investment\|Energy Supply\|Liquids\|Hydrogen	0.145	0.924
Final Energy\|Industry excl Non-Energy Use\|Liquids\|Biomass	0.144	0.942
Secondary Energy\|Heat\|Gas	0.144	0.989
Final Energy\|Solids\|Biomass	0.144	0.888
Capacity\|Methanol	0.141	0.956
Final Energy\|Liquids\|Efuel	0.140	0.982
Capacity Additions\|Heat\|Hydrogen	0.139	1.000
Capacity Additions\|Electricity\|Hydrogen\|CC	0.139	1.000
Capacity Additions\|Electricity\|Hydrogen	0.139	1.000
Capacity Additions\|Heat\|Storage Reservoir	0.135	0.954
Capacity Additions\|Electricity\|Solar\|PV	0.133	0.927
Capacity Additions\|Electricity\|Solar	0.133	0.927
Final Energy\|Transportation\|Liquids\|Efuel	0.132	0.983
Investment\|Energy Supply\|Heat\|Storage Reservoir	0.130	0.972
Capacity Additions\|Electricity\|Wind\|Onshore	0.129	0.922
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Kernnetz	0.129	0.998
Length Additions\|Electricity\|Transmission	0.127	0.963
Capacity Additions\|Electricity\|Non-Renewable Waste	0.123	0.997
Capacity Additions\|Heat\|Non-Renewable Waste	0.123	0.997
Final Energy\|Heat	0.123	0.977
Secondary Energy\|Heat\|Biomass	0.120	0.996
Investment\|Energy Supply\|Electricity\|Transmission\|DC	0.119	0.952
Capacity\|Heat\|Storage Reservoir	0.119	0.991
Capacity\|Heat\|Storage Converter	0.117	0.988
Capacity Additions\|Heat\|Storage Converter	0.117	0.988
Secondary Energy\|Heat\|Electricity\|Resistive	0.116	0.975
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Kernnetz\|IPCEI	0.116	1.000
Investment\|Energy Supply\|Liquids	0.115	0.951
Investment\|Energy Supply\|Electricity\|Wind Onshore	0.115	0.951
Investment\|Energy Supply\|Electricity\|Wind\|Onshore	0.115	0.951
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Kernnetz\|PCI	0.115	0.998
Secondary Energy\|Liquids\|Hydrogen	0.114	0.976
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution\|Kernnetz\|PCI+IPCEI	0.113	0.998
Secondary Energy Input\|Hydrogen\|Liquids	0.112	0.978
Emissions\|CO2\|Model\|CCU	0.112	0.971
Trade\|Primary Energy\|Biomass\|Volume	0.111	0.991
Investment\|Energy Supply\|Hydrogen\|Transmission and Distribution	0.110	0.997
Investment\|Energy Supply\|Hydrogen\|Transmission	0.110	0.997
Primary Energy\|Biomass\|Heat	0.108	0.995
Final Energy\|Residential and Commercial\|Solids\|Biomass	0.107	0.940
Final Energy\|Residential and Commercial\|Solids	0.107	0.940
Primary Energy\|Gas\|Heat	0.106	0.991
Final Energy\|Industry excl Non-Energy Use	0.106	0.986
Investment\|Energy Supply\|Electricity\|Transmission\|AC	0.106	0.988
Secondary Energy\|Electricity\|Coal\|Hard Coal	0.105	1.000
Investment\|Energy Supply\|Hydrogen	0.105	0.996
Investment\|Energy Supply\|Heat\|Resistive heater	0.104	0.977
Primary Energy\|Oil\|Electricity	0.104	0.965

NRMSE: Normalized (combined-min-max) Root Mean Square Error
Pearson: Pearson correlation coefficient
Threshold: NRMSE > 0.1
Only variables reaching the threshold are shown. Find the equivalent plot for all of them below.

Plots

Main branch	Feature branch

⚠️ Note: Only the first 20 variables are shown, but more are above the threshold. Find all of them in the artifacts.

General

Plots comparison

Main branch	Feature branch

Files comparison

	Status	NRMSE	MAE (norm)
csvs/prices.csv	⚠️Changed	0.020	0.07
csvs/curtailment.csv	⚠️Changed	0.000	0.13
csvs/price_statistics.csv	⚠️Changed	0.077	0.13
csvs/cumulative_cost.csv	⚠️Changed	0.001	0.10
csvs/metrics.csv	⚠️Changed	0.001	0.09
csvs/nodal_cfs.csv	⚠️ NaN mismatch
csvs/supply.csv	⚠️ NaN mismatch
csvs/nodal_supply_energy.csv	⚠️ NaN mismatch
csvs/capacities.csv	⚠️ NaN mismatch
csvs/costs.csv	⚠️ NaN mismatch
csvs/nodal_costs.csv	⚠️ NaN mismatch
csvs/nodal_capacities.csv	⚠️ NaN mismatch
csvs/cfs.csv	⚠️ NaN mismatch
csvs/supply_energy.csv	⚠️ NaN mismatch
csvs/energy.csv	⚠️ NaN mismatch
csvs/market_values.csv	✅ Almost equal	0.096	0.04
csvs/weighted_prices.csv	✅ Equal

NRMSE: Normalized (combined-min-max) Root Mean Square Error
MAE (norm): Mean Absolute Error on normalized data (min-max)
Status Threshold: MAE (norm) > 0.05 and NRMSE > 0.3

Model Metrics

Benchmarks

Comparing industry_sector_ratios_de (d086d39) with main (496f95e).
Branch is 4 commits ahead and 0 commits behind.
Last updated on 2024-12-03 09:11:22 CET.

lindnemi

This PR seems to a bit more complex, lets briefly discuss it via zoom / in person

lindnemi · 2024-12-03T16:02:41Z

config/config.yaml

@@ -62,7 +62,7 @@ scenario:
  ll:
  - vopt
  clusters:
-  - 27 #current options: 27, 49
+  - 49 #current options: 27, 49


please revert this before merging

toniseibold and others added 4 commits December 2, 2024 11:03

first stab at modifying sector ratios

fa19075

refining sector ratio modification

c11265e

[pre-commit.ci] auto fixes from pre-commit.com hooks

1ee534e

for more information, see https://pre-commit.ci

document changes

d086d39

toniseibold marked this pull request as ready for review December 2, 2024 20:07

toniseibold requested a review from lindnemi December 2, 2024 20:07

lindnemi reviewed Dec 3, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modify Industry Sector Ratios #293

Modify Industry Sector Ratios #293

toniseibold commented Dec 2, 2024 •

edited

Loading

github-actions bot commented Dec 2, 2024 •

edited

Loading

lindnemi left a comment

lindnemi Dec 3, 2024

Modify Industry Sector Ratios #293

Are you sure you want to change the base?

Modify Industry Sector Ratios #293

Conversation

toniseibold commented Dec 2, 2024 • edited Loading

github-actions bot commented Dec 2, 2024 • edited Loading

Validator Report

lindnemi left a comment

Choose a reason for hiding this comment

lindnemi Dec 3, 2024

Choose a reason for hiding this comment

toniseibold commented Dec 2, 2024 •

edited

Loading

github-actions bot commented Dec 2, 2024 •

edited

Loading