fix: extra functionality not called

[lkstrp]

• Works with just moving the file to the pypsa-eur placeholder. Not sure why it was setup like this in the first place

Checklist

• Workflow with target rule ariadne_all completes without errors
• The latest main has been merged into the PR

[github-actions]

Validator Report

I am the Validator. Download all artifacts here.
I'll be back and edit this comment for each new commit.

Ariadne Variables

Comparison

	NRMSE	Pearson
Price|Secondary Energy|Gases|Hydrogen	0.627	0.000
System Cost|OPEX|Electricity|Storage Reservoir|Hydro Dam Reservoir	0.541	-0.838
System Cost|OPEX|Electricity|Storage Converter|Hydro Dam Reservoir	0.541	-0.838
System Cost|OPEX|Electricity|Hydro	0.529	-0.478
System Cost|OPEX|Electricity|Storage Converter	0.382	0.289
System Cost|OPEX|Electricity|Storage Reservoir	0.382	0.289
Capacity|Electricity|Solar|PV|Open Field	0.217	0.987
System Cost|Capex|Electricity|Solar|PV|Open Field	0.204	0.986
Price|Secondary Energy|Gases|Biomass	0.189	0.000
Capacity|Electricity|Storage Converter	0.177	0.949
Investment|Energy Supply|Electricity|Solar|PV|Open Field	0.137	1.000
System Cost|OPEX|Heat|Biomass|w/ CCS	0.121	0.997
System Cost|OPEX|Electricity|Gas|w/ CCS	0.121	1.000
System Cost|Capex|Heat|Biomass|w/ CCS	0.111	0.998
System Cost|OPEX|Electricity|Storage Converter|Stationary Batteries	0.110	0.991
System Cost|Capex|Electricity|Gas|w/ CCS	0.110	0.998

NRMSE: Normalized (combined-min-max) Root Mean Square Error
Pearson: Pearson correlation coefficient
Threshold: NRMSE > 0.1
Only variables reaching the threshold are shown. Find the equivalent plot for all of them below.

Plots

Main branch	Feature branch

General

Plots comparison

Main branch	Feature branch

Files comparison

	Status	NRMSE	MAE (norm)
csvs/metrics.csv	⚠️Changed	0.000	0.08
csvs/cumulative_cost.csv	⚠️Changed	0.000	0.09
csvs/prices.csv	⚠️Changed	0.001	0.06
csvs/market_values.csv	⚠️Changed	0.005	0.05
csvs/price_statistics.csv	⚠️Changed	0.002	0.18
csvs/nodal_capacities.csv	⚠️ NaN mismatch
csvs/nodal_cfs.csv	⚠️ NaN mismatch
csvs/nodal_costs.csv	⚠️ NaN mismatch
csvs/costs.csv	✅ Almost equal	0.000	0.03
csvs/cfs.csv	✅ Almost equal	0.038	0.00
csvs/nodal_supply_energy.csv	✅ Almost equal	0.000	0.00
csvs/curtailment.csv	✅ Almost equal	0.000	0.04
csvs/supply_energy.csv	✅ Almost equal	0.000	0.03
csvs/capacities.csv	✅ Almost equal	0.000	0.00
csvs/supply.csv	✅ Almost equal	0.002	0.00
csvs/energy.csv	✅ Almost equal	0.000	0.03
csvs/weighted_prices.csv	✅ Equal

NRMSE: Normalized (combined-min-max) Root Mean Square Error
MAE (norm): Mean Absolute Error on normalized data (min-max)
Status Threshold: MAE (norm) > 0.05 and NRMSE > 0.3

Model Metrics

Benchmarks

Comparing fix-extra-funcs (2e70a4f) with main (234d70b).
Branch is 6 commits ahead and 1 commits behind.
Last updated on 2025-01-23 14:19:37 CET.

[toniseibold]

In general it works and is looking good. It is also nice to include it with the infrastructure already provided by pypsa-eur.
However, what doesn't quite sit right with me is placing the custom_extra_functionality script into the data folder. The data folder in my opinion is for universal files that are used as input only.
The custom_extra_functionality is changing the optimization problem profoundly so I would like to see it somewhere in ./scripts/...
@lindnemi / @fneum what is your opinion on this? Is that something you'd like to see and if so only for pypsa-de or even pypsa-eur?

[lkstrp]

I couldn't agree more.

We can just move the file and change the path in the default config for pypsa-de:

pypsa-de/config/config.default.yaml

Line 912 in c1e59d1

custom_extra_functionality: "../data/custom_extra_functionality.py"

In Eur it is probably meant as a "data file" for users who do not touch any code and just add custom functionality with some changes in the config. It still doesn't make sense for this to be tracked by git. But this is a pypsa-eur problem. The data folder also needs a general clean up over there.

For DE I would just move the file back to scripts and change the default config.