Error with CCL constraint after 0.4.0 release.

[RogerKwak]

Checklist

• I am using the current main branch or the latest release. Please indicate.
• I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.
• CCL Error

Describe the Bug

It seems there is an issue with the CCL functionality after updating to PyPSA-Earth 0.4.0.

Neither the code nor the configuration has changed from version 0.3.0, but the problem arose after updating to version 0.4.0, suggesting there might be an issue with the CCL constraint in the new version.

The data/agg_p_nom_minmax.csv file is definitely in the path, and I have also added the following code in config.yaml:

constraints:
CCL: true

By the way, the normal network like elec_s_36_ec_lcopt_Co2L-3H.nc works fine.

Error Message

If applicable, paste any terminal output to help illustrating your problem.
In some cases it may also be useful to share your list of installed packages: conda list.

(pypsa-earth) C:\Users\User\pypsa-earth>snakemake --core all results/networks/elec_s_36_ec_lcopt_Co2L-CCL-3H.nc
Set parameter Username
Academic license - for non-commercial use only - expires 2025-05-30
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pulp\apis\gurobi_api.py:170: UserWarning: GUROBI error: Unknown parameter '_test'.
  warnings.warn("GUROBI error: {}.".format(e))
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\networkclustering.py:16: UserWarning: The namespace `pypsa.networkclustering` is deprecated and will be removed in PyPSA v0.24. Please use `pypsa.clustering.spatial instead`.
  warnings.warn(
Demand data folder: data\ssp2-2.6\2036\era5_2018, load path is ['data\\ssp2-2.6\\2036\\era5_2018\\Asia.nc'].
Expected files: Asia.nc
Building DAG of jobs...
Provided cores: 32
Rules claiming more threads will be scaled down.
Job stats:
job              count
-------------  -------
solve_network        1
total                1

Select jobs to execute...

[Tue Aug  6 20:40:01 2024]
rule solve_network:
    input: networks/elec_s_36_ec_lcopt_Co2L-CCL-3H.nc
    output: results/networks/elec_s_36_ec_lcopt_Co2L-CCL-3H.nc
    log: logs\solve_network\elec_s_36_ec_lcopt_Co2L-CCL-3H_solver.log, logs/solve_network/elec_s_36_ec_lcopt_Co2L-CCL-3H_python.log
    jobid: 0
    benchmark: benchmarks/solve_network/elec_s_36_ec_lcopt_Co2L-CCL-3H
    reason: Missing output files: results/networks/elec_s_36_ec_lcopt_Co2L-CCL-3H.nc
    wildcards: simpl=, clusters=36, ll=copt, opts=Co2L-CCL-3H
    threads: 20
    resources: tmpdir=C:\Users\User\AppData\Local\Temp, mem=17020

Changing to shadow directory: C:\Users\User\pypsa-earth\.snakemake\shadow\tmpugaeql3x
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\networkclustering.py:16: UserWarning: The namespace `pypsa.networkclustering` is deprecated and will be removed in PyPSA v0.24. Please use `pypsa.clustering.spatial instead`.
  warnings.warn(
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\components.py:318: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '[]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  attrs.loc[bool_b, "default"] = attrs.loc[bool_b].isin({True, "True"})
INFO:pypsa.io:Imported network elec_s_36_ec_lcopt_Co2L-CCL-3H.nc has buses, carriers, generators, global_constraints, lines, links, loads, storage_units, stores
INFO:pypsa.linopf:Prepare linear problem
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\linopf.py:530: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  pd.concat([bus_injection(*arg) for arg in args], axis=1)
C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\linopf.py:538: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  (-get_as_dense(n, "Load", "p_set", sns) * n.loads.sign)
Need to specify the path to a .csv file containing aggregate capacity limits per country in config['electricity']['agg_p_nom_limit'].
Traceback (most recent call last):
  File "C:\Users\User\pypsa-earth\.snakemake\shadow\tmpugaeql3x\.snakemake\scripts\tmpz5xw3ydp.solve_network.py", line 159, in add_CCL_constraints
    agg_p_nom_minmax = pd.read_csv(agg_p_nom_limits, index_col=list(range(2)))
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\parsers\readers.py", line 1026, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\parsers\readers.py", line 620, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\parsers\readers.py", line 1620, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\parsers\readers.py", line 1880, in _make_engine
    self.handles = get_handle(
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\common.py", line 873, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: 'data/agg_p_nom_minmax.csv'
ERROR:__main__:Need to specify the path to a .csv file containing aggregate capacity limits per country in config['electricity']['agg_p_nom_limit'].
Traceback (most recent call last):
  File "C:\Users\User\pypsa-earth\.snakemake\shadow\tmpugaeql3x\.snakemake\scripts\tmpz5xw3ydp.solve_network.py", line 159, in add_CCL_constraints
    agg_p_nom_minmax = pd.read_csv(agg_p_nom_limits, index_col=list(range(2)))
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\parsers\readers.py", line 1026, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\parsers\readers.py", line 620, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\parsers\readers.py", line 1620, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\parsers\readers.py", line 1880, in _make_engine
    self.handles = get_handle(
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pandas\io\common.py", line 873, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: 'data/agg_p_nom_minmax.csv'
Adding per carrier generation capacity constraints for individual countries
INFO:__main__:Adding per carrier generation capacity constraints for individual countries
ERROR:_helpers:An error happened in module 'C:\\Users\\User\\pypsa-earth\\scripts\\solve_network.py', function 'add_CCL_constraints': local variable 'agg_p_nom_minmax' referenced before assignment
Traceback (most recent call last):
  File "C:\Users\User\pypsa-earth\.snakemake\shadow\tmpugaeql3x\.snakemake\scripts\tmpz5xw3ydp.solve_network.py", line 574, in <module>
    n = solve_network(
  File "C:\Users\User\pypsa-earth\.snakemake\shadow\tmpugaeql3x\.snakemake\scripts\tmpz5xw3ydp.solve_network.py", line 526, in solve_network
    network_lopf(
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\linopf.py", line 1558, in network_lopf
    fdp, problem_fn = prepare_lopf(
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\site-packages\pypsa\linopf.py", line 1217, in prepare_lopf
    extra_functionality(n, snapshots)
  File "C:\Users\User\pypsa-earth\.snakemake\shadow\tmpugaeql3x\.snakemake\scripts\tmpz5xw3ydp.solve_network.py", line 499, in extra_functionality
    add_CCL_constraints(n, config)
  File "C:\Users\User\pypsa-earth\.snakemake\shadow\tmpugaeql3x\.snakemake\scripts\tmpz5xw3ydp.solve_network.py", line 184, in add_CCL_constraints
    minimum = agg_p_nom_minmax["min"].dropna()
UnboundLocalError: local variable 'agg_p_nom_minmax' referenced before assignment
[Tue Aug  6 20:40:16 2024]
Error in rule solve_network:
    jobid: 0
    input: networks/elec_s_36_ec_lcopt_Co2L-CCL-3H.nc
    output: results/networks/elec_s_36_ec_lcopt_Co2L-CCL-3H.nc
    log: logs\solve_network\elec_s_36_ec_lcopt_Co2L-CCL-3H_solver.log, logs/solve_network/elec_s_36_ec_lcopt_Co2L-CCL-3H_python.log (check log file(s) for error details)

RuleException:
CalledProcessError in file C:\Users\User\pypsa-earth\Snakefile, line 841:
Command 'C:/Users/User/anaconda3/envs/pypsa-earth/python.exe "C:\Users\User\pypsa-earth\.snakemake\shadow\tmpugaeql3x\.snakemake\scripts\tmpz5xw3ydp.solve_network.py"' returned non-zero exit status 1.
  File "C:\Users\User\pypsa-earth\Snakefile", line 841, in __rule_solve_network
  File "C:\Users\User\anaconda3\envs\pypsa-earth\lib\concurrent\futures\thread.py", line 58, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake\log\2024-08-06T203956.969944.snakemake.log