Skip to content

Latest commit

 

History

History
98 lines (86 loc) · 15.2 KB

README.md

File metadata and controls

98 lines (86 loc) · 15.2 KB
Raw

Overview

This utility allows to create sectoral water demand maps at the desired spatial resolution and for the desired geographical areas. The maps indicate, for each pixel, the time-varying water demand map to supply for domestic, livestock, industrial, and energetic (cooling) water consumption. The temporal discretization of the information is monthly for domestic and energy demand, yearly for industrial and livestock demand. The users can select the historical temporal span of the output datasets (start year and end year). The maps of sectoral water demand are required by the LISFLOOD OS water use module. Clearly, the scripts and the outputs of this utility can be used also for other applications, as well as for stand-alone analysis of historical water demand for anthropogenic use. Due to the large uncertainty of the global data sets, the distinction between demand and withdrawal is here omitted.

This README file provides detailed technical information about the input datasets and the usage of this utility. The methodology is explained in the manuscript: Choulga, M., Moschini, F., Mazzetti, C., Grimaldi, S., Disperati, J., Beck, H., Salamon, P., and Prudhomme, C.: Technical note: Surface fields for global environmental modelling, EGUsphere, 2023 (preprint).

The global sectoral water demand maps at 3 arcmin (or 0.05 degrees) resolution, 1979-2019, produced using the scripts of this utility can be downloaded from Joint Research Centre Data Catalogue - LISFLOOD static and parameter maps for GloFAS - European Commission (europa.eu)

Data

The utility consists of five scripts which require several following datasets and files. The full list of datasets is provided below. For each dataset, this documentation provides the link and the instructions relative to available version at the time of creation of the scripts. The correct functioning of the scripts is guaranteed for those datasets. Clearly, updated versions are regularly published: one relevant example is Global Human Settlement - Datasets - European Commission (europa.eu) (see Global Human Settlement Layer (GHSL)). Albeit the functioning of the utility is expected to be maintained, the users are invited to report problems or suggestions via GitHub issues.

  1. FAO AQUASTAT sectoral water withdrawal estimates. Select "All Countries" and the following seven fields in the variable groups "Geography and population" and "Water use": "Gross Domestic Product (GDP)", "Industry, value added to GDP", "Agricultural water withdrawal", "Industrial water withdrawal", "Municipal water withdrawal", "Total water withdrawal", and "Irrigation water withdrawal". Select All Years, then click on the "Show Data" button. On the new page, click on the "Download" button that will appear on the top right corner to download the xlsx file containing all data. Convert the Excel file to CVS and save it as aquastat_clean.csv in aquastat_folder specified in the configuration file.
  2. Global Human Settlement Layer (GHSL) POP R2019A residential population estimates for target years 1975, 1990, 2000, and 2015. You will find data as an archive here. For each year, download the 0.0025° (9 arcsec, folders ending by "_9ss", e.g. "GHS_POP_E2015_GLOBE_R2019A_4326_9ss") maps in WGS84 projection and resample them to 0.01° using gdalwarp -t_srs EPSG:4326 -tr 0.01 0.01 -r average -of GTiff GHS_POP_E<year>_GLOBE_R2019A_4326_9ss_V1_0.tif GHS_POP_E<year>_GLOBE_R2019A_4326_9ss_V1_0_reprojected.tif, where <year> is 1975, 1990, 2000, or 2015. We resample the maps using average instead of sum as the latter is broken. Put resampled files in ghsl_folder.
  3. Global Change Analysis Model (GCAM) regional water withdrawal and electricity consumption estimates. Download the Windows release package version 5.4, execute run-gcam.bat, wait for the model to finish. Execute run-model-interface.bat, click "File" > "Open" > "DB Open", select output/database_basexdb, and select all scenarios and all regions. Select water demand, select water withdrawals by sector, and click "Run query". Select energy transformation, select electricity, select elec consumption by demand sector, and click "Run query". For each tab, select all data with ctrl-a and click "Edit" > "Copy". Open a blank Google Sheets spreadsheet, press ctrl-v, manually add the headers, click "File" > "Download" > "Comma-separated values", save as water_demand.csv and elec_consumption.csv, respectively, and put both files in gcam_folder.
  4. Gridded Livestock of the World (GLW 3) species distribution dataset. Download the eight zip files (each representing a different species) and extract them to glw_folder.
  5. Huang et al. (2018) global gridded water withdrawal estimates. These estimates are not incorporated in our dataset, but are only used for the sake of comparison. Download all 7z files and extract them to huang_folder.
  6. Multi-Source Weather (MSWX) daily and monthly mean air temperature. Download using rclone as explained in the FAQ on the web page. Put the daily and monthly netCDFs in <mswx_folder>/Past/Temp/Daily and <mswx_folder>/Past/Temp/Monthly, respectively, where mswx_folder is specified in the configuration file.
  7. US Census Bureau state borders shape file. Download cb_2018_us_state_500k.zip, unzip it, open the shape file in QGIS, open the Field Calculator, enter STATEFP in "Expression", enter STATEFP_ in "Output field name", and click OK. Rasterize to 0.01° using gdal_rasterize -l cb_2018_us_state_500k -a STATEFP_ -tr 0.01 0.01 -a_nodata 0.0 -te -180.0 -90.0 180.0 90.0 -ot Float32 -of GTiff cb_2018_us_state_500k.shp cb_2018_us_state_500k_rasterized.tif. Put cb_2018_us_state_500k_rasterized.tif in us_states_folder.
  8. USGS NWIS water withdrawal estimates for 1985--present. For each state in the "Geographic Area" drop-down menu, select "State Data", "ALL Years", "State Total", and "ALL Categories" and click "Submit". Then select "Tab-separated data" and click "Submit". Do this for each state and put files in usgs_water_use_folder.
  9. USGS water withdrawal data files for 1985 and 1990 (to supplement the NWIS data). Download "Data file of state-level data" for each year (us85st.txt and us90st.txt) and put the files in usgs_water_use_folder.
  10. Vassolo and Döll (2005) industrial and thermoelectric water withdrawal maps (included in this repository with permission from Petra Döll). The industrial withdrawal shape file is rasterized using gdal_rasterize -l "industry_paper_vassolo&doell" -a MANUF_WWD -tr 0.5 0.5 -a_nodata 0.0 -te -180.0 -90.0 180.0 90.0 -ot Float32 -of GTiff "industry_paper_vassolo&doell.shp" manuf_wwd.tif. Thermoelectric shape file rasterized using same command but with WWD_PS and wwd_ps.tif. Put the files in vassolo_doll_folder.
  11. Thematic Mapping country borders shape file. Download TM_WORLD_BORDERS-0.3.zip, unzip, and rasterize to 0.01° using gdal_rasterize -l TM_WORLD_BORDERS-0.3 -a UN -tr 0.01 0.01 -a_nodata 0.0 -te -180.0 -90.0 180.0 90.0 -ot Float32 -of GTiff TM_WORLD_BORDERS-0.3.shp TM_WORLD_BORDERS_UN_rasterized.tif. Put the result in world_borders_folder.
  12. World Bank manufacturing value added and gross domestic product data. Search for "Manufacturing, value added (constant 2010 US$)" and "GDP (constant 2010 US$)", download as CSV, and put in world_bank_folder (Remove Metadata csv files, if any).

The locations of the datasets and files are specified in the configuration file (see the config.cfg example). The script also requires a template map which defines the output resolution and area. The template map should be in netCDF-4 format and contain lat and lon variables and a data variable (any name). The location of the template map is specified using templatemap_path in the configuration file. The data are produced for the period spanning year_start to year_end and saved in netCDF-4 format to output_folder (all specified in the configuration file).

Methods

The module consists of five scripts:

  1. step1_population_density.py:
    1. Resamples the native resolution of the GHSL population grids to the template map resolution.
    2. Computes population grids (in density per km2) for each year using linear interpolation and nearest-neighbor extrapolation.
  2. step2_domestic_demand.py:
    1. Produces a map of the R parameter (measures the relative withdrawal difference between the warmest and coldest months).
    2. Computes annual time series of population for each country.
    3. Loads the country-scale AQUASTAT municipal water withdrawal estimate and produces annual time series using linear interpolation and population-based forward and backward extrapolation.
    4. Loads the state-level USGS withdrawal data and computes annual values using linear interpolation and nearest-neighbour extrapolation.
    5. For each year, computes country- and state-scale per capita water demand and produces a global map.
    6. Annual per capita water demand for countries without any AQUASTAT information is estimated using nearest-neighbour interpolation.
    7. Spatially disaggregates the country- and state-scale annual per capita water demand estimates using the population estimates to obtain global annual withdrawal grids.
    8. Temporally downscales the annual withdrawal grids using monthly MSWX air temperature grids and the map of the R parameter.
  3. step3a_industrial_demand.py:
    1. Computes country-scale values of the Vassolo and Döll (2005) industrial and thermoelectric withdrawal grids.
    2. Processes the country-scale AQUASTAT industrial withdrawal data.
    3. Processes the country-scale World Bank manufacturing value added (MVA) data.
    4. Processes the GCAM regional industry and thermoelectric withdrawals and spatially downscales them to the country scale based on population.
    5. Processes the GCAM regional electricity consumptions and assigns values to each country.
    6. Computes annual country-scale manufacturing and thermoelectric withdrawal time series as follows:
      1. If AQUASTAT and GCAM data are available, temporally gap-fills the AQUASTAT industrial withdrawals using linear interpolation and MVA- or population-based extrapolation, temporally gap-fills the GCAM manufacturing and thermoelectric withdrawals using linear interpolation and nearest-neighbor extrapolation, rescales the GCAM manufacturing and thermoelectric withdrawals to match the Vassolo and Döll (2005) estimates, and temporally disaggregates the AQUASTAT industrial withdrawals into manufacturing and thermoelectric using GCAM.
      2. If only GCAM data are available, simply use the rescaled GCAM manufacturing and thermoelectric withdrawals.
      3. If neither AQUASTAT nor GCAM data are available, set the manufacturing and thermoelectric withdrawals to zero.
    7. Computes monthly heating degree days (HDD) and cooling degree days (CDD) grids using daily MSWX air temperature.
  4. step3b_industrial_demand.py:
    1. Loads the annual country-scale manufacturing and thermoelectric withdrawal time series, the USGS manufacturing and thermoelectric withdrawal estimates, and the GCAM electricity consumptions from the preceding scripts.
    2. Gap-fills the annual USGS manufacturing and thermoelectric withdrawal time series using linear interpolation and nearest-neighbor extrapolation.
    3. Spatially downscales the country- and state-scale manufacturing and thermoelectric withdrawals using population grids to obtain annual withdrawal grids.
    4. Temporally downscales the annual thermoelectric withdrawal grids based on the GCAM electricity consumptions and the HDD and CDD grids.
  5. step4_livestock_demand.py:
    1. Computes country-scale livestock withdrawals by taking the difference between the AQUASTAT agriculture and irrigation withdrawals.
    2. Processes the raw GLW grids for the different species and estimates the total livestock mass.
    3. Computes the total livestock mass for each country.
    4. Spatially downscales the GCAM regional livestock withdrawals to the country scale using the GLW-based total livestock mass estimates.
    5. Loops through countries and gap-fills the annual GCAM time series using linear interpolation and nearest-neighbor extrapolation, and rescales the time series to match the AQUASTAT estimate (if available).
    6. Gap-fills the annual USGS manufacturing and thermoelectric withdrawal time series using linear interpolation and nearest-neighbor extrapolation.
    7. For each year, spatially downscales the country- and state-scale livestock withdrawals using the GLW total livestock mass grid.

System requirements

The script can be run on a normal desktop PC, support is provided only for Linux. Physical memory requirements depend on the desired geographical extent and resolution of the maps dataset. For instance, the generation of the sectoral water demand dataset for the global domain at 3arcmin (or 0.05 degrees) resolution required 32 GB.

Instructions

Install main lisflood-utilities package in a conda environment:

conda create --name <env> python=3.7 -c conda-forge
conda activate <env>
conda install -c conda-forge pcraster eccodes gdal
pip install lisflood-utilities

Then clone the repository:

git clone --single-branch --branch master https://github.com/ec-jrc/lisflood-utilities
cd lisflood-utilities/src/lisfloodutilities/water-demand-historic

Produce a configuration file with the correct paths and folders (based on the included example).

Run the scripts as follows:

python step1_population_density.py <config file>
python step2_domestic_demand.py <config file>
...