Merge pull request #400 from nens/issue-306

Scenario archive stuff

source/c_3di_live_sessions.rst

@@ -88,37 +88,28 @@ You can:
 Storing results
 ---------------
 
-Results can be stored by clicking **User menu**, then clicking **Quit Simulation** and then **Quit, Store Results**. There are two options:
+Raw results always stored for 7 days
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-- Download results directly via the browser
-- Store them to the Lizard platform (see https://docs.lizard.net/a_lizard.html) 
+:ref:`Raw results<outputs>` will always be stored for 7 days, even if you choose the option *Quit, don't store results*. 
 
-Stored (raw) results can also be downloaded using the"3Di Models and Simulations" in the 3Di Modeller Interface, see: :ref:`mi_download_res`. Note that these raw results are only available for 7 days.
+Direct download
+^^^^^^^^^^^^^^^
 
-The options in Lizard storage are as follows:
+When you quit the session (*User menu* > *Quit Simulation*), you can download these files directly to your computer by choosing > *Quit, Store Results* > *Direct download*. You may also :ref:`download them from the 3Di Modeller Interface<mi_download_res>`.
 
-- raw data and logging
-- basic processed results
-- arrival time map
-- damage estimation (NL only)
+Store in scenario archive
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The **Basic processed results** option includes the following derivations from simulation results for Lizard users:
+.. note::
+    This option is only available if your subscription includes the Scenario Archive.
+
+Another option you have when you quit the session, is to store the results in the :ref:`scenario_archive`. When you choose this option, you also have several options for **automated post-processing**. 
 
 .. figure:: image/d3.9_store_results.png
     :alt: Storing results
 
-- Water level - temporal
-- Water depth - temporal
-- Maximum flow velocity
-- Maximum rate of rise
-- Maximum water depth
-- Flood hazard rating
-
-The **Damage estimation** option uses a module called *WaterSchadeSchatter* (currently only available in The Netherlands)
-which provides two products derived from the maximum water depth.
-
-- Damage estimation map
-- Damage estimation table
+For an explanation of the available options, see :ref:`scenario_archive`.
 
 
 .. _live_simulation_template:

source/d_scenario_archive.rst

@@ -1,35 +1,53 @@
 .. _scenario_archive:
 
 Scenario Archive
-=================
+================
 
-3Di users that have access to Lizard can view and playback stored scenarios. Depending on your contract and location you may be able to view estimates of damage caused by inundation or flooding (waterschadeschatter).
-For Lizard users, you can view your scenarios here: https://demo.lizard.net/viewer/
-If scenario's are set to public, you do not need an account to view them.
-If you also have Lizard management rights, you can manage your scenario's here: https://demo.lizard.net/management/data_management/scenarios
-For more information, please visit the Lizard documentation page: https://docs.lizard.net/c_scenarios.html 
+.. note::
+    This option is only available if your subscription includes the Scenario Archive.
 
+The Scenario Archive (Lizard) is used to store and view scenarios.
 
-After simulating with 3Di, the scenario can be processed to several rasters to visualise your results. These rasters can be viewed using the Lizard portal and in the 3Di Modeller Interface using WMS. *Note: these post-processed results are only available for Lizard subscription holders.*
+You can access the Scenario Archive here: https://demo.lizard.net/. For instructions on how to use the Scenario Archive, see the `Lizard documentation<http://docs.lizard.net>`_, for example, the sections about finding scenarios in the `Catalog<https://docs.lizard.net/e_catalog.html#scenarios>`_, and the section about the `Viewer<https://docs.lizard.net/e_viewer.html#>`_. 
 
-The following results are available: 
+Scenario's that are stored in the Scenario Archive can also be accessed in the 3Di Modeller Interface, using the `Lizard plugin<https://docs.lizard.net/d_qgisplugin.html>`
 
-**Water levels:**
-Water level relative to mean sea level (m) at a certain time step or maximum water level per calculation cell. 
+A 3Di scenario consists of one or more of the following components:
 
-**Water depth:**
-Water depth (m) relative to surface level (water level minus surface level). Surface level is defined by the DEM used by the model. 
+- :ref:`Raw results<outputs>`
+- :ref:`basic_processed_results`
+- :ref:`arrival_time`
+- :ref:`damage_estimation`
 
-**Rainfall:**
-Rain intensity per calculation cell (mm/h).
+.. _basic_processed_results:
+
+Basic processed results
+-----------------------
+
+Basic processed results are:
+
+Water level (timeseries)
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Water level (m MSL) relative to datum (usually mean sea level, MSL), for each output timestep. The water level data per computional cell is spatially interpolated to the pixel resolution. 
+
+Max water level
+^^^^^^^^^^^^^^^
+
+Maximum water level (m MSL) relative to datum (usually mean sea level, MSL). The maximum water level data per computional cell is spatially interpolated to the pixel resolution. 
+
+Water depth (timeseries)
+^^^^^^^^^^^^^^^^^^^^^^^^
+Water depth (m) relative to surface level (water level minus surface level), for each output timestep. Surface level is defined by the DEM used by the model. The water level data per computional cell is spatially interpolated to the pixel resolution. 
 
-**Maximum flow velocity:**
-Maximum flow velocity per calculation cell (m/s). Can for example be used for flood damage estimations. 
+Max water depth
+^^^^^^^^^^^^^^^
 
-**Rise velocity:** 
-Rate of rise (m/s), defined by the difference in water level per second. Can for example be used for flood damage estimations. 
+Maximum water depth (m) relative to surface level (maximum water level minus surface level). Surface level is defined by the DEM used by the model. The maximum water level data per computional cell is spatially interpolated to the pixel resolution.
+
+Flood hazard rating
+^^^^^^^^^^^^^^^^^^^
 
-**Flood hazard rating:**
 Rating used to indicate the degree of danger caused by flooding. 
 The flood hazard rating is calculated as follows: 
 
@@ -43,48 +61,36 @@ HR = d * (v + 0.5) + DF
 
 When water depth is smaller than or equal to 0.25 than DF = 0.5, else DF = 1. 
 
+Rain
+^^^^
 
+Rain intensity per calculation cell (mm/h).
 
-View stored results
----------------------
-
-To view results in Lizard, follow these steps
-
-
-#.. Go to Lizard (e.g. demo.lizard.net) and Log on.
-
-#.. Expand the layer view and scroll down to add another layer
+Rate of rise
+^^^^^^^^^^^^ 
 
-.. figure:: image/d_lizard_add_layer.png
-   :alt: Lizard add layer
+Rate of rise (m/s), defined by the difference in water level per second. Can be used for flood damage estimations, for example.
 
-#.. Search for your scenario name and select it, then go back
+Max flow velocity
+^^^^^^^^^^^^^^^^^
 
-.. figure:: image/d_lizard_add_layer2.png
-   :alt: Lizard add layer 2
+Maximum flow velocity per calculation cell (m/s). The flow velocity in the calculation cell is the resultant of the flow velocity x (``ucx``) and y (``ucy``) at the cell center. Can be used for flood damage estimations, for example. 
 
-#. Select your scenario in the layers panel. Here you can view several processed results and download raw results. Select the layer ‘waterdiepte’ (or waterdepth).
 
-.. figure:: image/d_lizard_select_layer.png
-   :alt: Lizard select layer
+.. _arrival_time:
 
-#. Then click the small vizier on the selected layer. This will zoom your view to your model extent and the correct period in time in which your scenario was stored (see time frame at the bottom of your screen).
+Arrival time
+------------
 
-#. To playback your scenario click the play button. To pause click it again. You will notice that the image will sharpen as soon as you pause playback. You may also click on a point on the map to view a graph of the water depth at that location.
+.. todo::
+    Hier een stukje voor schrijven.
+
+
 
-#. Raw results can be downloaded from the menu directly. To download a raster select the export button and select the result you would like. You must choose a spatial projection and resolution. All data within your current view is exported. When the export is ready for download you receive a message in your inbox.
+.. _damage_estimation:
 
-.. figure:: image/d_export_button.png
-   :scale: 90%
-   :alt: Lizard export button
-   :align: center
-
-   Lizard export button
-
-Stored results can be managed using the following URL: <your-organisation>.lizard.net/management/scenarios
-
-Damage Estimation 
----------------------
+Damage estimation (NL only)
+---------------------------
 
 Depending on your location Lizard provides estimates of damage caused by inundation or flooding. To use the damage estimation your study or model area must be within the Netherlands. 
 
@@ -101,45 +107,3 @@ The estimated damages are available on a 0.5 m x 0.5 m resolution. Direct, indir
 
 Further documentation (only in Dutch) can be downloaded from :download:`here <pdf/nabewerking-3di-resultaten-in-lizard.pdf>`. The used damage table are available in :download:`Excel <other/3di-v2.2018-05-15.xlsx>` and :download:`CFG <other/3di-v2.2018-05-15.cfg>` (for use on `waterschadeschatter.nl <https://www.waterschadeschatter.nl>`_. The damage estimation in Lizard was developed together with Hoogheemraadschap Hollands Noorderkwartier.
 
-
-
-
-
-Load rasters in 3Di Modeller Interface using WMS
--------------------------------------------------
-To view post-processed results from your 3Di scenario in the 3Di Modeller Interface follow the following steps: 
-
-| 1. Find the scenario UUID in the scenario management screen of your Lizard portal. Go to ``{yourportal}.lizard.net``, click on **Management > Data > 3Di Scenarios** and search for your scenario. After opening, you can copy the UUID from the URL. 
-
-| 2. Compose WMS url. Fill out the name of the Lizard portal you are using and the UUID of your scenario in the following URL: 
-| ``https://{yourportal}.lizard.net/wms/scenario_{UUID of scenario}/?request=getcapabilities``
-
-| For example: 
-| https://demo.lizard.net/wms/scenario_c30ef7f2-c871-4d70-a087-8f078f9ebafd/?request=GetCapabilities
-
-.. TODO: dit moet helemaal anders, kan eigenlijk allemaal weg. In lizard docs pull request: https://docs.lizard.net/e_lizardwms.html#di-scenarios. aanpassen. uitleggen scenario>dan kun je het gewoon kopieeren. en dan hier naar lizard docs verwijzen. 
-
-| 3. In the 3Di Modeller Interface connect to the Lizard WMS server using the Data Source Manager. 
-| a) Choose WMS/WMTS as data source.
-| b) Create a new connection.
-| c) Give your scenario a name and copy the URL composed in the previous step. 
-| d) Under *Authentication* choose *Basic*.
-| e) You need to use a personal API key. If you do not have one yet, you can create one in the Lizard 3Di Management. Go to yourportal.lizard.net, go to **Management > Personal API keys > +New Item.** Use *__key__* as username and the personal API key you created as password. See the `Lizard documentation <https://docs.lizard.net/d_apitechnical.html#apiauthenticationanchor>`_ for more information. 
-| f) Click *OK* to save the connection. 
-
-.. figure:: image/d_wms_connection.png
-    :alt: Create WMS connection in QGIS
-
-1. When the connection is created, several layers appear (expand the *Title*-section to view full names of the layers). The layers can be added to the project by selecting them and clicking *Add*. 
-
-.. figure:: image/d_wms_layers_3di.png
-    :alt: 3Di WMS layers
-
-| 5. The water depth, water level and rain rasters can also be viewed as timeseries.
-| a) A temporal raster is indicated by a small clock icon in the layer panel.
-| b) Activate the *Temporal Controller* by clicking the clock sign on the toolbar.
-| c) Turn on *Fixed range temporal navigation* or *Animated temporal navigation*.
-| d) Choose for which time step of your simulation you want to see the water level or depth. 
-
-.. figure:: image/d_wms_temporal_controller_rasters.png
-    :alt: Temporal Controller WMS layers

source/h_gridadmin.rst

@@ -0,0 +1,6 @@
+.. _gridadmin:
+
+Computational grid file
+-----------------------
+
+The :ref:`computational_grid` is stored in a **gridadmin.h5** file. The contents of this file can be viewed in the 3Di Modeller Interface. If you are using Python, use :ref:`threedigrid` to work with this file.

source/h_outputs.rst

@@ -3,10 +3,19 @@
 Outputs
 =======
 
+The outputs that are produced directly by 3Di are called *Raw results*. Which of these files are created, depends on the settings used. E.g., if the simulation does not include structure control, no structure_control_actions_3di.nc file will be created.
+
 .. toctree::
    :maxdepth: 1
 
+   h_gridadmin
    h_results
-   h_water_quality_results
    h_aggregate_results
-   h_logging
+   h_structure_control_actions
+   h_water_quality_results
+   h_logging
+
+.. note::
+
+	The computional grid file (gridadmin.h5) is created from the schematisation by 3Di and is input for the 3Di computional core. As such it is not an output or result of the simulation, but is needed to open the other files.
+

source/h_results.rst

@@ -1,7 +1,7 @@
 .. _3dinetcdf:
 
 Results 3Di
-=================
+===========
 
 The results of a simulation are written to a `NetCDF <https://en.wikipedia.org/wiki/NetCDF>`_ file called `results_3di.nc`, which follows the `CF Conventions <http://cfconventions.org/>`_ . The CF convention stipulates that the 2D and 1D mesh data are stored in separate parts of the file. The results_3di.nc file contains snapshots of values of all relevant flow variables (1D and 2D). The output timestep, i.e. the interval at which these snapshot values are written to the NetCDF file, is set at the start of the simulation.
 

source/h_structure_control_actions.rst

@@ -0,0 +1,6 @@
+.. structure_control_actions:
+
+Structure control actions file
+------------------------------
+
+If there are any structure control actions during the simulation, these are logged to the structure_control_actions_3di.nc file. Use :ref:`threedigrid` to work with this file.

source/i_downloading_results.rst

@@ -3,7 +3,18 @@
 Downloading results
 ====================
 
-Simulation results can be downloaded using the :ref:`models_simulation_panel`. To download results:
+There are two options for downloading simulation results:
+
+- :ref:`dl_via_models_simulations`, within 7 days
+
+- :ref:`dl_via_lizard_plugin`, if your subscription includes the Scenario Archive
+
+.. _dl_via_models_simulations:
+
+Download via 3Di Models and Simulation panel
+--------------------------------------------
+
+Simulation results can be downloaded using the :ref:`models_simulation_panel`, within 7 days after the simulation has finished. To download results:
 
 #) Activate the *3Di Models and Simulation panel* by clicking the pictogram (|modelsSimulations|) in the toolbar.
 #) Click the *Results* icon (|download_results|).
@@ -14,9 +25,22 @@ The results will be downloaded to your local 3Di working directory, so that you
 .. note::
     The raw results are only available for 7 days after a simulation has finished.
 
-
 .. |modelsSimulations| image:: /image/pictogram_modelsandsimulations.png
     :scale: 90%
 
 .. |download_results| image:: /image/pictogram_download_results.png
-	:scale: 60%
+	:scale: 60%
+
+Download via the Lizard plugin
+------------------------------
+
+If you have stored simulation results in the Scenario Archive, you can download them using the `Lizard plugin<https://docs.lizard.net/d_qgisplugin.html>`_.
+
+Instructions for storing simulation results in the Scenario Archive can be found here:
+
+- In the 3Di Modeller Interface, see :ref:`this section<simulate_api_qgis_post_processing>`
+- In 3Di Live, see :ref:`this section<store_results>`
+- If you have already run a simulation, find it on :ref:`threedi_management` and post-process the results. Note that this last option is only available within 7 days after you have run the simulation.
+
+.. note::
+    This option is only available if your subscription includes the Scenario Archive.

source/i_mi_analysing_results.rst

@@ -14,6 +14,7 @@ These tools relate the time series of the flow variables (water levels, volumes,
    :caption: Tools for analysing results in the 3Di Modeller Interface
    :name: toctree_analysing_results
 
+   Load results as WMS layers <i_wms_layers>
    Download results <i_downloading_results>
    Load and visualise results <i_loading_and_visualising_results>
    Water depth and water level rasters <i_water_depth_level_rasters>

source/i_wms_layers.rst

@@ -0,0 +1,8 @@
+.. i_wms_layers:
+
+Load results as WMS layers
+==========================
+
+You can add the post-processed results from the :ref:`scenario_archive` to your project as WMS layers, such as maximum water depth maps. This has several advantages: you do not need to download anything, and you can easily view the water depth map for any time step, or see how the water depth map changes over time. 
+
+This is easy to do with the Lizard plugin. The documentation for this tool is `part of the Lizard documentation<https://docs.lizard.net/d_qgisplugin.html>`_

source/j_threedigrid.rst

@@ -1,4 +1,6 @@
-3Di Results API
-================
+.. _threedigrid:
 
-The 3Di Results API is a Python interface for reading and analyzing 3Di results. The Python package is called "threedigrid" and has its own documentation, which can be found `here <https://threedigrid.readthedocs.io/en/latest/>`_.
+3Di Results API (threedigrid)
+=============================
+
+The Python package "threedigrid" is a Python interface for reading and analyzing 3Di results. It has its own documentation, which can be found `here <https://threedigrid.readthedocs.io/en/latest/>`_.