Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update sprinkling class msw model #1232

Draft
wants to merge 69 commits into
base: master
Choose a base branch
from

Conversation

HendrikKok
Copy link
Contributor

Fixes #

Description

Checklist

  • Links to correct issue
  • Update changelog, if changes affect users
  • PR title starts with Issue #nr, e.g. Issue #737
  • Unit tests were added
  • If feature added: Added/extended example

JoerivanEngelen and others added 30 commits May 1, 2024 09:51
Fixes #964 

# Description
Implements from_imod5 classmethod to StructuredDiscretization
classmethod.

NOTE: I had to modify the default regridding method of "idomain" as the
"mode" method appears to have a bug with perfectly aligned structured
grids.
This bug requires further investigation. Will post an issue if found.

# Checklist
<!---
Before requesting review, please go through this checklist:
-->

- [x] Links to correct issue
- [ ] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #965 

# Description
Adds a function to import an npf package from an imod5 project file.
Adds tests.

# Checklist


- [x] Links to correct issue
- [ ] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Fixes #961

# Description
implements importing from a imod5 project file for storage and initial
conditions.

# Checklist
- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Fixes #969

# Description
implements importing recharge packages from a project file. The recharge
can be imported using a planar grid,
in which case it assigns the recharge to the uppermost active cell for
each column.
The recharge can also be imported from a non-planar (so fully 3d) grid
in which case it is just regridded to the target grid.

# Checklist
- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
# Conflicts:
#	imod/mf6/dis.py
#	imod/mf6/package.py
#	imod/tests/test_typing/test_typing_grid.py
#	imod/typing/grid.py
Fixes #967 

# Description
implements importing drainage packages from imod5 data. Adds tests. 
Methods for allocation and conductivity assignment can be specified per
package, but it requires knowledge on what the packages are called in
the imod5 project file (like "drn-1" and "drn-2"). If not specified,
defaults are in place.

# Checklist

- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Fixes #1041

# Description
Adds a function that imports a simulation from an imod5 project file. It
creates a simulation, containing 1 GroundwaterFlowModel, containing the
packages for which we support importing currently. This includes the
mandatory flow packages like NPF and STO, and some boundary conditions
(like DRN and RCH) but not yet CHD or WEL.

# Checklist
- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Fixes #1053 

# Description
On importing wells, the filter top and bottom are now parsed and stored
in the project file data structure

# Checklist

- [X] Links to correct issue
- [X] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1051

# Description
Now validates that well top > well bottom. 

# Checklist

- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #968 

# Description
Imports riverpackages from IMOD5. The infiltration factor (which does
not exist in MF6) is accounted for by creating a drainage package.

# Checklist
<!---
Before requesting review, please go through this checklist:
-->

- [x] Links to correct issue
- [ ] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Fixes #1052 

# Description
when importing wells from imod5: If not all wells could be assigned to
the modflow6 model (this can be due to inactive cells,
or due to constraints of minimum thickness/permeability) then a log
message will be generated to document in the log
file that not all wells were successfully imported. 

# Checklist
- [X] Links to correct issue
- [X] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Fixes #502 

# Description
Converts hfb's from imod5 to mf6, but only in the following cases:

- the hfb has an assgined layer that is not zero
- the hfb is mappable to the LayeredHorizontalFlowBarrierResistance
class.

It creates a single mf6 hfb package containing all the hfb's that are
present in the model domain area.
The hfb package is added to the simulation. 

# Checklist

- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Follow-up on: #1070

# Description
For some reason, I erronously merged #1070. This implements my
suggestion to concatenate pandas dataframes.

# Checklist
<!---
Before requesting review, please go through this checklist:
-->

- [x] Links to correct issue
- [ ] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [ ] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: luitjan <[email protected]>
# Conflicts:
#	docs/api/changelog.rst
#	imod/mf6/dis.py
#	imod/mf6/drn.py
#	imod/mf6/ic.py
#	imod/mf6/npf.py
#	imod/mf6/oc.py
#	imod/mf6/package.py
#	imod/mf6/rch.py
#	imod/mf6/riv.py
#	imod/mf6/sto.py
#	pixi.lock
Fixes #1083 

# Description
When importing wells, we now make sure that when the ipf file defines 2
or more wells with the same x, y , filter_top, filter_bottom and id,
then the ID of the second well is appended with "_1", and so on for
successive findings of a well with the same characteristics (_2, _3
etc).

# Checklist

- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1064 

# Description
Imports wells as grid-agnostic well packages, based on filter top and
filter bottom, and x/y location as stated in the imod5_dataset object.

# Checklist
- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Fixes #966

# Description
Imports chd packages present in imod5_data. These packages are imported
one by one. If none are present, then a chd package is created at all
locations where ibound == -1. In that case the starting head of the
simulation is assigned as the chd value for each cell with ibound == -1.

# Checklist
- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #598

# Description
There was some confusion (also by me) about the use of the
``LayeredHorizontalFlowBarrier`` classes, so @HendrikKok explained me
they were intended for single layers. I therefore changed the following
to clarify:

- Rename to ``SingleLayerHorizontalFlowBarrier`` classes
- Throw ``ValidationError`` if multiple layers are added to dataset
- Added missing test for SingleLayerHorizontalFlowBarrier class

This means multiple SingleLayerHorizontalFlowBarrier need to be added to
GroundwaterFlowModel object, whereas MODFLOW 6 only accepts one single
HFB. So these have to be merged together. I think the best opportunity
for this is to merge the low level ``Mf6HorizontalFlowBarrier`` objects.

# Checklist
- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
JoerivanEngelen and others added 24 commits July 29, 2024 18:15
Fixes #1122, should be merged after
#1121 (so duplication in
changeset)

# Description
Adds ``from_imod5_data`` method to ``LayeredWell`` object.

# Checklist
- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [x] **If feature added**: Added/extended example
Forgot to push fixes for review comments, this impelements this
Fixes #1120

# Description
this PR contains fixes made to be able to import a version of the LHM
that is steady state.
**Probably not all these changes should be merged. The review should
reveal what to keep and what not.**

Changes:
-sto package can be absent in the imod5 project file. If that is the
case, now a default sto package is made which imposes steady state.
-rch package can now be absent in the project file. Then no recharge is
present in the simulation.
- allocation option and distribution option for river imports were hard
coded to different defaults . The old defaults gave an error, which
could be a new issue in itself.
-also for GHB a new default was set, because the old one gave an error.
This could be a new issue in itself.


# Checklist


- [X] Links to correct issue
- [ ] Update changelog, if changes affect users
- [X] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [X] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Joeri van Engelen <[email protected]>
Fixes #1139 

# Description
Loading project files turned out to be a time-sink for project file
imports- especially when loading prj-file data with transient arrays
such as chd-levels per timestep.
These arrays were loaded in a function called
"apply_factor_and_addition" which accounted for the fact that the user
can specify a factor to multiply each array and a constant to add to it.
These 2 (factor and addition) can be different for every timestep.
Now they are applied at an earlier stage, when loading the ipf data. 

No new tests were added, because the tests in test_import_prj.py already
test the application of factors and additions. (this PR is for improving
performance and not changing behavior)

# Checklist
<!---
Before requesting review, please go through this checklist:
-->

- [x] Links to correct issue
- [ ] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [ ] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1116

# Description
- ``open_projectfile_data`` now groups wells by ipf name
- ``GridAgnosticWell.from_imod5_data`` now validates associated IPFs
whether they meet our requirements
- ``GridAgnosticWell.from_imod5_data`` deactivates unassociated IPFs if
not present in certain timestep anymore
- Remove imod.formats.prj.convert_to_disv
- Adds lots of test cases to test the well import from projectfiles

# Checklist
- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1132 

# Description
This PR adds the following:

- Increase performance for writing the HFB files, by using new xugrid
create_snap_to_grid_dataframe.
- Fixes error when multiple barriers in a single dataset would be
snapped to same edge. These lines are now aggregated.
- Add some basic infrastructure for user acceptance test for writing
LHM, the path to the LHM projectfile should be set in a .env as LHM_PRJ.
This can later be extended to TeamCity.

This reduces the time spent writing the HFB packages from 34 minutes to
12.5 minutes on my local machine. I've profiled the causes of the
remaining bottlenecks to writing. I'll post a follow-up issue for that.

# Checklist

- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1158 

# Description
Fix remaining HFB bottlenecks. This reduces writing the HFB package from
the LHM from 12.5 minutes to 2 minutes.

- Cache results ``xu.Ugrid2d.from_structured``, as this is a costly
operation
- Use ``and`` instead of ``&`` operator in the ``scalar_None`` function,
to enable shortcutting.
- Remove ``mask_all_packages`` function in ``from_imod5_data``, call in
tests.
- Create separate pixi task for slow unittests, where just in time
compilation is enabled. ``pixi run unittests`` still runs all unittests,
by starting two pixi tasks ``unittests_njit`` & ``unittests_jit``.

# Checklist
- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1166 and somewhat fixes #1163

# Description
- Moves ``pd.to_datetime`` logic to separate place, use
``errors="coerce"`` setting, which turns datetimes beyond the year 2261
to ``pd.NaT``. These seem to be consequently ignored in the
``resample_timeseries``.
- Truncate well names to max 16 characters
- Support wells defined in "steady-state" entry in the projectfile.
- Add test cases for this.

# Checklist
- [x] Links to correct issue
- [ ] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1164


# Description
This implements the following cleanup utilities for the robin boundary
conditions, namely the DRN, GHB, and RIV.

- Cleanup helper functions, which are part of the public API (so that
people could also use them outside MODFLOW 6)
- Cleanup methods to the River, GeneralHeadBoundary, Drain package for
convenience. These call the corresponding helper function. Save the user
the hassle of manually finding the right arguments to clean up grids.
- Add a utility method: Call function on grids, which separates grid
variables from settings, calls a function on them, and then merges these
dicts again.
- Converted the fixtures in ``test_mf6_riv`` to regular functions, as
they were used as such anyway. Without this, RivDisCases could not be
used outside the ``test_mf6_riv`` module, now it can be imported.

# Checklist
<!---
Before requesting review, please go through this checklist:
-->

- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1176 

# Description

- Assign wells with point filters (filters with a zero length) to
layers. Before, these would be removed in the compute_overlap method.
- Add test case for wells with a point filter
- Set validation so that screen_top can equal screen_bottom
- Rename vague variable name "df" to "df_factor"

# Checklist
<!---
Before requesting review, please go through this checklist:
-->

- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example

---------

Co-authored-by: Sunny Titus <[email protected]>
Fixes #1187

# Description
Adds cleanup utility for wells, as well as a method on the ``Well``
object. ``LayeredWell`` doesn't have ``cleanup`` method yet, as cleaning
up almost purely focuses on adjusting filter screen levels for now. The
function as well as the method are a bit more involved than the other
cleanup utilities, because we are dealing here with a comparison of
point data and grids.

# Checklist
- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1216 

# Description
iMOD5 interprets recharge rate in mm/d, whereas MODFLOW6 is unit
consistent, and thus uses m/d.
This PR adds a conversion function to do that. I updated the tests to
also test if this conversion is done.

# Checklist
- [x] Links to correct issue
- [ ] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1210 

# Description
This PR modifies the allocation and conductance distribution methods somewhat, to allow boundary
conditions to be allocated to the first model layer when located above
model top. This is similar to how iMOD5 does these things.

# Checklist
- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1209

# Description
- Add setting to ``to_mf6_pkg``: ``error_on_well_removal``, which throws
an error if wells are removed entirely during well assignment.
- Separated code to collect filtered well ids in separate method. This
method is now also called to collect ids of errors being removed in
error message.
- Added attribute ``_is_from_imod5`` to Modflow6Simulation to keep track
if the simulation is loaded from iMOD5 data or not. Based on this, some
validation settings might be set to be a bit more permissive. (Or call
some extra cleanup?)
- ``is_from_imod5`` property to WriteContext to keep track of the same
thing within the writing context.

# Checklist
- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [x] Unit tests were added
- [ ] **If feature added**: Added/extended example
Fixes #1222

# Description
- Add a ``ValidationContext`` dataclass
- Add a ``_validation_context`` attribute to ``Modflow6Simulation``;
this replaces the ``_is_from_imod5`` attribute.
- The ``ValidationContext`` contains an attribute for strict well
checks, turned on by default. This is set to False when calling
``from_imod5`` or for split simulations.
- Adds a ``_to_mf6_pkg`` method in a similar design as proposed in
#1223, this to preserve public API.
- Refactor ``WriteContext``, to make it a dataclass again. I had to
ignore type annotation for ``write_directory``, otherwise MyPy would
throw errors. The whole property shebang presumably started with MyPy
throwing errors. Reverting it back to a dataclass reduces the lines of
code considerably, which makes it more maintainable.
- Use jit for examples run, this speeds them up considerably. The
examples ran into a TimeOut on TeamCity, and this reduces the change of
that happening again.

# Checklist
<!---
Before requesting review, please go through this checklist:
-->

- [x] Links to correct issue
- [x] Update changelog, if changes affect users
- [x] PR title starts with ``Issue #nr``, e.g. ``Issue #737``
- [ ] Unit tests were added
- [ ] **If feature added**: Added/extended example
Copy link

sonarcloud bot commented Oct 2, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

@HendrikKok HendrikKok changed the base branch from master to imod5_converter_feature_branch October 17, 2024 18:06
Base automatically changed from imod5_converter_feature_branch to master October 23, 2024 10:00
Copy link

sonarcloud bot commented Nov 14, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants