Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/vdyp 400 grow qmd+lh #60

Merged
merged 81 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
bcd320a
Added formalization of NDEBUG (Debug Settings)
mjunkin Jul 2, 2024
9ddd683
Added formalization of NDEBUG (Debug Settings)
mjunkin Jul 2, 2024
08b8e8c
Control Variable parser tests
mjunkin Jul 3, 2024
23bec2e
GROW up to and including HDGROW
mjunkin Jul 4, 2024
2072910
Housekeeping
mjunkin Jul 8, 2024
92eca3f
Unit test extensions and improvements for Forward
mjunkin Jul 8, 2024
f7900cb
Changes needed for BA growth calculation
mjunkin Jul 8, 2024
01eba20
Merge branch 'feature/VDYP-305' into feature/VDYP-363-grow
mjunkin Jul 8, 2024
a106b4d
Merge branch 'feature/VDYP-305' into feature/VDYP-363-grow
mjunkin Jul 8, 2024
d0eb540
Implementation of Caching Control Map wrapper
mjunkin Jul 9, 2024
b9a9326
Pushed this.compVarAdjustments down to ForwardResolvedControlMapImpl,…
mjunkin Jul 9, 2024
6bbbc6f
Refer now to common EMPIRICAL_OCCUPANCY
mjunkin Jul 9, 2024
ca24794
Removing python tools from this branch - now on feature/python-tools
mjunkin Jul 9, 2024
03fc6dc
growBasalArea - next steps
mjunkin Jul 10, 2024
50c1db2
Merge branch 'feature/fortran-analysis-tools' into feature/VDYP-363-grow
mjunkin Jul 10, 2024
0bd9561
Merge branch 'main' into feature/VDYP-363-grow
mjunkin Jul 10, 2024
053c4d0
BA Growth - code complete; common path testing complete.
mjunkin Jul 11, 2024
53d8f25
Grow basal area (of primary layer)
mjunkin Jul 12, 2024
b008832
Grow: quad-mean-diameter implementation and testing
mjunkin Jul 15, 2024
c145320
Component size parsing and representation changes
mjunkin Jul 23, 2024
672f0a3
Reimplementation of CompVarAdjustments for usability
mjunkin Jul 23, 2024
dcd7725
UtilizationClass changes for usability
mjunkin Jul 23, 2024
43ca25c
Next grow steps
mjunkin Jul 23, 2024
94ed1a8
Housekeeping
mjunkin Jul 23, 2024
53d14d8
Merge branch 'main' into feature/VDYP-400-grow-qmd+lh
mjunkin Jul 23, 2024
55e505a
Code changes resulting from merge out from main after VDYP-305 merged…
mjunkin Jul 25, 2024
374732e
Begin refactoring to common data structures
mjunkin Jul 29, 2024
b2890ea
Switch to PolygonIdentifier from VdypPolygonDescription
mjunkin Jul 29, 2024
23f7a60
Initial changes in switch to using common VdypPolygon and children
mjunkin Jul 29, 2024
986df5d
Merge branch 'main' into feature/VDYP-400-grow-qmd+lh
mjunkin Jul 30, 2024
b93bfaa
Merging UtilizationClassVector changes
mjunkin Jul 30, 2024
8478393
adapt to monorepo before merge attempt
mjunkin Jul 30, 2024
aa1724e
Merge branch 'main' into feature/VDYP-400-grow-qmd+lh
mjunkin Jul 30, 2024
3775ae5
Update model classes in Common to have Site be a child of Species
smithkm Jul 31, 2024
9756fed
Next step in refactoring of Forward to use common Polygon data struct…
mjunkin Jul 31, 2024
bb8875c
Merge branch 'feature/VDYP-433' into feature/VDYP-400-grow-qmd+lh
mjunkin Jul 31, 2024
0169f9a
Continue integration to common Polygon classes.
mjunkin Aug 1, 2024
a04109c
Added estimateBaseAreaYield and estimateQuadMeanDiameterYield back in…
mjunkin Aug 1, 2024
05da764
Merge branch 'feature/VDYP-433' into feature/VDYP-400-grow-qmd+lh
mjunkin Aug 1, 2024
27dc829
Next steps in changing underlying model
mjunkin Aug 2, 2024
e1fe056
Merge branch 'feature/VDYP-433' into feature/VDYP-400-grow-qmd+lh
mjunkin Aug 2, 2024
5c218b7
Next swath of changes for "grow", including promoting a calculation m…
mjunkin Aug 6, 2024
1c3e3b9
Merge branch 'feature/VDYP-433' into feature/VDYP-400-grow-qmd+lh
mjunkin Aug 6, 2024
fc8e181
Merge branch 'main' into feature/VDYP-400-grow-qmd+lh
mjunkin Aug 13, 2024
a98bc28
Merge branch 'feature/VdypPolygon-class-set-changes' into feature/VDY…
mjunkin Aug 14, 2024
d27762c
Converting Forward to use the new polygon model
mjunkin Aug 15, 2024
780425b
Merge branch 'main' into feature/VDYP-400-grow-qmd+lh
mjunkin Aug 16, 2024
9c001b2
Merge out from main; complete refactoring out the now unneeded array …
mjunkin Aug 16, 2024
1141371
Get non-Forward unit tests running again
mjunkin Aug 20, 2024
97b99e6
Current forward unit tests are all running correctly.
mjunkin Aug 23, 2024
99f65d6
Begin adaptation of VDYP output code to Forward
mjunkin Aug 27, 2024
394a2f0
Implement output
mjunkin Aug 28, 2024
cf529a5
First end-to-end run with output
mjunkin Aug 29, 2024
dc03d5c
Changes to support one Site per Layer (ignoring Site information from…
mjunkin Aug 29, 2024
2dd242f
Merge branch 'main' into feature/VDYP-400-grow-qmd+lh
mjunkin Aug 29, 2024
2e8f9f2
Added GrowLoreyHeightsTest
mjunkin Aug 31, 2024
ba7c23f
Added ZipFileResolver
mjunkin Aug 31, 2024
00024bf
Forward unit tests up to GROW_6 + ... formatting!
mjunkin Sep 4, 2024
788fd7a
FipStartTest & ZipOutputFileResolverTest corrections
mjunkin Sep 4, 2024
f3ddda1
Add test for Grow, step 6
mjunkin Sep 4, 2024
1085140
Sonarcube quality checks
mjunkin Sep 4, 2024
562afd7
Added tests of UtilizationOperations and ForwardControlMapTest class
mjunkin Sep 5, 2024
053589a
Two test corrections, reformatting.
mjunkin Sep 5, 2024
be301f9
And one more test...
mjunkin Sep 5, 2024
7450b4a
Explicitly test CachingResolvedControlMap
mjunkin Sep 5, 2024
3308865
Added logging to "uncovered" code to disprove this...
mjunkin Sep 5, 2024
80da1ba
Run formatter (again)
mjunkin Sep 5, 2024
47ad8d5
Improve test of ZipFileResolver to behave on multiple platforms.
mjunkin Sep 6, 2024
8b1baae
Run formatter (again)
mjunkin Sep 6, 2024
750f720
Further VdypSpecies tests
mjunkin Sep 6, 2024
29265ed
Moved CachedResolvedControlMapTest to vdyp-common
mjunkin Sep 6, 2024
8870cc6
additional tests
mjunkin Sep 6, 2024
cda811e
more tests
mjunkin Sep 6, 2024
d552210
more tests
mjunkin Sep 6, 2024
8d62357
unti test correction
mjunkin Sep 6, 2024
d673ca8
More tests...
mjunkin Sep 6, 2024
00fe5fa
correct code quality issue
mjunkin Sep 6, 2024
c60977f
Code review changes
mjunkin Sep 10, 2024
210d841
Merge branch 'main' into feature/VDYP-400-grow-qmd+lh
mjunkin Sep 10, 2024
b1b7d54
Further adjustments after merge
mjunkin Sep 10, 2024
ff1731d
Replace VriAdjustInputWriter with VdypOutputWriter and remove VRI_OUT…
mjunkin Sep 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 0 additions & 48 deletions vdyp-lib/vdyp-common/GenusDefinitionMapTest.java

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package ca.bc.gov.nrs.vdyp.application;

public class InitializationIncompleteException extends RuntimeException {

private static final long serialVersionUID = -4549468703592060502L;

public InitializationIncompleteException(String reason) {
super(reason);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,14 @@
import ca.bc.gov.nrs.vdyp.common.Utils;
import ca.bc.gov.nrs.vdyp.common.ValueOrMarker;
import ca.bc.gov.nrs.vdyp.common_calculators.BaseAreaTreeDensityDiameter;
import ca.bc.gov.nrs.vdyp.controlmap.ResolvedControlMapImpl;
import ca.bc.gov.nrs.vdyp.io.FileSystemFileResolver;
import ca.bc.gov.nrs.vdyp.io.parse.coe.UpperCoefficientParser;
import ca.bc.gov.nrs.vdyp.io.parse.common.ResourceParseException;
import ca.bc.gov.nrs.vdyp.io.parse.control.BaseControlParser;
import ca.bc.gov.nrs.vdyp.io.parse.streaming.StreamingParser;
import ca.bc.gov.nrs.vdyp.io.parse.streaming.StreamingParserFactory;
import ca.bc.gov.nrs.vdyp.io.write.VriAdjustInputWriter;
import ca.bc.gov.nrs.vdyp.io.write.VdypOutputWriter;
import ca.bc.gov.nrs.vdyp.math.FloatMath;
import ca.bc.gov.nrs.vdyp.model.BaseVdypLayer;
import ca.bc.gov.nrs.vdyp.model.BaseVdypPolygon;
Expand Down Expand Up @@ -164,7 +165,7 @@ protected static void doMain(VdypStartApplication<?, ?, ?, ?> app, final String.
.filter(x -> !x.getName().contains("Volume")).toList();
}

protected VriAdjustInputWriter vriWriter;
protected VdypOutputWriter vriWriter;

protected Map<String, Object> controlMap = new HashMap<>();

Expand Down Expand Up @@ -226,7 +227,7 @@ public void init(FileSystemFileResolver resolver, Map<String, Object> controlMap

setControlMap(controlMap);
closeVriWriter();
vriWriter = new VriAdjustInputWriter(controlMap, resolver);
vriWriter = new VdypOutputWriter(controlMap, resolver);
}

protected abstract BaseControlParser getControlFileParser();
Expand All @@ -240,7 +241,7 @@ void closeVriWriter() throws IOException {

protected void setControlMap(Map<String, Object> controlMap) {
this.controlMap = controlMap;
this.estimationMethods = new EstimationMethods(controlMap);
this.estimationMethods = new EstimationMethods(new ResolvedControlMapImpl(controlMap));
}

protected <T> StreamingParser<T> getStreamingParser(ControlKey key) throws ProcessingException {
Expand Down Expand Up @@ -271,8 +272,8 @@ protected Coefficients getCoeForSpecies(BaseVdypSpecies<?> species, ControlKey c
protected L requireLayer(P polygon, LayerType type) throws ProcessingException {
if (!polygon.getLayers().containsKey(type)) {
throw validationError(
"Polygon \"%s\" has no %s layer, or that layer has non-positive height or crown closure.", polygon
.getPolygonIdentifier(), type
"Polygon \"%s\" has no %s layer, or that layer has non-positive height or crown closure.",
polygon.getPolygonIdentifier(), type
);
}

Expand All @@ -293,8 +294,8 @@ protected float getPercentTotal(L layer) throws StandProcessingException {
.sum();
if (Math.abs(percentTotal - 100f) > 0.01f) {
throw validationError(
"Polygon \"%s\" has %s layer where species entries have a percentage total that does not sum to 100%%.", layer
.getPolygonIdentifier(), LayerType.PRIMARY
"Polygon \"%s\" has %s layer where species entries have a percentage total that does not sum to 100%%.",
layer.getPolygonIdentifier(), LayerType.PRIMARY
);
}
return percentTotal;
Expand Down Expand Up @@ -323,7 +324,7 @@ protected List<S> findPrimarySpecies(Collection<S> allSpecies) {
continue;
}
var groupPrimary = copySpecies(
// groupSpecies.size() is at least 2 so findFirest will not be empty
// groupSpecies.size() is at least 2 so findFirst will not be empty
groupSpecies.stream().sorted(PERCENT_GENUS_DESCENDING).findFirst().orElseThrow(), builder -> {
var total = (float) groupSpecies.stream().mapToDouble(BaseVdypSpecies::getPercentGenus).sum();
builder.percentGenus(total);
Expand Down Expand Up @@ -507,7 +508,7 @@ public int findEmpiricalRelationshipParameterIndex(String specAlias, BecDefiniti
return group;
}

protected VriAdjustInputWriter getVriWriter() {
protected VdypOutputWriter getVriWriter() {
return vriWriter;
}

Expand All @@ -533,8 +534,8 @@ protected float estimatePrimaryBaseArea(

var decayBecAlias = bec.getDecayBec().getAlias();
Coefficients coe = weightedCoefficientSum(
List.of(0, 1, 2, 3, 4, 5), 9, 0, layer.getSpecies()
.values(), BaseVdypSpecies::getFractionGenus, s -> coeMap.get(decayBecAlias, s.getGenus())
List.of(0, 1, 2, 3, 4, 5), 9, 0, layer.getSpecies().values(), BaseVdypSpecies::getFractionGenus,
s -> coeMap.get(decayBecAlias, s.getGenus())
);

float ageToUse = clamp(breastHeightAge, 5f, 350f);
Expand Down Expand Up @@ -847,8 +848,8 @@ protected float estimatePrimaryQuadMeanDiameter(

var decayBecAlias = bec.getDecayBec().getAlias();
Coefficients coe = weightedCoefficientSum(
List.of(0, 1, 2, 3, 4), 8, 0, layer.getSpecies()
.values(), BaseVdypSpecies::getFractionGenus, s -> coeMap.get(decayBecAlias, s.getGenus())
List.of(0, 1, 2, 3, 4), 8, 0, layer.getSpecies().values(), BaseVdypSpecies::getFractionGenus,
s -> coeMap.get(decayBecAlias, s.getGenus())
);

var trAge = log(clamp(breastHeightAge, 5f, 350f));
Expand Down Expand Up @@ -1192,7 +1193,8 @@ public void computeUtilizationComponentsPrimary(
// Calculate tree density components
for (var uc : VdypStartApplication.UTIL_CLASSES) {
treesPerHectareUtil.set(
uc, BaseAreaTreeDensityDiameter
uc,
BaseAreaTreeDensityDiameter
.treesPerHectare(baseAreaUtil.getCoe(uc.index), quadMeanDiameterUtil.get(uc))
);
}
Expand All @@ -1210,7 +1212,8 @@ public void computeUtilizationComponentsPrimary(

for (var uc : VdypStartApplication.UTIL_CLASSES) {
treesPerHectareUtil.setCoe(
uc.index, BaseAreaTreeDensityDiameter
uc.index,
BaseAreaTreeDensityDiameter
.treesPerHectare(baseAreaUtil.getCoe(uc.index), quadMeanDiameterUtil.getCoe(uc.index))
);
}
Expand All @@ -1227,9 +1230,8 @@ public void computeUtilizationComponentsPrimary(

// EMP091
estimationMethods.estimateWholeStemVolume(
UtilizationClass.ALL, adjustCloseUtil.getCoe(
4
), spec.getVolumeGroup(), loreyHeightSpec, quadMeanDiameterUtil, baseAreaUtil, wholeStemVolumeUtil
UtilizationClass.ALL, adjustCloseUtil.getCoe(4), spec.getVolumeGroup(), loreyHeightSpec,
quadMeanDiameterUtil, baseAreaUtil, wholeStemVolumeUtil
);

if (compatibilityVariableMode == CompatibilityVariableMode.ALL) {
Expand All @@ -1244,28 +1246,28 @@ public void computeUtilizationComponentsPrimary(

// EMP092
estimationMethods.estimateCloseUtilizationVolume(
UtilizationClass.ALL, adjustCloseUtil, spec
.getVolumeGroup(), loreyHeightSpec, quadMeanDiameterUtil, wholeStemVolumeUtil, closeVolumeUtil
UtilizationClass.ALL, adjustCloseUtil, spec.getVolumeGroup(), loreyHeightSpec,
quadMeanDiameterUtil, wholeStemVolumeUtil, closeVolumeUtil
);

// EMP093
estimationMethods.estimateNetDecayVolume(
spec.getGenus(), bec.getRegion(), UtilizationClass.ALL, adjustCloseUtil, spec
.getDecayGroup(), vdypLayer.getBreastHeightAge()
.orElse(0f), quadMeanDiameterUtil, closeVolumeUtil, closeVolumeNetDecayUtil
spec.getGenus(), bec.getRegion(), UtilizationClass.ALL, adjustCloseUtil, spec.getDecayGroup(),
vdypLayer.getBreastHeightAge().orElse(0f), quadMeanDiameterUtil, closeVolumeUtil,
closeVolumeNetDecayUtil
);

// EMP094
estimationMethods.estimateNetDecayAndWasteVolume(
bec.getRegion(), UtilizationClass.ALL, adjustCloseUtil, spec
.getGenus(), loreyHeightSpec, quadMeanDiameterUtil, closeVolumeUtil, closeVolumeNetDecayUtil, closeVolumeNetDecayWasteUtil
bec.getRegion(), UtilizationClass.ALL, adjustCloseUtil, spec.getGenus(), loreyHeightSpec,
quadMeanDiameterUtil, closeVolumeUtil, closeVolumeNetDecayUtil, closeVolumeNetDecayWasteUtil
);

if (this.getId().isStart()) {
// EMP095
estimationMethods.estimateNetDecayWasteAndBreakageVolume(
UtilizationClass.ALL, spec
.getBreakageGroup(), quadMeanDiameterUtil, closeVolumeUtil, closeVolumeNetDecayWasteUtil, closeVolumeNetDecayWasteBreakUtil
UtilizationClass.ALL, spec.getBreakageGroup(), quadMeanDiameterUtil, closeVolumeUtil,
closeVolumeNetDecayWasteUtil, closeVolumeNetDecayWasteBreakUtil
);
}
}
Expand Down Expand Up @@ -1316,12 +1318,12 @@ public void computeUtilizationComponentsPrimary(
"For species {}, Species LH (7.5cm+): {}, Species BA (7.5cm+): {}, Weighted LH (7.5cm+): {}"
).log();
vdypLayer.getLoreyHeightByUtilization().scalarInPlace(
UtilizationClass.SMALL, x -> x
+ spec.getLoreyHeightByUtilization().getSmall() * spec.getBaseAreaByUtilization().getSmall()
UtilizationClass.SMALL,
x -> x + spec.getLoreyHeightByUtilization().getSmall() * spec.getBaseAreaByUtilization().getSmall()
);
vdypLayer.getLoreyHeightByUtilization().scalarInPlace(
UtilizationClass.ALL, x -> x
+ spec.getLoreyHeightByUtilization().getAll() * spec.getBaseAreaByUtilization().getAll()
UtilizationClass.ALL,
x -> x + spec.getLoreyHeightByUtilization().getAll() * spec.getBaseAreaByUtilization().getAll()
);
}
{
Expand Down Expand Up @@ -1354,14 +1356,14 @@ protected void computeLayerUtilizationComponentsFromSpecies(VdypLayer vdypLayer)
vdypLayer.getSpecies().values().stream().forEach(spec -> {
var ba = spec.getBaseAreaByUtilization();
hlVector.pairwiseInPlace(
spec.getLoreyHeightByUtilization(), (float x, float y, UtilizationClass uc) -> x
+ y * ba.get(uc)
spec.getLoreyHeightByUtilization(),
(float x, float y, UtilizationClass uc) -> x + y * ba.get(uc)
);
});
var ba = vdypLayer.getBaseAreaByUtilization();
hlVector.scalarInPlace((float x, UtilizationClass uc) -> ba.get(uc) > 0 ? x / ba.get(uc) : x);

// Update percent based on updated areas
// Update percent based on updated areas
vdypLayer.getSpecies().values().stream().forEach(spec -> {
spec.setPercentGenus(100 * spec.getBaseAreaByUtilization().getAll() / ba.getAll());
});
Expand Down Expand Up @@ -1456,44 +1458,38 @@ protected void computeUtilizationComponentsVeteran(VdypLayer vdypLayer, BecDefin
var adjust = new Coefficients(new float[] { 0f, 0f, 0f, 0f }, 1);

// EMP091
EstimationMethods.estimateWholeStemVolume(
controlMap, utilizationClass, volumeAdjustCoe.getCoe(1), vdypSpecies.getVolumeGroup(), hlSp,
estimationMethods.estimateWholeStemVolume(
utilizationClass, volumeAdjustCoe.getCoe(1), vdypSpecies.getVolumeGroup(), hlSp,
quadMeanDiameterUtil, baseAreaUtil, wholeStemVolumeUtil
);

adjust.setCoe(4, volumeAdjustCoe.getCoe(2));
// EMP092
EstimationMethods.estimateCloseUtilizationVolume(
controlMap, utilizationClass, adjust, vdypSpecies.getVolumeGroup(), hlSp, quadMeanDiameterUtil,
estimationMethods.estimateCloseUtilizationVolume(
utilizationClass, adjust, vdypSpecies.getVolumeGroup(), hlSp, quadMeanDiameterUtil,
wholeStemVolumeUtil, closeUtilizationVolumeUtil
);

adjust.setCoe(4, volumeAdjustCoe.getCoe(3));
// EMP093
EstimationMethods.estimateNetDecayVolume(
controlMap, vdypSpecies.getGenus(), bec.getRegion(), utilizationClass, adjust,
vdypSpecies.getDecayGroup(), vdypLayer.getBreastHeightAge().orElse(0f), quadMeanDiameterUtil,
closeUtilizationVolumeUtil, closeUtilizationNetOfDecayUtil
estimationMethods.estimateNetDecayVolume(
vdypSpecies.getGenus(), bec.getRegion(), utilizationClass, adjust, vdypSpecies.getDecayGroup(),
vdypLayer.getBreastHeightAge().orElse(0f), quadMeanDiameterUtil, closeUtilizationVolumeUtil,
closeUtilizationNetOfDecayUtil
);

adjust.setCoe(4, volumeAdjustCoe.getCoe(4));
// EMP094
final var netDecayCoeMap = Utils.<Map<String, Coefficients>>expectParsedControl(
controlMap, ControlKey.VOLUME_NET_DECAY_WASTE, java.util.Map.class
);
final var wasteModifierMap = Utils.<MatrixMap2<String, Region, Float>>expectParsedControl(
controlMap, ControlKey.WASTE_MODIFIERS, ca.bc.gov.nrs.vdyp.model.MatrixMap2.class
);
EstimationMethods.estimateNetDecayAndWasteVolume(
bec.getRegion(), utilizationClass, adjust, vdypSpecies.getGenus(), hlSp, netDecayCoeMap,
wasteModifierMap, quadMeanDiameterUtil, closeUtilizationVolumeUtil,
closeUtilizationNetOfDecayUtil, closeUtilizationNetOfDecayAndWasteUtil
estimationMethods.estimateNetDecayAndWasteVolume(
bec.getRegion(), utilizationClass, adjust, vdypSpecies.getGenus(), hlSp, quadMeanDiameterUtil,
closeUtilizationVolumeUtil, closeUtilizationNetOfDecayUtil,
closeUtilizationNetOfDecayAndWasteUtil
);

if (getId().isStart()) {
// EMP095
EstimationMethods.estimateNetDecayWasteAndBreakageVolume(
controlMap, utilizationClass, vdypSpecies.getBreakageGroup(), quadMeanDiameterUtil,
estimationMethods.estimateNetDecayWasteAndBreakageVolume(
utilizationClass, vdypSpecies.getBreakageGroup(), quadMeanDiameterUtil,
closeUtilizationVolumeUtil, closeUtilizationNetOfDecayAndWasteUtil,
closeUtilizationNetOfDecayWasteAndBreakageUtil
);
Expand Down
Loading
Loading