Improved NoData handling [Science]

[gtamkin]

Make NDV specifiable in CLI. If not specified, default to TOA NDV. Use consistently through entire application (regression, CSV, etc.)

[gtamkin]

[10/30/23 6:02 PM] Tamkin, Glenn S. (GSFC-606.3)[InuTeq, LLC]
SR-Lite & Senegal observations after our 10/30/23 ~2 PM conversation:

• The original SR-Lite testing included TOA/CCDC/Cloudmask trios where ALL files had the same NoData value (-9999)

• Unfortunately, the CSV statistics calculation, which occurs after the COG is processed, currently assumes that the NoData value is always -9999. So if the TOA NoData value = -100001, it will be reprojected. At best, this wastes CPU cycles and takes much longer than if the TOA is -9999. At worst, our calculations are simply wrong. Still investigating this, but clearly we don't want to reproject the largest input file (TOA) to match the others (CCDC, Cloudmask) if we can avoid it. Note that even if the CSV calculations are wrong, this wouldn't impact the visual map accuracy.

3. Mark has made it clear that we always want the TOA attributes to drive any reprojections. Again, this is first tests where TOA does not match the CCDC NoData value, so the runs take longer and need more evaluation.

4. Any of the above can be changed, but Mark would have to direct another release. SR-Lite could certainly be tweaked, but in the short term, I wonder how costly it is to create TOA/CCDC/Cloudmask trios that all have a NoData value = -9999.

In general, I agree that SR-Lite should be able to cleanly handle NoData value mismatches. However, this is an untested scenario (by me at least). So, I'll need to closely debug the application. In the meantime, is it possible for someone to set the NoData value to -9999 for a single TOA/CCDC/Cloudmask set and rerun SR-Lite as test?

heart 1
[10/30/23 7:03 PM] Tamkin, Glenn S. (GSFC-606.3)[InuTeq, LLC]
Update #1: All of the above is still true. However, I just found a bug in the NoData value update logic. The bug occurs if the TOA and the CCDC have different NoData values. So, until there is an updated SR-Lite release available, the three inputs [TOA/CCDC/Cloudmask] have to be aligned to the same NoData value of -9999 before running SR-Lite.
heart 1

[gtamkin]

added alignNoDataValue() to always use TOA idv :
context[Context.TARGET_NODATA_VALUE] = dst_ndv = self.get_ndv(str(context[Context.FN_TOA]))
This method is called when re-projecting images.