-
-
Notifications
You must be signed in to change notification settings - Fork 119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Normalize VCE RARE spellings of great lakes. #4029
base: main
Are you sure you want to change the base?
Conversation
RTD build is failing due to Dagster removing their intersphinx .inv file; see also dagster-io/dagster#27328 |
I've commented out the Dagster intersphinx line on |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! I think we're probably good to merge without this, but also we might consider adding lakes Hurron and St. Clair to the check_unexpected_counties
asset check.
src/pudl/transform/vcerare.py
Outdated
@@ -369,6 +391,7 @@ def check_rows(context: AssetCheckExecutionContext) -> AssetCheckResult: | |||
row_counts = { | |||
"etl_full": 136437000, | |||
"etl_fast": 27287400, | |||
"__ASSET_JOB": 136437000, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FMI: Is this is the default job name when you materialize an asset outside of a job?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we could swap this to the annual checks like we do for ferc714
For more information, see https://pre-commit.ci
Overview
Closes #4007 .
What problem does this address?
The names of a couple of the great lakes in the VCE RARE dataset had unusual or creative spellings. This PR normalizes the spelling of Lake Huron and Lake Saint Clair in VCE RARE.
What did you change?
_spot_fix_great_lakes_values
called by_prep_lat_long_fips_df
to catch the cell values_spot_fix_great_lakes_columns
called byone_year_hourly_available_capacity_factor
to catch the column namesDocumentation
Make sure to update relevant aspects of the documentation.
Tasks
Testing
How did you make sure this worked? How can a reviewer verify this?
core_vcerare
asset group in Dagsterout_vcerare__hourly_available_capacity_factor
and observe:lake_hurron
does not appear in columncounty_id_fips
;lake_huron
does appearlake_st_clair
does not appear in columncounty_id_fips
;lake_saint_clair
does appeare.g.
To-do list