Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make asset checks run in integration test #3990

Merged
merged 5 commits into from
Dec 22, 2024
Merged

Conversation

jdangerx
Copy link
Member

@jdangerx jdangerx commented Dec 13, 2024

Overview

Closes #3928 .

What problem does this address?

#3711 inadvertently disabled the asset checks again, in service of fixing pudl_etl 😞

Getting rid of pudl_etl (#3161) seemed out of scope here, but I made the tests use the same Definitions as the nightly builds again.

Testing

How did you make sure this worked? How can a reviewer verify this?

Made the pandera checks all fail, ran make integration-test, and checked to make sure that the integration tests bailed out once we ran into a failure.

Then got other people to fix the failing asset checks and re-ran integration tests. They passed.

To-do list

Preview Give feedback

@jdangerx jdangerx force-pushed the 3928-asset-checks-run-again branch from 11ccfbe to ab560e3 Compare December 20, 2024 16:40
@jdangerx jdangerx changed the title WIP: Use same dagster Definitions for test and prod. Make asset checks run in integration test Dec 20, 2024
@jdangerx jdangerx marked this pull request as ready for review December 20, 2024 18:56
@jdangerx jdangerx enabled auto-merge December 20, 2024 19:00
Copy link
Member

@bendnorman bendnorman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay! Thanks for fixing this.

Comment on lines -53 to -57
return Definitions(
assets=pudl.etl.default_assets,
resources=pudl.etl.default_resources,
jobs=jobs,
).get_job_def("etl_job")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see... we redefined the definitions here but didn't add the asset checks so they weren't getting run. Importing defs from pudl.etl includes the assets checks.

@jdangerx jdangerx added this pull request to the merge queue Dec 22, 2024
Merged via the queue into main with commit ba58b94 Dec 22, 2024
17 checks passed
@jdangerx jdangerx deleted the 3928-asset-checks-run-again branch December 22, 2024 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Make asset checks run during integration tests
3 participants