You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a function to the extraction pipeline that takes a job definition JSON/dictionary as input, then validates it against the criteria outlined in config/table_definitions.yaml. The goal is to raise exceptions early when passing jobs that are invalid for a given table.
Some possible checks to make:
min_year should always be <= max_year
Job batches should contain only a single job for a given table
Jobs are within the parameters outlined in config/table_definitions.yaml
taxyr values are numeric (and years)
cur values are one of Y, N, or D
The text was updated successfully, but these errors were encountered:
Add a function to the extraction pipeline that takes a job definition JSON/dictionary as input, then validates it against the criteria outlined in
config/table_definitions.yaml
. The goal is to raise exceptions early when passing jobs that are invalid for a given table.Some possible checks to make:
min_year
should always be <=max_year
config/table_definitions.yaml
taxyr
values are numeric (and years)cur
values are one ofY
,N
, orD
The text was updated successfully, but these errors were encountered: