Skip to content

Commit

Permalink
add artifacts_static_datasets var to upload_results
Browse files Browse the repository at this point in the history
  • Loading branch information
npeshkov committed Dec 9, 2024
1 parent 91d0a6e commit 8a13e3b
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 0 deletions.
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,24 @@ Note that model materializations and `on_schema_change` configs are defined in t

> Configurations made in your dbt_project.yml file will override any configurations in a package (either in the dbt_project.yml file of the package, or in config blocks).

### Artifact to upload

By default, all non-execution artifacts are uploaded. Some users may be interested in choosing which datasets to upload. Thus, we allow setting `artifact_static_datasets` variable:

```yml
vars:
artifacts_static_datasets: ['exposures', 'seeds', 'snapshots', 'invocations', 'sources', 'tests', 'models']
```

> Note: execution-type arficats are always uploaded.

It is possible to upload different sets of datasets depending on which dbt command is run. For example, upload all datasets on `dbt run`, upload only execution-type artifacts for other dbt invocations:

```yml
vars:
artifacts_static_datasets: "{{ 'all' if invocation_args_dict.which == 'run' else [] }}"
```

### Environment Variables

If the project is running in dbt Cloud, the following five columns (<https://docs.getdbt.com/docs/dbt-cloud/using-dbt-cloud/cloud-environment-variables#special-environment-variables>) will be automatically populated in the fct_dbt__invocations model:
Expand Down
6 changes: 6 additions & 0 deletions macros/upload_results/upload_results.sql
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@
{% if execute %}

{% set datasets_to_load = ['exposures', 'seeds', 'snapshots', 'invocations', 'sources', 'tests', 'models'] %}

{# When 'artifacts_static_datasets' var is provided, use it instead of default datasets #}
{% if var("artifacts_static_datasets", "all") != "all" %}
{% set datasets_to_load = fromjson(var("artifacts_static_datasets")) %}
{% endif %}

{% if results != [] %}
{# When executing, and results are available, then upload the results #}
{% set datasets_to_load = ['model_executions', 'seed_executions', 'test_executions', 'snapshot_executions'] + datasets_to_load %}
Expand Down

0 comments on commit 8a13e3b

Please sign in to comment.