Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Setting time_ingestion_partitioning to True in model config, creates empty table in Bigquery #866

Open
2 tasks done
Tonayya opened this issue Feb 19, 2025 · 3 comments

Comments

@Tonayya
Copy link

Tonayya commented Feb 19, 2025

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When the flag time_ingestion_partitioning is set to True on a model, the table that is created in Bigquery is empty/has no rows.

Steps to Reproduce

  • In a bare bone dbt Cloud project with a Bigquery connection, add a model as follows:

{{
    config(
        materialized="table",
        partition_by={
            "field": "created_at",
            "data_type": "timestamp",
            "granularity": "day",
            "time_ingestion_partitioning": true
        },
    )
}}

SELECT 1 as id, current_timestamp() as created_at
  • Run dbt run --select partitioning_stuff
  • We can see that no rows have been created:

Image

  • Checking in Bigquery, we can see that the table is empty:

Image

  • Dropping the table in Bigquery, then changing the model code to the below, where we remove the time_ingestion_partitioning flag:

{{
    config(
        materialized="table",
        partition_by={
            "field": "created_at",
            "data_type": "timestamp",
            "granularity": "day"
        },
    )
}}

SELECT 1 as id, current_timestamp() as created_at
  • Running dbt run --select partitioning_stuff
  • We can see that 1 row is created as expected:

Image

  • Checking in Bigquery, we can see that the row is created as expected as well:

Image

Expected Behavior

Tables should not be empty.

Steps To Reproduce

As outlined above.

Relevant log output

Environment

- OS:
- Python:
- dbt:

Which database adapter are you using with dbt?

No response

Additional Context

No response

@meioli
Copy link

meioli commented Feb 20, 2025

+1

1 similar comment
@trangkramp
Copy link

+1

@dbeatty10
Copy link
Contributor

Thanks for opening this @Tonayya !

Since time_ingestion_partitioning is BigQuery-specific behavior, I'm going to transfer this to dbt-adapters repo where the code for the dbt-bigquery adapter is hosted.

Original issue in dbt-core: #866

@dbeatty10 dbeatty10 transferred this issue from dbt-labs/dbt-core Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants