Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not able to reference _TABLE_SUFFIX column in limited queries of wildcard table sources #868

Open
3 of 11 tasks
jchacko-atg opened this issue Mar 4, 2025 · 0 comments
Open
3 of 11 tasks
Labels
triage:product In Product's queue type:bug Something isn't working as documented

Comments

@jchacko-atg
Copy link

Is this a new bug?

  • I believe this is a new bug
  • I have searched the existing issues, and I could not find an existing issue for this bug

Which packages are affected?

  • dbt-adapters
  • dbt-tests-adapter
  • dbt-athena
  • dbt-athena-community
  • dbt-bigquery
  • dbt-postgres
  • dbt-redshift
  • dbt-snowflake
  • dbt-spark

Current Behavior

When running dbt run with the --empty flag wildcard tables aren't able to reference the _TABLE_SUFFIX column. This is because the _TABLE_SUFFIX isn't implicitly passed through with the select *.

For example,

    select
        *,
        _TABLE_SUFFIX as table_suffix
    from {{ source('sharded_table') }}
    where length(_TABLE_SUFFIX) = 8

compiles to,

    select
        *,
        _TABLE_SUFFIX as table_suffix
    from (select * from `project`.`dataset`.`sharded_table_*` where false limit 0)
    where length(_TABLE_SUFFIX) = 8

Expected Behavior

If the table id ends in a *, the adapter should pass through the table suffix column for limited queries.

    select
        *,
        _TABLE_SUFFIX as table_suffix
    from (select *, _TABLE_SUFFIX from `project`.`dataset`.`sharded_table_*` where false limit 0)
    where length(_TABLE_SUFFIX) = 8

Steps To Reproduce

  1. Create a source with a wildcard table.
  2. Reference it in a model and attempt to query the _TABLE_SUFFIX column. This query should build with dbt build <model>.
  3. Now try dbt build <model> --empty. You'll hit an error: Unrecognized name: _TABLE_SUFFIX at ....

Relevant log output

Environment

- OS: Linux (6.5.0-1025-azure)
- Python: Python 3.11.8
- dbt: 1.9.2
- dbt-adapters:
   - bigquery: 1.9.1

Additional Context

No response

@jchacko-atg jchacko-atg added triage:product In Product's queue type:bug Something isn't working as documented labels Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage:product In Product's queue type:bug Something isn't working as documented
Projects
None yet
Development

No branches or pull requests

1 participant