Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Redshift's drop column statement causes dbt to fail if a column name begins with a question mark #841

Open
4 of 11 tasks
MiConnell opened this issue Feb 22, 2025 · 0 comments · May be fixed by #842
Open
4 of 11 tasks
Labels
triage:product In Product's queue type:bug Something isn't working as documented

Comments

@MiConnell
Copy link

Is this a new bug?

  • I believe this is a new bug
  • I have searched the existing issues, and I could not find an existing issue for this bug

Which packages are affected?

  • dbt-adapters
  • dbt-tests-adapter
  • dbt-athena
  • dbt-athena-community
  • dbt-bigquery
  • dbt-postgres
  • dbt-redshift
  • dbt-snowflake
  • dbt-spark

Current Behavior

I had an errant column in a table that was named ?column?, and dbt failed when trying to drop the column with the below error.

Database Error in model dim_date (models/marts/dim_date.sql)
  syntax error at or near "?" in context "."dim_date" drop column ?", at line 3, column 65

This only happens on incremental models as far as I can tell since dbt does not drop and recreate the table.

I believe double quoting column names should fix this. I'm unable to test on other adapters but believe the same issue would occur.

Expected Behavior

dbt successfully drops all unused columns in a table regardless of their naming format by double quoting the column name.

Steps To Reproduce

  1. Create an incremental model in dbt
  2. Manually add a column named ?column? to the table with alter table my_model add column "?column?"
  3. Run dbt run -s my_model

Relevant log output

In "prod"."dev_mc"."dim_date":
        Schema changed: True
        Source columns not in target: []
        Target columns not in source: [Column(column='?column?', dtype='integer', char_size=None, numeric_precision=32, numeric_scale=0)]
        New column types: []
...
[0m12:21:45.606694 [debug] [Thread-1 (]: On model.edw.dim_date: /* {"app": "dbt", "dbt_version": "1.9.2", "profile_name": "edw", "target_name": "prod", "node_id": "model.edw.dim_date"} */

    alter table "prod"."dev_mc"."dim_date" drop column ?column?
[0m12:21:45.671261 [debug] [Thread-1 (]: Redshift adapter: Redshift error: syntax error at or near "?" in context "."dim_date" drop column ?", at line 3, column 65

Environment

- OS: MacOS 15.3.1
- Python: Python 3.10.12
- dbt-adapters:
Core:
  - installed: 1.9.2
  - latest:    1.9.2 - Up to date!

Plugins:
  - redshift: 1.9.0 - Up to date!
  - postgres: 1.9.0 - Up to date!

Additional Context

No response

@MiConnell MiConnell added triage:product In Product's queue type:bug Something isn't working as documented labels Feb 22, 2025
@MiConnell MiConnell linked a pull request Feb 22, 2025 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage:product In Product's queue type:bug Something isn't working as documented
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant