Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Change in Defined Redshift Sort Key Sequence During Execution #865

Open
2 tasks done
hareesh-anjane opened this issue Feb 7, 2025 · 1 comment
Open
2 tasks done

Comments

@hareesh-anjane
Copy link

hareesh-anjane commented Feb 7, 2025

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

I am using dbt to create materialized view in Redshift along with distribution and sort keys defined. Upon creation, I noticed that sort key ordering is different than the one provided in config block or yaml file(tried both the config block and yaml file).

Adapter: dbt-redshift 1.8.1

Expected Behavior


config(
        dist="all",
        sort= ["column_1","column_2","column_3","column_4","column_5"]
    )

Expected output in dbt.log file

create materialized view "warehouse"."materialized_view_example"
        backup yes
        diststyle all
        sortkey (column_1,Column_2,column_3,Column_4,Column_5)

Steps To Reproduce

Define the sort keys and dist keys in config block or in model.yml file .

I chose to define them in config block as per dbt docs


config(
        dist="all",
        sort= ["column_1","column_2","column_3","column_4","column_5"]
    )

Refer the dbt.log file for create materialized view line and check the sequencing of sort keys:

create materialized view "warehouse"."materialized_view_example"
        backup yes
        diststyle all
        sortkey (column_5,Column_2,column_3,Column_1,Column_4)

Relevant log output

Environment

- OS: Windows 11
- Python: pyenv
- dbt: dbt-core 1.8.0

Which database adapter are you using with dbt?

redshift

Additional Context

https://docs.getdbt.com/reference/resource-configs/redshift-configs#using-sortkey-and-distkey

Tables in Amazon Redshift have two powerful optimizations to improve query performance: distkeys and sortkeys. Supplying these values as model-level configurations apply the corresponding settings in the generated CREATE TABLE DDL. Note that these settings will have no effect on models set to view or ephemeral models.

dist can have a setting of all, even, auto, or the name of a key.
sort accepts a list of sort keys, for example: ['reporting_day', 'category']. dbt will build the sort key in the same order the fields are supplied.
sort_type can have a setting of interleaved or compound. if no setting is specified, sort_type defaults to compound.

@dbeatty10
Copy link
Contributor

Thanks for reaching out @hareesh-anjane !

Since sort keys are Redshift-specific behavior, I'm going to transfer this to dbt-adapters repo where the code for the dbt-redshift adapter is hosted.

Original issue in dbt-core: #865

@dbeatty10 dbeatty10 transferred this issue from dbt-labs/dbt-core Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants