Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

redshift_admin_table_stats.is_encoded is always true #43

Open
1 of 5 tasks
tpilewicz opened this issue Apr 30, 2021 · 0 comments
Open
1 of 5 tasks

redshift_admin_table_stats.is_encoded is always true #43

tpilewicz opened this issue Apr 30, 2021 · 0 comments
Labels

Comments

@tpilewicz
Copy link

Describe the bug

From my understanding, field is_encoded from model redshift_admin_table_stats should indicate whether the table we're looking at has at least one encoded column. But currently the field is computed by looking in pg_attribute if any table attribute is encoded. The result is that if at least one attribute is encoded, the table appears as having at least one column encoded. Worse: in our redshift, the tableoid attribute is almost always encoded. Hence all tables have their is_encoded field set to true.

Steps to reproduce

Run the following in your redshift:

select attencodingtype, count(1)
from pg_attribute
where attname = 'tableoid'
group by 1;

And the following:

select is_encoded count(1)
from redshift_admin_table_stats
group by 1;

Expected results

We have some tables for which no column is encoded. They should appear when running the above query.

Actual results

In redshift_admin_table_stats, the tables I just mentioned are flagged as is_encoded, which should not be the case.

System information

The contents of your packages.yml file:

packages:
  - package: fishtown-analytics/dbt_utils
    version: 0.6.3
  - package: fishtown-analytics/redshift
    version: 0.4.1
  - package: fivetran/jira_source
    version: [">=0.1.0","<0.2.0"]
  - package: fivetran/jira
    version: 0.1.5
  - package: fivetran/intercom_source
    version: 0.1.2
  - package: fivetran/intercom
    version: 0.1.2

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

installed version: 0.18.0
   latest version: 0.19.1

Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation

Plugins:
  - bigquery: 0.18.0
  - snowflake: 0.18.0
  - redshift: 0.18.0
  - postgres: 0.18.0

The operating system you're using:
OSX 11.2.3

The output of python --version:
Python 3.6.8

Additional context

Willing to open a PR if you confirm this should be the desired behavior. From what I understand we just need to restrict the max(attencodingtype) to attributes that have a strictly positive attnum?

@tpilewicz tpilewicz added the bug label Apr 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant