Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect expected types in config block #1907

Open
mattimoller opened this issue Feb 3, 2025 · 1 comment
Open

Incorrect expected types in config block #1907

mattimoller opened this issue Feb 3, 2025 · 1 comment

Comments

@mattimoller
Copy link

Seems like 3.0.10 and specifically #1780 introduced some changes to the expected properties of the config block and making a bunch of the fields required vs optional. Seems like the only way to get typing to work now is to write out all the 20+ required fields in the config, which isn't a viable option. I'm no TS expert so could be something wrong with our setup here, open to any suggestions!

Consider the below my_table.js

publish(`my_table`, {
    type: "table",
    schema: "my_schema",
    bigquery: {
        labels: {
            source: "my_source"
        }
    },
    tags: ["table"],
    assertions: {
        nonNull: ["id"]
    }
}).query(`select 1 as id`)

This table compiles and can be run both in Dataform 3.0.9 and 3.0.10 but in 3.0.10 our type checks complain about the below

Error

definitions/my_table.js:1:21 - error TS2345: Argument of type '{ type: string; schema: string; bigquery: { labels: { source: string; }; }; tags: string[]; assertions: { nonNull: string[]; }; }' is not assignable to parameter of type 'Contextable<ITableContext, string> | ILegacyTableConfig | ILegacyViewConfig | ILegacyIncrementalTableConfig | undefined'.
  Type '{ type: string; schema: string; bigquery: { labels: { source: string; }; }; tags: string[]; assertions: { nonNull: string[]; }; }' is missing the following properties from type 'ILegacyIncrementalTableConfig': dependencies, database, fileName, name, and 21 more.

Steps to recreate:

  1. Install dataform version 3.0.10 or higher (see attached package.json)
  2. Run npx tsc --project jsconfig.json --noEmit --allowJs --checkJs;

Package.json dependencies:

{
  "name": "any_name",
  "dependencies": {
    "@dataform/core": "3.0.10",
    "dataform-scd": "https://github.com/dataform-co/dataform-scd/archive/0.5.tar.gz",
    "date-fns": "4.1.0"
  },
  "devDependencies": {
    "@eslint/js": "^9.19.0",
    "@types/jest": "^29.5.14",
    "eslint": "^9.19.0",
    "eslint-plugin-jest": "^28.11.0",
    "globals": "^15.14.0",
    "jest": "^29.7.0",
    "prettier": "^3.4.2",
    "typescript": "^5.7.3"
  }
}

Jsconfig
jsconfig.json

Imported dataform types (types/dataform/globals.d.ts)

import { session } from "@dataform/core"
import datasets = require("../../includes/datasets.js")

declare global {
  const publish: typeof session.publish
  const declare: typeof session.declare
  const assert: typeof session.assert
  const operate: typeof session.operate
  const dataform: { projectConfig: typeof session.projectConfig }
  const datasets: typeof datasets
}
@Ekrekr
Copy link
Contributor

Ekrekr commented Feb 3, 2025

Hi Mathias,

Thanks for the heads up. I expect a bug was introduced in #1780 that is causing this. We'll patch a fix for it in the next few days - for now just stick with 3.0.9.

Thanks,
Elias

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants