Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support for new microbatching incremental strategy #715

Open
1 task done
jessedobbelaere opened this issue Sep 11, 2024 · 0 comments
Open
1 task done
Labels
enhancement New feature or request

Comments

@jessedobbelaere
Copy link
Contributor

Is this your first time submitting a feature request?

  • I have searched the existing issues, and I could not find an existing issue for this feature

Describe the feature

Previously, for large time-series models we needed a custom insert-by-period macro. In this post dbt announced support for the "microbatching" strategy coming to dbt-core 1.9.

This issue is a placeholder to start preparing for the dbt 1.9 release with this new incremental strategy. Work can be tracked in this epic dbt-labs/dbt-core#10624

Describe alternatives you've considered

No response

Who will this benefit?

Datasets that are too big to run on initial full load without batching (Athena query times out)

Are you interested in contributing this feature?

No response

Anything else?

Closes #190 and probably #697

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant