Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ADAP-1131: Migrate dbt-redshift into dbt-adapters #610

Open
wants to merge 680 commits into
base: main
Choose a base branch
from
Open

ADAP-1131: Migrate dbt-redshift into dbt-adapters #610

wants to merge 680 commits into from
+9,130 −1

Conversation

mikealfare
Copy link
Contributor

Migrate dbt-redshift into this repo using the following shell script and call:

repo=$1
source_branch=$2
target_branch=$3

# create a remote for the legacy adapter repo and fetch the latest commits
git remote remove old || true
git remote add old https://github.com/dbt-labs/$repo.git
git fetch old

# merge the updated branch from the legacy repo into the dbt-adapters repo
git checkout $target_branch
git merge old/$source_branch --allow-unrelated-histories

# remove the remote that was created by this process
git remote remove old || true
source ./scripts/migrate-adapter.sh dbt-redshift monorepo/stage monorepo/migrate-dbt-redshift

Update workflows to include dbt-redshift.

Make updates related with code quality checks due to slightly different config.

Merge this PR as a regular merge instead of a squash merge in order to retain git blame history.

emmyoop and others added 30 commits March 28, 2023 17:40
@emmyoop
update to retrieve contract enforced from dict (#383)
c4d44b2 
* update to retrieve contract enforced from dict

* add dependecy on branch

* change ref

* Update Under the Hood-20230223-110314.yaml

* remove extra get

* revert branch dependency
@dependabot @VersusFacit
Update pytz requirement from ~=2022.7 to ~=2023.2 (#384)
97b7340 
Updates the requirements on [pytz](https://github.com/stub42/pytz) to permit the latest version.
- [Release notes](https://github.com/stub42/pytz/releases)
- [Commits](stub42/pytz@release_2022.7...release_2023.2)

---
updated-dependencies:
- dependency-name: pytz
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mila Page <[email protected]>
@dependabot @VersusFacit
Update types-pytz requirement from ~=2022.7 to ~=2023.2 (#385)
a9ae7ef 
Updates the requirements on [types-pytz](https://github.com/python/typeshed) to permit the latest version.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-pytz
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mila Page <[email protected]>
@FishtownBuildBot
Bumping version to 1.5.0b4 and generate changelog
b2c8039
@FishtownBuildBot
[Automated] Merged prep-release/1.5.0b4_4567977792 into target main d…
2bf02c1 
…uring release process
@nssalian
Test conversion for simple snapshot into a functional test (#397)
0682959 
* Test conversion Simple snapshot

* Remove branch change
@peterallenwebb
Model-Level Constraints (#395)
d348945 
* CT-2222: Redshift adapter support for model-level constrints

* CT-2222: Add changelog entry
@peterallenwebb
Repair accidental change to dev-requirements (#404)
5da1e4e
@VersusFacit
fix the json obj in msgs bug for both text and Json logs (#398)
6faee98 
* fix the json obj in msgs bug for both text and Json logs

* Clean up the code and fix the second bug by using a dummy message

* make the changie the log message a mite more explicit

---------

Co-authored-by: Mila Page <[email protected]>
@emmyoop
update to generalize constraints (#393)
ee1d08d 
* update to generalize constraints

* fix primary key logic

* remove special logic for primary key

* fix mypy

* update dev requirements
@FishtownBuildBot @McKnight-42
Cleanup main after cutting new 1.5.latest branch (#407)
6254b0f 
* Bumping version to 1.5.0rc1

* Clean up changelog on main

* pre-commit fixes

* update RELEASE_BRANCH env

---------

Co-authored-by: Matthew McKnight <[email protected]>
@github-actions @FishtownBuildBot
[create-pull-request] automated change (#411)
2f2f095 
Co-authored-by: Github Build Bot <[email protected]>
@emmyoop
revert changes to point to 1.4.latest (#412)
e31b4ba 
until 1.5.final is released
@dependabot
Update pip-tools requirement from ~=6.12 to ~=6.13 (#400)
abc7368 
Updates the requirements on [pip-tools](https://github.com/jazzband/pip-tools) to permit the latest version.
- [Release notes](https://github.com/jazzband/pip-tools/releases)
- [Changelog](https://github.com/jazzband/pip-tools/blob/main/CHANGELOG.md)
- [Commits](jazzband/pip-tools@6.12.0...6.13.0)

---
updated-dependencies:
- dependency-name: pip-tools
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@mikealfare @FishtownBuildBot
Update pytest requirement from ~=7.2 to ~=7.3 (#414)
f059c25 
* upgrading pytest

* Add automated changelog yaml from template for bot PR

* linter

---------

Co-authored-by: Github Build Bot <[email protected]>
@dependabot @mikealfare
Bump mypy from 1.1.1 to 1.2.0 (#396)
ed8983b 
Bumps [mypy](https://github.com/python/mypy) from 1.1.1 to 1.2.0.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](python/mypy@v1.1.1...v1.2.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mike Alfare <[email protected]>
@dependabot @mikealfare
Update types-pytz requirement from ~=2023.2 to ~=2023.3 (#391)
be47aba 
Updates the requirements on [types-pytz](https://github.com/python/typeshed) to permit the latest version.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-pytz
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mike Alfare <[email protected]>
@dependabot @mikealfare
Update pytz requirement from ~=2023.2 to ~=2023.3 (#390)
c811a1d 
Updates the requirements on [pytz](https://github.com/stub42/pytz) to permit the latest version.
- [Release notes](https://github.com/stub42/pytz/releases)
- [Commits](stub42/pytz@release_2023.2...release_2023.3)

---
updated-dependencies:
- dependency-name: pytz
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mike Alfare <[email protected]>
@dependabot @mikealfare
Update black requirement from ~=23.1 to ~=23.3 (#389)
e178459 
Updates the requirements on [black](https://github.com/psf/black) to permit the latest version.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@23.1.0...23.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mike Alfare <[email protected]>
@dependabot
Update tox requirement from ~=3.0 to ~=4.5 (#416)
aa98c95 
Updates the requirements on [tox](https://github.com/tox-dev/tox) to permit the latest version.
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](https://github.com/tox-dev/tox/commits/4.5.0)

---
updated-dependencies:
- dependency-name: tox
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@nssalian
Fix for #419: dbt-redshift 1.5 does not work with non-standard redshi…
b644963 
…ft hostnames (#420)

* Fix for region parsing in host name.WIP

* Edge cases and cleanup

* Minor nits

* clean up aws regions invocation

* Log message and doc nits

* Changie entry

* Skip validation if aws regions are not determined

* move the regions var earlier to effectively cache
@davidbloss
update used gh actions ahead of set-output, node12 deprecation (#418)
2c52b30
@nssalian
Fix execute signature based on core (#435)
52a666e
@FishtownBuildBot
Bumping version to 1.6.0b1 and generate changelog
2e97dcb
@FishtownBuildBot
[Automated] Merged prep-release/1.6.0b1_4962130872 into target main d…
adf0353 
…uring release process
@emmyoop
more github deprecations fixes (#440)
b6ea1d6 
* more github deprecations fixes

* swap out abandonded action
@emmyoop
updating set-output (#450)
272a19e 
* updating set-output

* add back run, move comment
@davidbloss @emmyoop
update used gh actions ahead of node12 deprecation (#453)
a406b3a 
Co-authored-by: Emily Rockman <[email protected]>
@VersusFacit @dbeatty10
Add autocommit feature to let dbt users run certain db commands in ma…
d13a376 
…cros (#458)

* Add a fix. Add a test.

* Add changelog

* Fix test profile

* Fix spelling of Redshift

* Update changelog

---------

Co-authored-by: Mila Page <[email protected]>
Co-authored-by: Doug Beatty <[email protected]>
@jiezhen-chen @McKnight-42
Default connect_timeout to None (#433)
c12b625 
* change connect_timeout to defaulted None

* update test_redshift_adapter to pass new changes

* test that connect_timeout parameter is called with the right value

* add new changelog

---------

Co-authored-by: Matthew McKnight <[email protected]>
@mikealfare
Copy link
Contributor Author

Integration tests run against this branch using the workflow on this branch: https://github.com/dbt-labs/dbt-adapters/actions/runs/12873006795

@mikealfare
Copy link
Contributor Author

The failed dbt-athena tests are all the same test, a known issue that is unrelated to dbt-redshift. The failed test in the linked dbt-redshift run is due to a missing role ARN. This was not in the 1pass vault, and amounts to a config update in GH settings.

@mikealfare mikealfare marked this pull request as ready for review January 20, 2025 17:41
@mikealfare mikealfare requested a review from a team as a code owner January 20, 2025 17:41
@github-christophe-oudar
Copy link
Contributor

Is there a typo in the name of this PR? I feel like you meant
ADAP-1131: Migrate dbt-redshift into dbt-adapters instead of ADAP-1131: Migrate dbt-bigquery into dbt-redshift no?

@mikealfare mikealfare changed the title ADAP-1131: Migrate dbt-bigquery into dbt-redshift ADAP-1131: Migrate dbt-redshift into dbt-adapters Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@mikealfare mikealfare

Labels
cla:yes The PR author has signed the CLA
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.