Skip to content

Commit

Permalink
Merge branch 'main' into 1677-fix-do-not-use-start-date-with-non-date…
Browse files Browse the repository at this point in the history
…-columns
  • Loading branch information
edgarrmondragon authored Jul 30, 2024
2 parents 0df7421 + cb163bf commit 86ce7a2
Show file tree
Hide file tree
Showing 135 changed files with 3,497 additions and 1,938 deletions.
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[flake8]
max-line-length = 88
exclude = cookiecutter
ignore = E, W
ignore = E, F, W
per-file-ignores =
# Don't require docstrings conventions in private modules
singer_sdk/helpers/_*.py:DAR
Expand Down
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# nested in that directory, on any level

# Default owners
* @edgarrmondragon @kgpayne
* @edgarrmondragon

# CI/CD
/.github/workflows/ @edgarrmondragon @meltano/engineering
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ body:
attributes:
label: Singer SDK Version
description: Version of the library you are using
placeholder: "0.37.0"
placeholder: "0.39.0"
validations:
required: true
- type: checkboxes
Expand Down
22 changes: 17 additions & 5 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,35 @@ updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: daily
interval: weekly
time: "12:00"
reviewers: [meltano/engineering]
labels: [deps]
labels: [Dependencies]
groups:
development-dependencies:
dependency-type: development
runtime-dependencies:
dependency-type: production
update-types:
- "minor"
- "patch"
- package-ecosystem: pip
directory: "/.github/workflows"
schedule:
interval: daily
interval: weekly
time: "12:00"
reviewers: [meltano/engineering]
labels: [deps]
labels: [Dependencies]
groups:
ci:
patterns:
- "*"
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly
reviewers: [meltano/engineering]
labels: [deps]
labels: [Dependencies]
groups:
actions:
patterns:
Expand Down
1 change: 1 addition & 0 deletions .github/semantic.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ types:
- docs
- feat
- fix
- packaging
- perf
- refactor
- revert
Expand Down
53 changes: 53 additions & 0 deletions .github/workflows/api-changes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: API Changes

on:
pull_request:
paths:
- singer_sdk/**
- .github/workflows/api-changes.yml
- CHANGELOG.md
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

permissions: # added using https://github.com/step-security/secure-repo
contents: read

jobs:
check-api-changes:
name: Check API Changes
runs-on: ubuntu-latest
env:
NOXSESSION: api
steps:
- name: Check out the repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.12

- name: Install tools
env:
PIP_CONSTRAINT: ${{ github.workspace }}/.github/workflows/constraints.txt
run: |
python -Im pip install -U pip
pipx install griffe nox
pipx inject nox nox-poetry
pipx list
- name: Set REF
id: set-ref
if: always() && !startsWith(github.head_ref, 'release/')
run: |
echo "ref=${{ github.event.pull_request.base.sha }}" >> $GITHUB_OUTPUT
# Check API against the latest commit on the base branch
- name: Run Nox
run: |
nox -- ${{ steps.set-ref.outputs.ref }}
2 changes: 1 addition & 1 deletion .github/workflows/codspeed.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
--with benchmark
--all-extras
- uses: CodSpeedHQ/action@v2
- uses: CodSpeedHQ/action@v3
with:
token: ${{ secrets.CODSPEED_TOKEN }}
run: pytest tests/ --codspeed
11 changes: 6 additions & 5 deletions .github/workflows/constraints.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
pip==24.0
poetry==1.8.2
poetry-plugin-export==1.7.1
poetry-dynamic-versioning==1.3.0
pre-commit==3.7.0
griffe==0.48.0
pip==24.2
poetry==1.8.3
poetry-plugin-export==1.8.0
poetry-dynamic-versioning==1.4.0
pre-commit==3.8.0
nox==2024.4.15
nox-poetry==1.0.3
43 changes: 33 additions & 10 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@ name: Release
on:
push:

permissions:
contents: write # Needed to upload artifacts to the release
id-token: write # Needed for OIDC PyPI publishing

jobs:
build:
runs-on: ubuntu-latest
Expand All @@ -24,19 +20,46 @@ jobs:
name: publishing
url: https://pypi.org/p/singer-sdk
if: startsWith(github.ref, 'refs/tags/')
permissions:
id-token: write # Needed for OIDC PyPI publishing
steps:
- uses: actions/download-artifact@v4
with:
name: Packages
path: dist
- name: Upload wheel to release
- name: Publish
uses: pypa/[email protected]

upload-to-release:
name: Upload files to release
runs-on: ubuntu-latest
needs: [build]
if: startsWith(github.ref, 'refs/tags/')
permissions:
contents: write # Needed for uploading files to the release
id-token: write # Needed for attestations
attestations: write # Needed for attestations

steps:
- uses: actions/download-artifact@v4
with:
name: Packages
path: dist
- name: Upload wheel and sdist to release
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: dist/*.whl
file: dist/singer_sdk*
tag: ${{ github.ref }}
overwrite: true
file_glob: true

- name: Publish
uses: pypa/[email protected]
- uses: actions/attest-build-provenance@v1
id: attest
with:
subject-path: "./dist/singer_sdk*"
- name: Upload attestations to release
uses: svenstaro/upload-release-action@v2
with:
file: ${{ steps.attest.outputs.bundle-path }}
tag: ${{ github.ref }}
overwrite: true
asset_name: attestations.intoto.jsonl
9 changes: 2 additions & 7 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,14 @@ jobs:
fail-fast: false
matrix:
session: [tests]
os: ["ubuntu-latest", "macos-13", "windows-latest"]
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
sqlalchemy: ["2"]
include:
- { session: tests, python-version: "3.12", os: "ubuntu-latest", sqlalchemy: "1" }
- { session: tests, python-version: "3.12", os: "macos-latest", sqlalchemy: "2" }
- { session: doctest, python-version: "3.12", os: "ubuntu-latest", sqlalchemy: "2" }
- { session: mypy, python-version: "3.12", os: "ubuntu-latest", sqlalchemy: "2" }
- { session: deps, python-version: "3.12", os: "ubuntu-latest", sqlalchemy: "2" }

steps:
- uses: actions/checkout@v4
Expand All @@ -74,7 +74,6 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
architecture: x64

- name: Upgrade pip
env:
Expand Down Expand Up @@ -114,9 +113,6 @@ jobs:
SAMPLE_TAP_GITLAB_GROUP_IDS: ${{ secrets.SAMPLE_TAP_GITLAB_GROUP_IDS }}
SAMPLE_TAP_GITLAB_PROJECT_IDS: ${{ secrets.SAMPLE_TAP_GITLAB_PROJECT_IDS }}
SAMPLE_TAP_GITLAB_START_DATE: "2022-01-01T00:00:00Z"
SAMPLE_TAP_GOOGLE_ANALYTICS_CLIENT_EMAIL: ${{ secrets.SAMPLE_TAP_GOOGLE_ANALYTICS_CLIENT_EMAIL }}
SAMPLE_TAP_GOOGLE_ANALYTICS_PRIVATE_KEY: ${{ secrets.SAMPLE_TAP_GOOGLE_ANALYTICS_PRIVATE_KEY }}
SAMPLE_TAP_GOOGLE_ANALYTICS_VIEW_ID: ${{ secrets.SAMPLE_TAP_GOOGLE_ANALYTICS_VIEW_ID }}

steps:
- uses: actions/checkout@v4
Expand All @@ -136,7 +132,6 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: ${{ env.NOXPYTHON }}
architecture: x64

- name: Upgrade pip
env:
Expand Down
7 changes: 4 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ repos:
cookiecutter/.*/meltano.yml|
cookiecutter/.*/.pre-commit-config.yaml|
cookiecutter/.*/dependabot.yml|
cookiecutter/.*/build.yml|
cookiecutter/.*/test.yml
)$
- id: end-of-file-fixer
Expand All @@ -42,14 +43,14 @@ repos:
)$
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.28.2
rev: 0.29.1
hooks:
- id: check-dependabot
- id: check-github-workflows
- id: check-readthedocs

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.4.3
rev: v0.5.5
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix, --show-fixes]
Expand All @@ -64,7 +65,7 @@ repos:
)$
- repo: https://github.com/pycqa/flake8
rev: 7.0.0
rev: 7.1.0
hooks:
- id: flake8
additional_dependencies:
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ version: 2
build:
os: ubuntu-22.04
tools:
python: "3.11"
python: "3.12"

sphinx:
builder: html
Expand Down
56 changes: 56 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,62 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## v0.39.0 (2024-07-27)

### ✨ New

- [#2432](https://github.com/meltano/sdk/issues/2432) Developers can now customize the default logging configuration for their taps/targets by adding `default_logging.yml` to their package
- [#2531](https://github.com/meltano/sdk/issues/2531) The `json` module is now avaiable to stream maps -- _**Thanks @grigi!**_
- [#2529](https://github.com/meltano/sdk/issues/2529) Stream sync context is now available to all instances methods as a `Stream.context` attribute

### 🐛 Fixes

- [#2554](https://github.com/meltano/sdk/issues/2554) Use mapped stream aliases when handling `ACTIVATE_VERSION` messages in the base target class
- [#2526](https://github.com/meltano/sdk/issues/2526) Moved up the supported Python versions in the Markdown output of `--about`

### ⚙️ Under the Hood

- [#2564](https://github.com/meltano/sdk/issues/2564) Make `SQLSink` a generic with a `SQLConnector` type parameter
- [#2540](https://github.com/meltano/sdk/issues/2540) Implement abstract `serialize_message` for Singer writers
- [#2259](https://github.com/meltano/sdk/issues/2259) Centralize JSON SerDe into helper functions -- _**Thanks @BuzzCutNorman!**_
- [#2525](https://github.com/meltano/sdk/issues/2525) Make `PyJWT` and `cryptography` dependencies optional
- [#2528](https://github.com/meltano/sdk/issues/2528) Moved class-level attributes to the top in REST tap template
- [#2132](https://github.com/meltano/sdk/issues/2132) Limit internal usage of pendulum

### 📚 Documentation Improvements

- [#2557](https://github.com/meltano/sdk/issues/2557) Document that `get_starting_timestamp` requires setting a non-null replication_key
- [#2556](https://github.com/meltano/sdk/issues/2556) Reference state partitioning in stream partitioning page
- [#2536](https://github.com/meltano/sdk/issues/2536) Prepare for RTD addons migration
- [#2535](https://github.com/meltano/sdk/issues/2535) Added more intersphinx links to Python and Faker docs
- [#2530](https://github.com/meltano/sdk/issues/2530) Explained how the request URL is generated from `url_base`, `path` and the sync context
- [#2527](https://github.com/meltano/sdk/issues/2527) Updated the footer
- [#2506](https://github.com/meltano/sdk/issues/2506) Fixed a typo in the stream maps docs

## v0.38.0 (2024-06-17)

### ✨ New

- [#2433](https://github.com/meltano/sdk/issues/2433) Tap developers can now disable HTTP redirects
- [#2426](https://github.com/meltano/sdk/issues/2426) Added an optional GitHub workflow to publish to PyPI with trusted publishers

### 🐛 Fixes

- [#2438](https://github.com/meltano/sdk/issues/2438) Null replication values are now handled when incrementing bookmarks
- [#2431](https://github.com/meltano/sdk/issues/2431) Updated cookiecutter VSCode `launch.json` to use `debugpy`
- [#2421](https://github.com/meltano/sdk/issues/2421) An error message is now logged every time schema validation fails for any record

### ⚙️ Under the Hood

- [#2455](https://github.com/meltano/sdk/issues/2455) Use parent `datetime.datetime` class in type conforming checks
- [#2453](https://github.com/meltano/sdk/issues/2453) Change to return type of `utc_now` from `pendulum.DateTime` to `datetime.datetime`

### 📚 Documentation Improvements

- [#2449](https://github.com/meltano/sdk/issues/2449) Add a short guide on defining a configuration schema
- [#2436](https://github.com/meltano/sdk/issues/2436) Documented how context fields are passed to a child stream
- [#2435](https://github.com/meltano/sdk/issues/2435) Using an empty list for `__key_properties__` to disable a stream primary keys is now recommended as an alternative to `null`

## v0.37.0 (2024-04-29)

### ✨ New
Expand Down
Loading

0 comments on commit 86ce7a2

Please sign in to comment.