diff --git a/docs/products/kafka/kafka-connect/howto/manage-connector-versions.md b/docs/products/kafka/kafka-connect/howto/manage-connector-versions.md new file mode 100644 index 000000000..f4cbedf67 --- /dev/null +++ b/docs/products/kafka/kafka-connect/howto/manage-connector-versions.md @@ -0,0 +1,308 @@ +--- +title: Manage connector versions in Aiven for Apache Kafka Connect® +sidebar_label: Manage connector versions +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +Multi-version connector support lets you control which connector version is used in your Aiven for Apache Kafka Connect® service. +It prevents compatibility issues from automatic updates, avoids breaking changes +when multiple dependencies rely on a specific connector version, and allows testing, +upgrading, or reverting versions to maintain production pipeline stability. You can +specify the version of a connector by setting the `plugin_versions` property in the +service configuration. + +## Key considerations for multi-version connectors + +- Deprecated connector versions may be removed during maintenance updates. If you + [pin](#pin-version) a deprecated version, the system alerts you and recommends an + upgrade. The connector will continue to run, but upgrading to a supported version is + recommended to avoid compatibility issues. +- Support is limited to the latest connector version and the most recent previous + version. Breaking changes, if any, are detailed in + [maintenance update notifications](/docs/platform/concepts/maintenance-window#maintenance-updates). +- Pinning a connector version applies to the entire plugin, ensuring that all + connectors provided by the plugin (such as source and sink connectors) use the same + version. +- Multi-version support is available for all connectors where Aiven has published and + supports more than one version. Support will continue to expand as new versions are + released. +- If no version is pinned, the latest available version is used. +- Refer to [Check available connector versions](#check-available-connector-versions) to + confirm which versions are supported before pinning a version. + +## Supported connectors and versions + +:::note +All versions listed in the table are supported unless explicitly noted otherwise. +::: + + +| Connector | Versions | +|---------------------|-----------------------| +| Debezium | `2.5.0` (recommended)
`1.9.7` (deprecated) | +| JDBC | `6.10.0` (recommended)
`6.9.0` (deprecated) | +| Snowflake | `2.3.0` (recommended)
`2.2.0` | + + +:::tip +This is a partial list of connectors, and it may change as new versions are released. To +view the most up-to-date versions, see +[Check available connector versions](#check-available-connector-versions). +::: + +:::note +If you are using version `1.9.7` of the Debezium connector, you can upgrade to +version `2.5.0` without raising a support ticket. To upgrade, pin version `2.5.0` in +your configuration. For details, see [Pin a connector version](#pin-version). +::: + +## Prerequisites + +- [Aiven for Apache Kafka® service](/docs/products/kafka/kafka-connect/howto/enable-connect) + with a [dedicated Aiven for Apache Kafka Connect® service](/docs/products/kafka/kafka-connect/get-started#apache_kafka_connect_dedicated_cluster) + enabled +- [Aiven CLI](/docs/tools/cli) +- [Aiven API](/docs/tools/api) +- [Aiven Provider for Terraform](/docs/tools/terraform) + +:::note +Multi-version support is only available for **dedicated Kafka Connect services**. Ensure +that your service is updated to enable this feature. +::: + +## Check available connector versions {#check-available-connector-versions} + +Before pinning a connector version, check which versions are available for your +Apache Kafka Connect service. Verify if a connector supports multi-versioning by +reviewing the listed versions. Use one of the following methods: + + + + + +1. Run the following command: + + ```bash + curl -X GET "https://api.aiven.io/v1/project//service//available-connectors" \ + -H "Authorization: Bearer " + ``` + +1. Review the response to see the available versions. If multiple versions are listed, + the connector supports multi-versioning. + + Example output: + + ```json + { + "plugin_name": "aiven-kafka-connect-jdbc", + "available_versions": [ + { + "version": "6.10.0", + "deprecated": false + }, + { + "version": "6.9.0", + "deprecated": true + } + ] + } + ``` + + + + +1. Run the following command: + + ```bash + avn service connector available + ``` + +1. Review the output to see the available versions. If multiple versions are listed, + the connector supports multi-versioning. + + Example output: + + ```json + { + "plugin_name": "aiven-kafka-connect-jdbc", + "available_versions": [ + { + "version": "6.10.0", + "deprecated": false + }, + { + "version": "6.9.0", + "deprecated": true + } + ] + } + ``` + + + + +In Terraform, you can specify the desired connector version using the `plugin_versions` +attribute in your configuration. This ensures that your Apache Kafka Connect service +uses the exact version you need. + +For detailed steps and examples, see [Pin a connector version](#pin-version). + + + + +## Pin a connector version {#pin-version} + +To use a specific connector version, update the `plugin_versions` property in the +service configuration. Pinning a **plugin version** applies to all connectors +(source and sink) provided by that plugin. For example, pinning +the `aiven-kafka-connect-jdbc` plugin to version `6.9.0` affects both the JDBC source +and sink connectors. + +:::note +Pinning a plugin version restarts Apache Kafka Connect and reloads all plugins. +Allow time for the process to complete. +::: + + + + +1. Run the following command: + + ```bash + curl -X PUT "https://api.aiven.io/v1/project//service/" \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "plugin_versions": { + "": "" + } + }' + ``` + + Parameters: + + - ``: Aiven project name. + - ``: Apache Kafka Connect service name. + - ``: Aiven API token. + - ``: Plugin name. For example, `aiven-kafka-connect-jdbc`. + - ``: Desired version of the plugin. For example, `6.9.0`. + +1. Verify the update in the service configuration: + + ```bash + curl -X GET "https://api.aiven.io/v1/project//service/" \ + -H "Authorization: Bearer " + ``` + + + + +1. Run the following command: + + ```bash + avn service update --project \ + -c 'plugin_versions={"": ""}' + ``` + + Parameters: + + - ``: Apache Kafka Connect service name. + - ``: Aiven project name. + - ``: Plugin name. For example, `aiven-kafka-connect-jdbc`. + - ``: Desired version of the plugin. For example, `6.9.0`. + +1. Verify the update: + + ```bash + avn service get + ``` + + + + +1. Update your Terraform configuration: + + ```hcl + resource "aiven_service" "kafka_connect" { + service_name = "" + project = "" + + user_config { + plugin_versions = { + "" = "" + } + } + } + ``` + + Parameters: + + - ``: Apache Kafka Connect service name. + - ``: Aiven project name. + - ``: Plugin name. For example, `aiven-kafka-connect-jdbc`. + - ``: Desired version of the plugin. For example, `6.9.0`. + +1. Apply the changes: + + ```bash + terraform apply + ``` + +1. Confirm the updated configuration: + + ```bash + terraform show + ``` + +For detailed information, refer to the +[Terraform Registry - `plugin_versions`](https://registry.terraform.io/providers/aiven/aiven/latest/docs/resources/kafka_connect#plugin_versions-1). + + + + +## Verify the connector version + +After pinning a version, confirm that the correct version is in use. + + + + +1. Run the following command: + + ```bash + curl -X GET "https://api.aiven.io/v1/project//service/" \ + -H "Authorization: Bearer " + + ``` + +1. Review the `plugin_versions` property in the response to verify the pinned version. + + + + +1. Run the following command: + + ```bash + avn service get + ``` + +1. Check the `plugin_versions` property in the output to confirm the version. + + + + +Verify the version in your Terraform state or configuration after applying changes: + +1. Run: + + ```bash + terraform show + ``` + +1. Locate the `plugin_versions` property under the Kafka Connect service resource. + It displays the pinned version of the connector. + + + diff --git a/sidebars.ts b/sidebars.ts index 28bd12534..1cd800dc5 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -922,6 +922,7 @@ const sidebars: SidebarsConfig = { 'products/kafka/kafka-connect/howto/bring-your-own-kafka-connect-cluster', 'products/kafka/kafka-connect/howto/enable-connect', 'products/kafka/kafka-connect/howto/enable-automatic-restart', + 'products/kafka/kafka-connect/howto/manage-connector-versions', 'products/kafka/kafka-connect/howto/manage-logging-level', 'products/kafka/kafka-connect/howto/request-new-connector', { diff --git a/static/includes/debezium-breakingchange.md b/static/includes/debezium-breakingchange.md index 27ad4a396..dd516efef 100644 --- a/static/includes/debezium-breakingchange.md +++ b/static/includes/debezium-breakingchange.md @@ -1,9 +1,14 @@ :::note **Breaking changes in Debezium 2.5** -Debezium version 2.5 introduces significant changes to the connector's configuration -and behavior. New setups [default to version 2.5](https://debezium.io/releases/2.5/release-notes), +Debezium version 2.5 introduces changes to the connector's configuration and behavior. +New setups [default to version 2.5](https://debezium.io/releases/2.5/release-notes), while existing setups using version 1.9 remain on that version for stability. -Test configurations with version 2.5 before upgrading. -For help with upgrades or queries, contact [Aiven support](mailto:support@aiven.io). + +If you are using version 1.9.7, you can upgrade to version 2.5.0 using the +[multi-version support](/docs/products/kafka/kafka-connect/howto/manage-connector-versions) +without raising a support ticket. Ensure compatibility by testing version 2.5 +configurations before upgrading. + +For further assistance, contact [Aiven support](mailto:support@aiven.io). :::