Skip to content

Commit

Permalink
Upgrade AWS SDK Go to v2
Browse files Browse the repository at this point in the history
Although aws-sdk-go-v2 was released long ago, historically, until now,
tfmigrate has used v1 as well as the old Terraform.

aws-sdk-go and aws-sdk-go-v2 have different priorities for loading
credentials. Specifically, aws-sdk-go v1 prioritizes reading environment
variables over profiles. This is a problem when switching between
multiple accounts in a profile while OIDC authenticating to AWS from
GitHub Actions.

Up to Terraform v1.5, aws-sdk-go v1 was used for S3 backend
authentication via hashicorp/aws-sdk-go-base, but aws-sdk-go-v2 is used
for Terraform / OpenTofu v1.6 and later.

Up to tfmigrate v0.3, we have been using aws-sdk-go v1 via
hashicorp/aws-sdk-go-base for history s3 storage authentication but will
be using aws-sdk-go-v2 after the next tfmigrate v0.4. This is a breaking
change, but the goal is to align with the behavior of Terraform /
OpenTofu v1.6 and later, so if you are affected, please adjust your AWS
authentication settings.

At the time of this writing, v2 of hashicorp/aws-sdk-go-base, which uses
aws-sdk-go-v2, is still beta, but it is already used in the stable
releases of Terraform and OpenTofu. Since there are multiple beta
versions of hashicorp/aws-sdk-go-base and slightly different versions in
use, the results of my investigation are shown below.

- tfmigrate: v0.3.24: v1.1.0
- terraform v1.5.7: v0.7.1
- terraform v1.6.0: v2.0.0-beta.35
- terraform v1.7.0: v2.0.0-beta.43
- terraform v1.8.0: v2.0.0-beta.45
- terraform v1.9.8 (latest): v2.0.0-beta.45
- opentofu v1.6.0: v2.0.0-beta.43
- opentofu v1.8.5 (latest): v2.0.0-beta.43
- terraform-provider-aws v3.76.1: v1.0.0
- terraform-provider-aws v4.0.0: v2.0.0-beta.5
- terraform-provider-aws v5.0.0: v2.0.0-beta.27
- terraform-provider-aws v5.29.0: v2.0.0-beta.43
- terraform-provider-aws v5.31.0: v2.0.0-beta.45
- terraform-provider-aws v5.74.0 (latest): v2.0.0-beta.59

As a special note, use_legacy_workflow has been removed in Terraform
v1.8.0 and OpenTofu v1.8.0, but UseLegacyWorkflow has been removed in
aws-sdk-go-base in v2.0.0-beta.49.

Considering the above, I think v2.0.0-beta.43 or v2.0.0-beta.45 would be
reasonable. However, at this point, I don't see any solid reason for
choosing v2.0.0-beta.45, so I will select v2.0.0-beta.43 to align with
OpenTofu.

I rewrote the s3 implementation for history storage, referring to the
aws-sdk-go-v2 migration guide.

https://aws.github.io/aws-sdk-go-v2/docs/migrating/

From the tfmigrate user's perspective, there are no breaking changes at
the configuration file level. Still, it should be noted that AWS
credentials have higher precedence in profiles than in environment
variables.

Also, as a notable side effect of the upgrade to v2, the path to the
file configuring the AWS profile is ~/.aws/config by default, but it can
now be specified in the environment variable AWS_CONFIG_FILE. This will
be useful for customizing the configuration in a CI/CD environment.
  • Loading branch information
minamijoyo committed Nov 7, 2024
1 parent 3f4250a commit 1d9da6a
Show file tree
Hide file tree
Showing 7 changed files with 211 additions and 97 deletions.
41 changes: 33 additions & 8 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,15 @@ go 1.22

require (
cloud.google.com/go/storage v1.36.0
github.com/aws/aws-sdk-go v1.55.5
github.com/aws/aws-sdk-go-v2 v1.32.3
github.com/aws/aws-sdk-go-v2/config v1.28.1
github.com/aws/aws-sdk-go-v2/credentials v1.17.42
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.18
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.35
github.com/aws/aws-sdk-go-v2/service/s3 v1.66.2
github.com/davecgh/go-spew v1.1.1
github.com/google/go-cmp v0.6.0
github.com/hashicorp/aws-sdk-go-base v1.1.0
github.com/hashicorp/aws-sdk-go-base/v2 v2.0.0-beta.43
github.com/hashicorp/go-version v1.3.0
github.com/hashicorp/hcl/v2 v2.6.0
github.com/hashicorp/logutils v1.0.0
Expand All @@ -26,8 +31,25 @@ require (
github.com/apparentlymart/go-textseg v1.0.0 // indirect
github.com/apparentlymart/go-textseg/v12 v12.0.0 // indirect
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.6 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.22 // indirect
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.25.5 // indirect
github.com/aws/aws-sdk-go-v2/service/iam v1.27.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.8.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sqs v1.28.4 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.3 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.32.3 // indirect
github.com/aws/smithy-go v1.22.0 // indirect
github.com/bgentry/speakeasy v0.1.0 // indirect
github.com/fatih/color v1.7.0 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
Expand All @@ -37,22 +59,25 @@ require (
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/hashicorp/errwrap v1.0.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.0 // indirect
github.com/hashicorp/go-multierror v1.0.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-hclog v1.5.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/terraform-plugin-log v0.9.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/mattn/go-colorable v0.0.9 // indirect
github.com/mattn/go-isatty v0.0.3 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.17 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-wordwrap v0.0.0-20150314170334-ad45545899c7 // indirect
github.com/posener/complete v1.1.1 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws v0.46.1 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.47.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.47.0 // indirect
go.opentelemetry.io/otel v1.22.0 // indirect
go.opentelemetry.io/otel/metric v1.22.0 // indirect
go.opentelemetry.io/otel/trace v1.22.0 // indirect
golang.org/x/crypto v0.22.0 // indirect
golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
golang.org/x/net v0.24.0 // indirect
golang.org/x/oauth2 v0.17.0 // indirect
golang.org/x/sync v0.6.0 // indirect
Expand Down
Loading

0 comments on commit 1d9da6a

Please sign in to comment.