Skip to content

Commit

Permalink
chore: commit for v2.6.1
Browse files Browse the repository at this point in the history
  • Loading branch information
YikaiHu committed Apr 11, 2024
1 parent 8676478 commit f39376f
Show file tree
Hide file tree
Showing 31 changed files with 125 additions and 66 deletions.
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [2.6.1] - 2024-04-11

### Fixed

- Fix the ECR transfer task creation issue #147
- Fix the S3 transfer task creation issue #141
- Fix the Gov Cloud S3 transfer task assets issue #140
- Fix the error s3 plugin address issue #145

## [2.6.0] - 2024-01-18

### Added
Expand Down
5 changes: 1 addition & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,7 @@ Create your first data transfer task, For the complete user guide, refer to
## FAQ

**Q. Which are the supported Regions of this solution?**</br>
You can deploy this solution in these Regions: N.Virginia (us-east-1), Ohio (us-east-2), N.California (us-west-1),
Oregon (us-west-2), Mumbai (ap-south-1), Seoul (ap-northeast-2), Singapore (ap-southeast-1), Sydney (ap-southeast-2),
Tokyo (ap-northeast-1), Canada (ca-central-1), Frankfurt (eu-central-1), Ireland (eu-west-1), London (eu-west-2),
Paris (eu-west-3), Stockholm (eu-north-1), São Paulo (sa-east-1), Beijing (cn-north-1), Ningxia (cn-northwest-1).
Please refer to [Supported Regions](https://awslabs.github.io/data-transfer-hub/en/plan-deployment/regions/).

**Q. If I want to copy data from U.S. Regions to China Regions, where should I deploy the solution?**</br>
You can either deploy the solution in U.S. Regions or deploy the solution in China Regions.
Expand Down
4 changes: 4 additions & 0 deletions docs/ECR_DEPLOYMENT_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@

[![Launch Stack](./images/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=DTHECRStack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferECRStack.template)

- 部署到 AWS GovCloud (US) 区

[![Launch Stack](../images/launch-stack.svg)](https://console.amazonaws-us-gov.com/cloudformation/home#/stacks/create/template?stackName=DTHECRStack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferECRStack.template)


1.单击**下一步**。 相应地为参数指定值。 如果需要,请更改堆栈名称。

Expand Down
6 changes: 5 additions & 1 deletion docs/ECR_DEPLOYMENT_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,11 @@ Please follow below steps to deploy this plugin via AWS Cloudformation.
- For AWS Global regions

[![Launch Stack](./images/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=DTHECRStack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferECRStack.template)


- For AWS GovCloud (US) Regions

[![Launch Stack](../images/launch-stack.svg)](https://console.amazonaws-us-gov.com/cloudformation/home#/stacks/create/template?stackName=DTHECRStack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferECRStack.template)

1. Click **Next**. Specify values to parameters accordingly. Change the stack name if required.

1. Click **Next**. Configure additional stack options such as tags (Optional).
Expand Down
3 changes: 3 additions & 0 deletions docs/S3_DEPLOYMENT_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@

[![Launch Stack](./images/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=DTHS3Stack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template)

- 部署到 AWS GovCloud (US) 区

[![Launch Stack](./images/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=DTHS3Stack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template)


1. 单击**下一步**。 相应地为参数指定值。 如果需要,请更改堆栈名称。
Expand Down
4 changes: 4 additions & 0 deletions docs/S3_DEPLOYMENT_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ Please follow below steps to deploy this solution via AWS Cloudformation.

[![Launch Stack](./images/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=DTHS3Stack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template)

- For AWS GovCloud (US) Regions

[![Launch Stack](../images/launch-stack.svg)](https://console.amazonaws-us-gov.com/cloudformation/home#/stacks/create/template?stackName=DTHS3Stack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template)

1. Click **Next**. Specify values to parameters accordingly. Change the stack name if required.

1. Click **Next**. Configure additional stack options such as tags (Optional).
Expand Down
6 changes: 3 additions & 3 deletions docs/en-base/architecture-overview/architecture-details.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ This solution provides a simple web console which allows you to create and manag
### Amazon S3 transfer engine
Amazon S3 transfer engine runs the Amazon S3 plugin and is used for transferring objects from their sources into S3 buckets. The S3 plugin supports the following features:

- Transfer Amazon S3 objects between AWS China Regions and AWS Regions
- Transfer objects from Alibaba Cloud OSS / Tencent COS / Qiniu Kodo to Amazon S3
- Transfer Amazon S3 objects between AWS China Regions and AWS Global Regions
- Transfer objects from other cloud providers to Amazon S3
- Transfer objects from S3 Compatible Storage service to Amazon S3
- Support near real time transfer via S3 Event
- Support transfer with object metadata
Expand All @@ -46,7 +46,7 @@ Amazon S3 transfer engine runs the Amazon S3 plugin and is used for transferring

Amazon ECR engine runs the Amazon ECR plugin and is used for transferring container images from other container registries. The ECR plugin supports the following features:

- Transfer Amazon ECR images between AWS China Regions and AWS Regions
- Transfer Amazon ECR images between AWS China Regions and AWS Global Regions
- Transfer from public container registry (such as Docker Hub, GCR.io, Quay.io) to Amazon ECR
- Transfer selected images to Amazon ECR
- Transfer all images and tags from Amazon ECR
Expand Down
4 changes: 3 additions & 1 deletion docs/en-base/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
The Data Transfer Hub solution provides secure, scalable, and trackable data transfer for Amazon Simple Storage Service (Amazon S3) objects and Amazon Elastic Container Registry (Amazon ECR) images. This data transfer helps customers easily create and manage different types (Amazon S3 object and Amazon ECR image) of transfer tasks between AWS [partitions](https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/partitions.html) (for example, aws, aws-cn, aws-us-gov), and from other cloud providers to AWS.
The Data Transfer Hub solution provides secure, scalable, and trackable data transfer for Amazon Simple Storage Service (Amazon S3) objects and Amazon Elastic Container Registry (Amazon ECR) images. This data transfer helps customers easily create and manage different types (Amazon S3 object and Amazon ECR image) of transfer tasks between AWS [partitions](https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/partitions.html) (for example, aws, aws-cn, aws-us-gov), and from other cloud providers to AWS at your own discretion.

If you have enabled the Direct Connect service in a specific AWS Region and a specific AWS China Region and have purchased a compliant cross-border dedicated line provided by a qualified operator to connect the AWS Region and their own VPC in the AWS China Region, you can use Data Transfer Hub's console to create a data transfer task and choose to use a dedicated line for data transfer at your own discretion.

This implementation guide provides an overview of the Data Transfer Hub solution, its reference architecture and components, considerations for planning the deployment, configuration steps for deploying the Data Transfer Hub solution to the AWS Cloud.

Expand Down
4 changes: 3 additions & 1 deletion docs/en-base/solution-overview/features-and-benefits.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
The solution supports the following key features:
Depending on the availability of your network environment, the solution supports the following key features:

- **Inter-Partition and Cross-Cloud Data Transfer**: to promote seamless transfer capabilities in one place
- **Auto scaling**: to allow rapid response to changes in file transfer traffic
Expand All @@ -11,5 +11,7 @@ The solution supports the following key features:

If you need to transfer Amazon S3 objects between AWS Regions, we recommend that you use [Cross-Region Replication][crr]. If you want to transfer Amazon S3 objects within the same AWS Region, we recommend using [Same-Region Replication][srr].

For data transfer between AWS China Region and AWS Global Region, you will be responsible for your compliance with all applicable laws and regulations on cross-border data transfer (including purchasing compliant cross-border dedicated lines provided by qualified operators for data transfer, performing necessary government approval or filing), and shall initiate data transfer at your own discretion. AWS does not assist you with this data transfer.

[crr]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication.html#crr-scenario
[srr]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication.html#srr-scenario
10 changes: 5 additions & 5 deletions docs/en-base/solution-overview/use-cases.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
Today, the China market is one of biggest markets in the world. Many international companies are seeking their success in China, as well as a number of Chinese companies are expanding their businesses globally. One of most important steps of in the business is moving their data.

S3 Cross-Region Replication and ECR Cross-Region Replication are popular but customers cannot use them to replicate data into China Regions. With the launch of Data Transfer Hub solution, customers can now create S3 and ECR data transfer tasks between AWS Regions and AWS China Regions in a web portal. Moreover, it supports replicating data from cloud providers to AWS.
S3 Cross-Region Replication and ECR Cross-Region Replication are popular but customers cannot use them to replicate data into China Regions. With the launch of Data Transfer Hub solution, customers can now create S3 and ECR data transfer tasks between AWS Global Regions and AWS China Regions in a web portal. Moreover, it supports replicating data from other cloud providers to AWS.

Data Transfer Hub supports the following use cases:
Depending on the availability of your network environment, Data Transfer Hub supports the following use cases:

* Copy Amazon S3 objects between AWS Regions and AWS China Regions.
* Copy data from other cloud providers’ object storage services (including Alibaba Cloud OSS, Tencent COS, Qiniu Kodo) to Amazon S3.
* Transfer Amazon ECR images between AWS Regions and AWS China Regions.
* Copy Amazon S3 objects between AWS Global Regions and AWS China Regions.
* Copy data from other cloud providers’ object storage services to Amazon S3.
* Transfer Amazon ECR images between AWS Global Regions and AWS China Regions.
* Transfer Dockers image from public docker registry (for example, Docker Hub, Google gcr.io, Red Hat Quay.io) to Amazon ECR.
14 changes: 12 additions & 2 deletions docs/en-base/user-guide/tutorial-ecr.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
The solution allows you to create an Amazon ECR transfer task in the following ways:

- [using the web console](#using-the-web-console)
- [using the ECR plugin](#using-the-dth-ecr-plugin)
- [using the ECR plugin](#using-the-ecr-plugin)
- [using AWS CLI](#using-aws-cli)

For a comparison between those options, refer to [Create Amazon S3 transfer task](./tutorial-s3.md).
You can make your choice according to your needs.

- The web console provides an intuitive user interface where you can start, clone or stop a data transfer task with a simple click. The frontend also provides metric monitoring and logging view, so you do not need to switch between different pages.

- The ECR plugin is a standalone CloudFormation template, and you can easily integrate it into your workflows. Because this option allows deployment without the frontend, it is useful if you want to deploy in AWS China Regions but do not have an ICP licensed domain.

- AWS CLI can quickly initiate data transfer tasks. Select this option if you want to leverage Data Transfer Hub in your automation scripts.

## Using the web console
You can use the web console to create an Amazon ECR transfer task. For more information about how to launch the web console, see [deployment](../../deployment/deployment-overview).
Expand Down Expand Up @@ -101,6 +107,10 @@ Please follow below steps to deploy this plugin via AWS Cloudformation.
- For AWS Global Regions

[![Launch Stack](../images/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=DTHECRStack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferECRStack.template)

- For AWS GovCloud (US) Regions

[![Launch Stack](../images/launch-stack.svg)](https://console.amazonaws-us-gov.com/cloudformation/home#/stacks/create/template?stackName=DTHECRStack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferECRStack.template)

1. Click **Next**. Specify values to parameters accordingly. Change the stack name if required.

Expand Down
6 changes: 5 additions & 1 deletion docs/en-base/user-guide/tutorial-s3.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
The solution allows you to create an Amazon S3 transfer task in the following ways:

- [using the web console](#using-the-web-console)
- [using the S3 plugin](#using-the-dth-s3-plugin)
- [using the S3 plugin](#using-the-s3-plugin)
- [using AWS CLI](#using-aws-cli)

You can make your choice according to your needs.
Expand Down Expand Up @@ -164,6 +164,10 @@ Please follow below steps to deploy this solution via AWS Cloudformation.
- For AWS Global Regions
[![Launch Stack](../images/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=DTHS3Stack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template)
- For AWS GovCloud (US) Regions
[![Launch Stack](../images/launch-stack.svg)](https://console.amazonaws-us-gov.com/cloudformation/home#/stacks/create/template?stackName=DTHS3Stack&templateURL=https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template)
1. Click **Next**. Specify values to parameters accordingly. Change the stack name if required.
Expand Down
6 changes: 3 additions & 3 deletions docs/zh-base/architecture-overview/architecture-details.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@

Amazon S3传输引擎运行Amazon S3插件,用于将对象从其来源传输到S3存储桶中。S3插件支持以下功能:

- 在AWS中国区域和AWS区域之间传输Amazon S3对象
- 从阿里云OSS/Tencent COS/Qiniu Kodo传输对象到Amazon S3
- 在AWS中国区域和其它AWS区域之间传输Amazon S3对象
- 从其它云提供商传输对象到Amazon S3
- 从S3兼容存储服务传输对象到Amazon S3
- 支持通过S3事件进行几乎实时的传输
- 支持传输对象元数据
Expand All @@ -47,7 +47,7 @@ Amazon S3传输引擎运行Amazon S3插件,用于将对象从其来源传输

Amazon ECR引擎运行Amazon ECR插件,用于从其他容器注册表传输容器镜像。ECR插件支持以下功能:

- 在AWS中国区域和AWS区域之间传输Amazon ECR镜像
- 在AWS中国区域和其它AWS区域之间传输Amazon ECR镜像
- 从公共容器注册表(如Docker Hub、GCR.io、Quay.io)传输到Amazon ECR
- 传输选定的镜像到Amazon ECR
- 从Amazon ECR传输所有镜像和标签
Expand Down
4 changes: 3 additions & 1 deletion docs/zh-base/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
**数据传输解决方案**(Data Transfer Hub)为 Amazon Simple Storage Service(Amazon S3)对象和 Amazon Elastic Container Registry(Amazon ECR)镜像提供安全、可扩展和可跟踪的数据传输。通过轻松地在亚马逊云科技(Amazon Web Services,AWS)不同[分区](https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/partitions.html)之间(例如,aws, aws-cn, aws-us-gov)或者从其它云服务提供商到 AWS 创建并管理多种类型的传输任务,该数据传输帮助客户拓展全球业务。
**数据传输解决方案**(Data Transfer Hub)为 Amazon Simple Storage Service(Amazon S3)对象和 Amazon Elastic Container Registry(Amazon ECR)镜像提供安全、可扩展和可跟踪的数据传输。该方案通过自主并轻松地在亚马逊云科技(Amazon Web Services,AWS)不同[分区](https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/partitions.html)之间(例如,aws, aws-cn, aws-us-gov)或者从其它云服务提供商到 AWS 创建并管理多种类型的传输任务,从而帮助客户拓展全球业务。

对于已经在特定海外区域和特定中国区域均开启了 Direct Connect 服务并已经采购了有资质的运营商提供的合规跨境专线连接该海外区域和该中国区域自用 VPC 的客户,您可以自主通过 Data Transfer Hub 的控制台创建数据传输任务,并自主选择使用专线进行传输。

本实施指南提供了数据传输解决方案的概述、参考架构和组件、规划部署的注意事项以及将数据传输解决方案部署到 AWS 云的配置步骤。

Expand Down
3 changes: 3 additions & 0 deletions docs/zh-base/solution-overview/features-and-benefits.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,13 @@
- 监控:跟踪数据流、诊断问题并确保数据传输过程的整体健康状况
- 开箱即用的部署

以上功能均受限于客户网络环境的可用性。

!!! note "注意"

如果您需要在AWS区域之间传输Amazon S3对象,我们建议使用[跨区域复制][crr]。如果您想在同一AWS区域内传输Amazon S3对象,我们建议使用[同区域复制][srr]。

对于 AWS 中国区域和海外区域之间的数据传输,您应遵守适用于您的关于数据跨境传输的所有法律法规(包括采购有资质的运营商提供的合规跨境专线进行数据传输、履行必要的政府审批或备案),并自主启动数据的传输。AWS 并不协助您进行该等数据传输。

[crr]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication.html#crr-scenario
[srr]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication.html#srr-scenario
10 changes: 5 additions & 5 deletions docs/zh-base/solution-overview/use-cases.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
当今的中国市场是世界上最大的市场之一。许多国际公司正在寻求在中国取得成功,同样,一些中国公司也在全球范围内扩展业务。在业务中有很重要的一步是对数据的移动。

虽然 S3 跨区域复制和 ECR 跨区域复制非常流行,但客户无法使用它们将数据复制到中国区域。随着 Data Transfer Hub 解决方案的推出,客户现在可以在 Web 门户中创建 AWS 区域和 AWS 中国区域之间的 S3 和 ECR 数据传输任务。此外,它还支持将数据从云提供商复制到 AWS。
虽然 S3 跨区域复制和 ECR 跨区域复制非常流行,但客户无法使用它们将数据复制到中国区域。随着 Data Transfer Hub 解决方案的推出,客户现在可以在 Web 门户中创建 AWS 中国区域和其它 AWS 区域之间的 S3 和 ECR 数据传输任务。此外,它还支持将数据从其它云服务提供商复制到 AWS。

Data Transfer Hub 支持以下用例:
受限于客户网络环境的可用性,Data Transfer Hub 支持以下用例:

* 在 AWS 区域和 AWS 中国区域之间复制 Amazon S3 对象。
* 将其他云提供商的对象存储服务(包括阿里云 OSS、腾讯 COS、七牛 Kodo)的数据复制到 Amazon S3。
* 在 AWS 区域和 AWS 中国区域之间传输 Amazon ECR 镜像。
* 在 AWS 中国区域和其它 AWS 区域之间复制 Amazon S3 对象。
* 将其它云提供商的对象存储服务的数据复制到 Amazon S3。
* 在 AWS 中国区域和其它 AWS 区域之间传输 Amazon ECR 镜像。
* 将来自公共 Docker 容器镜像仓库(例如 Docker Hub、Google gcr.io、Red Hat Quay.io)的 Docker 镜像传输到 Amazon ECR。
2 changes: 1 addition & 1 deletion docs/zh-base/user-guide/tutorial-cli-launch.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

1. 创建一个具有两个公有子网或两个拥有[NAT 网关][nat] 私有子网的Amazon VPC。

2. 根据需要替换`<CLOUDFORMATION_URL>``https://s3.amazonaws.com/solutions-reference/data-transfer-hub/latest/DataTransferS3Stack-ec2.template`
2. 根据需要替换`<CLOUDFORMATION_URL>``https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template`

3. 转到您的终端并输入以下命令。详情请参考**参数列表**

Expand Down
Loading

0 comments on commit f39376f

Please sign in to comment.