Skip to content
This repository has been archived by the owner on May 1, 2024. It is now read-only.

Spike: copy S3 database/schema data to another location in S3 #800

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Spike: copy S3 database/schema data to another location in S3 #800

wants to merge 8 commits into from

Commits on Feb 4, 2020

  1. Add CopyMysqlDatabaseFromS3ToS3Task.

    @brianhw
    brianhw committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    0503548 View commit details
    Browse the repository at this point in the history

  2. Use task instead of wrapper task.

    @brianhw
    brianhw committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    f0e2bb3 View commit details
    Browse the repository at this point in the history

  3. try setting signatureVersion.

    @brianhw
    brianhw committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    620b40d View commit details
    Browse the repository at this point in the history

  4. set part_size to ~3GB (from 64MB).

    @brianhw
    brianhw committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    6ed4cc3 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2020

  1. Try running aws s3 in a subprocess.

    @brianhw
    brianhw committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    e9102f2 View commit details
    Browse the repository at this point in the history

  2. Add awscli==1.14.46

    @brianhw
    brianhw committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    56751ad View commit details
    Browse the repository at this point in the history

  3. Add s3_to_s3 copy for Vertica schema output too.

    @brianhw
    brianhw committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    1cd167e View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2020

  1. Change Vertica names from database to schema.

    @brianhw
    brianhw committed Feb 6, 2020
    Configuration menu
    Copy the full SHA
    419a7b8 View commit details
    Browse the repository at this point in the history