Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
sethkor committed Jun 15, 2019
1 parent 8c19089 commit 29bffef
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ This is WIP, some further features to come. Features available:
- [X] Copy to S3
- [X] Copy from S3
- [X] Copy S3 to S3
- [ ] Copy S3 to S3 in another account with seperate credentials
- [X] Copy S3 to S3 in another account with seperate credentials

Please raise an issue if theres a specific feature you. would like considered or prioritised.

Expand All @@ -109,6 +109,8 @@ Please raise an issue if theres a specific feature you. would like considered or
The AWS KMS key ID that should be used to server-side encrypt the object in S3.
--acl=private Object ACL
--storage-class=STANDARD Storage Class
--dest-profile=DEST-PROFILE
Destination bucket AWS credentials/config file profile to use if different from --profile
Args:
<source> file or s3 location
Expand All @@ -125,6 +127,8 @@ For optimal throughput consider using a S3 VPC Gateway endpoint if you are execu

And remember the performance of the source storage device is important, you don't want to choke it reading lots of data at once. Use an optimized iops device or SAN.

For S3 to S3 with different AWS profiles, object must be downloaded from the source and the uploaded. To cater for large objects and to limit memory usage this is done utilizing multi parts by relies on GO garbage collector to tidy things up promptly which doesn't always happen Some further optimizations are WIP but expect this operation to consume a bit of memory.

### Multiparts
Multipart chunks size for Upload or Download is set to 5MB. Any objects greater than 5MB in size shal be sent in multiparts. `s3kor` will send up to 5 parts concurrently per object.

Expand Down

0 comments on commit 29bffef

Please sign in to comment.