-
-
Notifications
You must be signed in to change notification settings - Fork 907
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Force region to us-west-2
won't raise error, it always try new request with us-west-1
#893
Comments
Thanks for the report. Indeed a bad bug. I will have a look at that.
Le 4 juil. 2017 6:47 AM, "Charles Hsu" <[email protected]> a écrit :
… $ s3cmd ls s3://test/ --region us-west-2 --debug
DEBUG: s3cmd version 2.0.0
DEBUG: ConfigParser: Reading file '/Users/charles/.s3cfg'
DEBUG: ConfigParser: access_key->de...1_chars...o
DEBUG: ConfigParser: access_token->
DEBUG: ConfigParser: add_encoding_exts->
DEBUG: ConfigParser: add_headers->
DEBUG: ConfigParser: ca_certs_file->
DEBUG: ConfigParser: cache_file->
DEBUG: ConfigParser: check_ssl_certificate->True
DEBUG: ConfigParser: check_ssl_hostname->True
DEBUG: ConfigParser: cloudfront_host->cloudfront.amazonaws.com
DEBUG: ConfigParser: default_mime_type->binary/octet-stream
DEBUG: ConfigParser: delay_updates->False
DEBUG: ConfigParser: delete_after->False
DEBUG: ConfigParser: delete_after_fetch->False
DEBUG: ConfigParser: delete_removed->False
DEBUG: ConfigParser: dry_run->False
DEBUG: ConfigParser: enable_multipart->True
DEBUG: ConfigParser: encoding->US-ASCII
DEBUG: ConfigParser: encrypt->False
DEBUG: ConfigParser: expiry_date->
DEBUG: ConfigParser: expiry_days->
DEBUG: ConfigParser: expiry_prefix->
DEBUG: ConfigParser: follow_symlinks->False
DEBUG: ConfigParser: force->False
DEBUG: ConfigParser: get_continue->False
DEBUG: ConfigParser: gpg_command->None
DEBUG: ConfigParser: gpg_decrypt->%(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
DEBUG: ConfigParser: gpg_encrypt->%(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
DEBUG: ConfigParser: gpg_passphrase->...-3_chars...
DEBUG: ConfigParser: guess_mime_type->True
DEBUG: ConfigParser: host_base->192.168.190.21
DEBUG: ConfigParser: host_bucket->192.168.190.21
DEBUG: ConfigParser: human_readable_sizes->False
DEBUG: ConfigParser: invalidate_default_index_on_cf->False
DEBUG: ConfigParser: invalidate_default_index_root_on_cf->True
DEBUG: ConfigParser: invalidate_on_cf->False
DEBUG: ConfigParser: kms_key->
DEBUG: ConfigParser: limit->5
DEBUG: ConfigParser: limitrate->0
DEBUG: ConfigParser: list_md5->False
DEBUG: ConfigParser: log_target_prefix->
DEBUG: ConfigParser: long_listing->False
DEBUG: ConfigParser: max_delete->-1
DEBUG: ConfigParser: mime_type->
DEBUG: ConfigParser: multipart_chunk_size_mb->10
DEBUG: ConfigParser: multipart_max_chunks->10000
DEBUG: ConfigParser: preserve_attrs->True
DEBUG: ConfigParser: progress_meter->True
DEBUG: ConfigParser: proxy_host->
DEBUG: ConfigParser: proxy_port->0
DEBUG: ConfigParser: put_continue->False
DEBUG: ConfigParser: recursive->False
DEBUG: ConfigParser: recv_chunk->4096
DEBUG: ConfigParser: reduced_redundancy->False
DEBUG: ConfigParser: requester_pays->False
DEBUG: ConfigParser: restore_days->1
DEBUG: ConfigParser: restore_priority->Standard
DEBUG: ConfigParser: secret_key->09...29_chars...1
DEBUG: ConfigParser: send_chunk->4096
DEBUG: ConfigParser: server_side_encryption->False
DEBUG: ConfigParser: signature_v2->False
DEBUG: ConfigParser: simpledb_host->
DEBUG: ConfigParser: skip_existing->False
DEBUG: ConfigParser: socket_timeout->300
DEBUG: ConfigParser: stats->False
DEBUG: ConfigParser: stop_on_error->False
DEBUG: ConfigParser: storage_class->
DEBUG: ConfigParser: urlencoding_mode->normal
DEBUG: ConfigParser: use_http_expect->False
DEBUG: ConfigParser: use_https->False
DEBUG: ConfigParser: use_mime_magic->True
DEBUG: ConfigParser: verbosity->WARNING
DEBUG: Updating Config.Config bucket_location -> us-west-2
DEBUG: Updating Config.Config cache_file ->
DEBUG: Updating Config.Config follow_symlinks -> False
DEBUG: Updating Config.Config verbosity -> 10
DEBUG: Unicodising 'ls' using US-ASCII
DEBUG: Unicodising 's3://test/' using US-ASCII
DEBUG: Command: ls
DEBUG: Bucket 's3://test':
DEBUG: CreateRequest: resource[uri]=/
DEBUG: Using signature v4
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: canonical_headers = host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T044130Z
DEBUG: Canonical Request:
GET
/test/
delimiter=%2F&max-keys=5
host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T044130Z
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/us-west-2/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=b28bd10c8bbe451401434cc1b507ddeffa777e3b1269e62a1d4cb0180ab930fc', 'x-amz-date': '20170704T044130Z'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: ConnMan.get(): creating new connection: http://192.168.190.21
DEBUG: non-proxied HTTPConnection(192.168.190.21, None)
DEBUG: format_uri(): /test/?delimiter=%2F&max-keys=5
DEBUG: Sending request method_string='GET', uri=u'/test/?delimiter=%2F&max-keys=5', headers={'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/us-west-2/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=b28bd10c8bbe451401434cc1b507ddeffa777e3b1269e62a1d4cb0180ab930fc', 'x-amz-date': '20170704T044130Z'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (http://192.168.190.21#1)
DEBUG: Response:
{'data': "<?xml version='1.0' encoding='UTF-8'?>\n<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-west-2' is wrong; expecting 'us-west-1'</Message><RequestId>txe7f3c30a46054a15b2297-00595b1c7a</RequestId><Region>us-west-1</Region></Error>",
'headers': {'content-type': 'application/xml',
'date': 'Tue, 04 Jul 2017 04:41:30 GMT',
'transfer-encoding': 'chunked',
'x-amz-id-2': 'txe7f3c30a46054a15b2297-00595b1c7a',
'x-amz-request-id': 'txe7f3c30a46054a15b2297-00595b1c7a',
'x-openstack-request-id': 'txe7f3c30a46054a15b2297-00595b1c7a',
'x-trans-id': 'txe7f3c30a46054a15b2297-00595b1c7a'},
'reason': 'Bad Request',
'status': 400}
INFO: Forwarding request to us-west-1
DEBUG: Using signature v4
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: canonical_headers = host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T044130Z
DEBUG: Canonical Request:
GET
/test/
delimiter=%2F&max-keys=5
host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T044130Z
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-date': '20170704T044130Z', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/us-west-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=a11cdfc0fa87d60837e4b186269f4d979add96fcba1e1b0db7287a9668b2811d'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: ConnMan.get(): re-using connection: http://192.168.190.21#1
DEBUG: format_uri(): /test/?delimiter=%2F&max-keys=5
DEBUG: Sending request method_string='GET', uri=u'/test/?delimiter=%2F&max-keys=5', headers={'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-date': '20170704T044130Z', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/us-west-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=a11cdfc0fa87d60837e4b186269f4d979add96fcba1e1b0db7287a9668b2811d'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (http://192.168.190.21#2)
DEBUG: Response:
{'data': '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>test</Name><Prefix/><Marker/><MaxKeys>5</MaxKeys><Delimiter>/</Delimiter><IsTruncated>false</IsTruncated><Contents><Key>100m</Key><LastModified>2017-06-14T03:58:47.776Z</LastModified><ETag>"2f282b84e7e608d5852449ed940bfc51"</ETag><Size>104857600</Size><Owner><ID>demo</ID><DisplayName>demo</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents><Contents><Key>200m</Key><LastModified>2017-06-14T04:06:41.695Z</LastModified><ETag>"f9fac3e603b2d0f0b769a2be4cc3912f"</ETag><Size>209715200</Size><Owner><ID>demo</ID><DisplayName>demo</DisplayName></Owner><StorageClass>STANDARD</StorageClass></Contents></ListBucketResult>',
'headers': {'content-length': '743',
'content-type': 'application/xml',
'date': 'Tue, 04 Jul 2017 04:41:30 GMT',
'x-amz-id-2': 'tx51ed233540d44381b620d-00595b1c7a',
'x-amz-request-id': 'tx51ed233540d44381b620d-00595b1c7a',
'x-openstack-request-id': 'tx51ed233540d44381b620d-00595b1c7a',
'x-trans-id': 'tx51ed233540d44381b620d-00595b1c7a'},
'reason': 'OK',
'status': 200}
2017-06-14 03:58 104857600 s3://test/100m
2017-06-14 04:06 209715200 s3://test/200m
infinity request when my s3 in another region 'US'
DEBUG: ===== Send_request inner request to determine the bucket region =====
DEBUG: CreateRequest: resource[uri]=/
DEBUG: Using signature v4
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: canonical_headers = host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T042429Z
DEBUG: Canonical Request:
GET
/test/
location=
host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T042429Z
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=ca736e60c07eee05b018ac4fd079858067e813e4859fc44508b43194da590091', 'x-amz-date': '20170704T042429Z'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: ConnMan.get(): re-using connection: http://192.168.190.21#274
DEBUG: format_uri(): /test/?location
DEBUG: Sending request method_string='GET', uri=u'/test/?location', headers={'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=ca736e60c07eee05b018ac4fd079858067e813e4859fc44508b43194da590091', 'x-amz-date': '20170704T042429Z'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (http://192.168.190.21#275)
DEBUG: Response:
{'data': "<?xml version='1.0' encoding='UTF-8'?>\n<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'US'</Message><RequestId>tx336ab0a0430d4380b24ea-00595b187c</RequestId><Region>US</Region></Error>",
'headers': {'content-type': 'application/xml',
'date': 'Tue, 04 Jul 2017 04:24:28 GMT',
'transfer-encoding': 'chunked',
'x-amz-id-2': 'tx336ab0a0430d4380b24ea-00595b187c',
'x-amz-request-id': 'tx336ab0a0430d4380b24ea-00595b187c',
'x-openstack-request-id': 'tx336ab0a0430d4380b24ea-00595b187c',
'x-trans-id': 'tx336ab0a0430d4380b24ea-00595b187c'},
'reason': 'Bad Request',
'status': 400}
DEBUG: Error getlocation inner request: maximum recursion depth exceeded in cmp
DEBUG: Using signature v4
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: canonical_headers = host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T042429Z
DEBUG: Canonical Request:
GET
/test/
location=
host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T042429Z
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-date': '20170704T042429Z', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/US/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=e420dc4d5908f0321a70e02eb31502d06d45ee3b9e6cc085109b70ff79a50d12'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: ConnMan.get(): re-using connection: http://192.168.190.21#275
DEBUG: format_uri(): /test/?location
DEBUG: Sending request method_string='GET', uri=u'/test/?location', headers={'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-date': '20170704T042429Z', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/US/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=e420dc4d5908f0321a70e02eb31502d06d45ee3b9e6cc085109b70ff79a50d12'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (http://192.168.190.21#276)
DEBUG: Response:
{'data': '<?xml version=\'1.0\' encoding=\'UTF-8\'?>\n<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">US</LocationConstraint>',
'headers': {'content-length': '130',
'content-type': 'application/xml',
'date': 'Tue, 04 Jul 2017 04:24:28 GMT',
'x-amz-id-2': 'tx0eb9f7cb06154c02af756-00595b187c',
'x-amz-request-id': 'tx0eb9f7cb06154c02af756-00595b187c',
'x-openstack-request-id': 'tx0eb9f7cb06154c02af756-00595b187c',
'x-trans-id': 'tx0eb9f7cb06154c02af756-00595b187c'},
'reason': 'OK',
'status': 200}
DEBUG: ===== END send_request inner request to determine the bucket region ('us-east-1') =====
DEBUG: Using signature v4
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: canonical_headers = host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T042429Z
DEBUG: Canonical Request:
GET
/test/
location=
host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T042429Z
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-date': '20170704T042429Z', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=ca736e60c07eee05b018ac4fd079858067e813e4859fc44508b43194da590091'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: ConnMan.get(): re-using connection: http://192.168.190.21#276
DEBUG: format_uri(): /test/?location
DEBUG: Sending request method_string='GET', uri=u'/test/?location', headers={'x-amz-content-sha256': u'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'x-amz-date': '20170704T042429Z', 'Authorization': u'AWS4-HMAC-SHA256 Credential=demo/20170704/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=ca736e60c07eee05b018ac4fd079858067e813e4859fc44508b43194da590091'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (http://192.168.190.21#277)
DEBUG: Response:
{'data': "<?xml version='1.0' encoding='UTF-8'?>\n<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'US'</Message><RequestId>tx67555c5d8661468287291-00595b187c</RequestId><Region>US</Region></Error>",
'headers': {'content-type': 'application/xml',
'date': 'Tue, 04 Jul 2017 04:24:28 GMT',
'transfer-encoding': 'chunked',
'x-amz-id-2': 'tx67555c5d8661468287291-00595b187c',
'x-amz-request-id': 'tx67555c5d8661468287291-00595b187c',
'x-openstack-request-id': 'tx67555c5d8661468287291-00595b187c',
'x-trans-id': 'tx67555c5d8661468287291-00595b187c'},
'reason': 'Bad Request',
'status': 400}
INFO: Forwarding request to US
DEBUG: ===== Send_request inner request to determine the bucket region =====
DEBUG: CreateRequest: resource[uri]=/
DEBUG: Using signature v4
DEBUG: get_hostname(test): 192.168.190.21
DEBUG: canonical_headers = host:192.168.190.21
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170704T042429Z
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#893>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABAUpHxVtxwIK3fDCvBDXCg9SKuSX_wtks5sKcPPgaJpZM4OM4Te>
.
|
Hi @fviard, do you have idea how to force the region for s3cmd in the current release?
to
It works, but not sure if that impact other features. |
@charz I don't know if you still have the issue your issues, but I'm looking at the issue today, and I have a clearer mind.
It is probably setup with the string "US" defined as a region. Some s3 servers just ignore the "region" parameter of requests, |
I close the issue, because so I think there is not an issue on s3cmd side here. |
For the second point, "US", I see the issue #859 that is related. |
infinity request when my s3 in another region 'US'
It looks like the rest of requests always go with region
us-west-1
The text was updated successfully, but these errors were encountered: