Unable to use in ecs-fargate with IAM role

[bdavj]

Trying to use nginx-s3-gateway in ECS-Fargate, using an IAM role. I've tried substituting the image in with a simple docker image that runs an S3 ls, which works fine, so comms and task role authentication appears to be working fine.
Using the same account, with access credentials locally, I can directory list and pull files fine.

When running in ecs-fargate, without the key/secret etc filled, I get no errors on startup, however all page loads result in a 404. I've attempted to remove the restriction on the x-amzn headers, and indeed the proxy error for 404s to try and get more detail, however all I see is the vanilla 404, and the GET request logged (even with debug logging on).

Is there any way to get a bit more information as to where the S3 call is failing?

[4141done]

Hi @bdavj - sorry to hear you're having issues.

I just tested the DEBUG configuration option with an authentication issue and I think ran into the same thing. By default we cache 404 responses for one minute so you're probably seeing the cached response. I'd see a helpful error the first time but subsequent calls just logged a GET with a 404 response.

I suggest that you try making sure that the values below are set either not to cache as below or are set to a very low value like 1s:

S3_STYLE=virtual-v2
S3_SERVICE=s3
DEBUG=true
AWS_SIGS_VERSION=4
PROXY_CACHE_VALID_NOTFOUND=0
PROXY_CACHE_VALID_FORBIDDEN=0

At that point, the debug output should give you something like this:

2024/05/17 19:45:27 [info] 77#77: *1 js: S3 Request URI: GET /b/c/d.txt
2024/05/17 19:45:27 [info] 77#77: *1 js: S3 Request URI: GET /b/c/d.txt
2024/05/17 19:45:27 [info] 77#77: *1 js: AWS v4 Auth Canonical Request: [GET
/b/c/d.txt

host:my-bucket-name.s3.us-east-1.amazonaws.com:443
x-amz-content-sha256:REDACTED
x-amz-date:20240517T194527Z
x-amz-security-token:REDACTED

host;x-amz-content-sha256;x-amz-date;x-amz-security-token
REDACTED]
2024/05/17 19:45:27 [info] 77#77: *1 js: AWS v4 Auth Canonical Request Hash: [REDACTED]
2024/05/17 19:45:27 [info] 77#77: *1 js: AWS v4 Auth Signing String: [AWS4-HMAC-SHA256
20240517T194527Z
20240517/us-east-1/s3/aws4_request
REDACTED]
2024/05/17 19:45:27 [info] 77#77: *1 js: AWS v4 Signing Key Hash: [REDACTED]
2024/05/17 19:45:27 [info] 77#77: *1 js: AWS v4 Authorization Header: [REDACTED]
2024/05/17 19:45:27 [info] 77#77: *1 js: AWS v4 Auth header: [AWS4-HMAC-SHA256 Credential=REDACTED/20240517/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token,Signature=REDACTED]
192.168.65.1 - - [17/May/2024:19:45:27 +0000] "GET /b/c/d.txt HTTP/1.1" 404 146 "-" "curl/8.4.0" "-"

I'm sure you know but stating for posterity I'll mention that you'll need to either tail the integrated docker logs or specifically look at /var/log/nginx/error.log.

Comment back here if the above doesn't work and we can explore other directions.