Java S3 client cannot access files after updating to 1.4.0

[jjimenezgarcia]

Hello, I'm having an issue after an update from 1.3.0 to 1.4.0.

I access my files through s3 client Java API. My buckets keep their layout properly after updating, so I assume that is not the problem.
After the update, I can no more access those files from my Java application (but I can through command line, so files do exist, they have not disappeared). The strangest is that I can upload files by the same Java application, and only then I can access those new files. Which means, I lose access to all pre-existing files, but can work with new ones without problem.

Hope you can help. This is my Java code:

[ChenSammi]

Are there any logs or error messages can be shared when access fails?

[adoroszlai]

1.5.0 is not released yet.

[jjimenezgarcia]

But don't know if the ETag is the only responsible. As I told @ivandika3 files can be accessed without problem using 'aws s3api' from command line or python.

Java seems to be my only problem.

[adoroszlai]

files can be accessed without problem using 'aws s3api' from command line or python.

The same situation was described HDDS-10521 which @ivandika3 pointed out.

Can you please check ETag in aws s3api get-object response?

[ivandika3]

Could you try to set System.setProperty("com.amazonaws.services.s3.disableGetObjectMD5Validation", "true"); before initializing the S3 Java client and see whether the file can be downloaded?

Ref: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/internal/SkipMd5CheckStrategy.html

The reason I suspected the Etag, was because the issue only comes up when downloading using AWS Java SDK, although it worked on AWS CLI.

You can also try to debug it in the IDE and let us know which part of code was causing the issue.

[jjimenezgarcia]

You are a 100% right. I was just listing the objects and getting those dated ETag. Sorry for the prior misunderstanding.

When I download it with get-object. Old files (uploadded on 1.3.0) have no ETag, as shown here:

{
    "AcceptRanges": "bytes",
    "LastModified": "Wed, 08 May 2024 09:26:32 GMT",
    "ContentLength": 31503,
    "ETag": "\"null\"",
    "CacheControl": "no-cache",
    "ContentType": "application/octet-stream",
    "Expires": "Thu, 09 May 2024 07:44:06 GMT",
    "Metadata": {}
}

But new files do have a hashed ETag, as shown here in a 1.4.0 added file:

{
    "AcceptRanges": "bytes",
    "LastModified": "Tue, 07 May 2024 11:46:40 GMT",
    "ContentLength": 1461,
    "ETag": "\"6838af242c1394bc088d0d9991256a29\"",
    "CacheControl": "no-cache",
    "ContentType": "application/octet-stream",
    "Expires": "Thu, 09 May 2024 07:44:51 GMT",
    "Metadata": {}
}

So that's the reason of the ArrayIndexOutOfBoundsException.

What would the solution be? I'm deploying S3G with docker image apache/ozone:1.4.0, which I see is the latest docker image released.

EDIT: I will try your solution @ivandika3, answered while you were writting, dind't see your message.

[jjimenezgarcia]

Sorry @ivandika3 , do you know how I could initialize that inside the docker container?

Trying to do so but clearly stuck, as the app gets initialized not with a jar file, but with the specific ozone command inside the container.

[ivandika3]

@jjimenezgarcia Where does your AWS S3 client code deployed? It's a client-side property, just need to include it in the client-side code. There is no need to redeploy anything server-side.

[jjimenezgarcia]

Oh sorry, I thought it was a server-side configuration. Will try then.