Allow use of non-dualstack endpoints for S3 blocks storage

[sam-mcbr]

Describe the bug
Cannot configure the S3 blocks storage endpoint to point to a non-dualstack endpoint. In our config we have the blocks_storage.s3.endpoint set to s3.us-east-1.amazonaws.com. However, we observe the Cortex pods only reaching out to <bucket name>.s3.dualstack.us-east-1.amazonaws.com.

To Reproduce
Steps to reproduce the behavior:

1. Start Cortex (v1.18.1)
2. In the config, set the blocks_storage.s3.endpoint to a non-dualstack endpoint such as s3.us-east-1.amazonaws.com
3. Send data to Cortex
4. Observe that the endpoint the pods are connecting to is still <bucket name>.s3.dualstack.us-east-1.amazonaws.com

Expected behavior
We would expect Cortex to support hitting this non-dualstack endpoint. If we configure the endpoint as s3.us-east-1.amazonaws.com, Cortex should hit <bucket name>.s3.us-east-1.amazonaws.com.

Environment:

• Infrastructure: Kubernetes
• Deployment tool: helm

Additional Context
Minio does support this configuration (minio/minio-go#1945) as of v7.0.69 which is available in Cortex 1.17.1 and up. The value defaults to true for backwards compatibility. This function should allow the configuration of this option.

[mehta-ankit]

To add some context, we are seeing performance issues on our K8s clusters especially with cilium and its cache.
Saw this comment and wanted to try this and not use dualstack s3 endpoints.

[yeya24]

That's a configuration already exists in the S3 client we use. https://github.com/thanos-io/objstore/blob/a54d0f04b42aea5ef4e0fd0b9bb4ad75629ea070/providers/s3/s3.go#L120

Just need to expose this configuration in Cortex. Help wanted

[sam-mcbr]

Am working on a PR!