Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DeltaTable.create raises aws region errors with moto #3111

Open
jensqin opened this issue Jan 9, 2025 · 1 comment
Open

DeltaTable.create raises aws region errors with moto #3111

jensqin opened this issue Jan 9, 2025 · 1 comment
Labels
binding/python Issues for the Python package bug Something isn't working

Comments

@jensqin
Copy link

jensqin commented Jan 9, 2025

Environment

Delta-rs version: 0.23.2

Binding: 0.23.2

Environment:

  • Cloud provider: moto
  • OS: macOS
  • Other:

Bug

What happened: DeltaTable.create raises aws region errors with moto

[undefined]OSError: Generic S3 error: Error performing list request: Received redirect without LOCATION, this normally indicates an incorrectly configured region

What you expected to happen: successful run with no errors

How to reproduce it:

import boto3
import pytest
from deltalake import DeltaTable, Field, Schema
from moto import mock_aws


def test_deltalake():
    with mock_aws():
        conn = boto3.resource("s3", region_name="us-east-1")
        conn.create_bucket(Bucket="mybucket")

        bucket = S3Bucket("mybucket", "us-east-1")
        bucket.save_object("hello", "world")

        body = conn.Object("mybucket", "hello").get()["Body"].read()
        assert body == b"world"

        schema = Schema([Field("x", "integer"), Field("y", "integer")])

        storage_options = {"AWS_REGION": "us-east-1"}
        dt = DeltaTable.create(
            "s3://mybucket/test", schema, storage_options=storage_options
        )
        dt = DeltaTable("s3://mybucket/test", storage_options=storage_options)
        assert dt is not None

More details:

@jensqin jensqin added the bug Something isn't working label Jan 9, 2025
@ion-elgreco
Copy link
Collaborator

ion-elgreco commented Jan 9, 2025

@jensqin have you tried with "aws_region" lowercase?

This issue might already been fixed in main since this PR: #3107

@rtyler rtyler added the binding/python Issues for the Python package label Jan 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/python Issues for the Python package bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants