Observing response time differ largely for concurrent calls, for dynamodb query

[joyceliaoxy]

• Async AWS SDK for Python version: 13.0.0
• Python version: 3.10
• Operating System: MacOS

Description

So I am writing a SDK, and use aioboto3 as the base lib of my SDK. I have a a separate class to wrapping aioboto3 session, table and resources. And another class to perform the DB method, like put_item, delete_item, get_item... So what happens is that I tried to use the SDK I created to send multiple concurrent query to dynamodb. I got response time at the client side like:

• reqeust 1 response 100mili-sec
• requet2 response 1000mili-sec
• request3 response 800mili-sec
• request4 response 1500mili-sec

What I Did

The aioboto3 session wraping code:

class DbConnection:
    def __init__(self):
        self.session = aioboto3.Session()
    async def  db_client(self):
         async with self.session.client('dynamodb', region_name='us-east-2') as client:
             return client

class DbOperation:
    async def init(self):
        self.db_client = DbConnection.db_client()
     
    async def get_item(self, id):
        query_params = xxx
        sent_time = time.perf_count()
        response = await self.db_client.query(**query_params)
        response_time = time.perf_count() - sent_time

[terricain]

Returning a client within an async context manager will essentially close the client, you'd be somewhat better off doing something like: (untested)

class DbConnection:
    def __init__(self):
        self.session = aioboto3.Session()
    async def  db_client(self):
         return self.session.client('dynamodb', region_name='us-east-2').__aenter__()

class DbOperation:
    async def init(self):
        self.db_client = await DbConnection().db_client()
     
    async def get_item(self, id):
        query_params = xxx
        sent_time = time.perf_count()
        response = await self.db_client.query(**query_params)
        response_time = time.perf_count() - sent_time

[joyceliaoxy]

I've added this change to my code, but I still observe this big diff. My test setup is like I have a container running in a EKS pod and in the same region with DynamoDB, let's say I send 10 requests to get the same item, then my response time looks like:
request_1: response time: 91 ms
request_2: response time: 200 ms
request_3: response time: 700 ms
request_4: response time: 1000 ms
request_5: response time: 1500 ms
request_6: response time: 100 ms
request_7: response time: 250 ms
request_8: response time: 800 ms
request_9: response time: 1600 ms
request_10: response time: 2000 ms

[joyceliaoxy]

Another add-on question, if I use aiotboto3 resource whether it will be faster than aioboto3 client or not? if so how to construct resource connection in a correct way?