Feature request: Add EventHandler for triggering asynchronous execution

[sinofseven]

Use case

I would like to use monolithic Lambda as a mechanism to build asynchronous execution Lambda Functions.

Python Runtime's SnapStart charges a cache fee for each version of the Lambda Function.
I believe that reducing the number of Lambda Functions created by Monolithic Lambda would be a useful cost saving measure.

While it is easy to build monolithic Lambda with API Gateway's backend Lambda because of EventHandler, it is not easy to build monolithic Lambda for asynchronous Lambda execution.
I would like to create a new EventHandler to facilitate this.

Solution/User Experience

app = AsynchronousTriggerResolver()


@app.sns("<String to identify>")
def process_sns_01():
    pass


@app.sns("<String to identify>")
def process_sns_02():
    pass


@app.event_bridge("<String to identify>")
def process_event_bridge():
    pass


@app.cloudwatch_logs("<String to identify>")
def process_cloudwatch_logs():
    pass


def lambda_handler(event, context):
    return app.resolve(event, context)

I am thinking that it would be good if we could use it like an EventHandler for API Gateway.

Alternative solutions

Acknowledgment

• This feature request meets Powertools for AWS Lambda (Python) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[leandrodamascena]

Hi @sinofseven Sorry for the super super late reply, but things are a bit slower at the end of the year because we are presenting at re:Invent, we need to deal with some internal stuff and most people are going on annual leave.

Regarding this issue, I can say that I really like the idea and we are discussing it in our roadmap for 2025. This could be a generic event handler for many services: S3, StepFunctions, EventBridge, SQS, SNS and we can decide which part of the event we can use as a route, regular expressions, part of the string, many possibilities. This will make life easier for people who are dealing with EventBridge for example and want to use a single Lambda for some events. I see a lot of benefits in this new utility and it would be cool to release it in 2025.

Do you want to create an RFC for this? Ideas? Prototyping?

[leandrodamascena]

Sorry, I pressed enter before complete the message.

We can work together on this RFC. We are really thinking about this utility in 2025.

[sinofseven]

@leandrodamascena

Thank you, your reply.

Do you want to create an RFC for this? Ideas? Prototyping?

Currently, I have only ideas.

I was waiting for it to be triaged according to “Contributing via Pull Requests” in CONTRIBUTION.md.

Now that it has been triaged, I am thinking of implementing a prototype.

My idea is to create an EventHandler for asynchronous Lambda triggers.

Since the current EventHandler is divided into one for REST API, one for HTTP API, and one for AppSync, I think it would be better to create an EventHandler for each type of Lambda execution (synchronous, asynchronous, event source mapping). I think it would be a good idea.
(For synchronous execution, I think it would be better to create one for each service such as API Gateway, AppSync, StepFunctions, Cognito, etc.)
(For event source mapping, I am wondering if it is better to use EventHandler for Lambda for asynchronous execution, or if I should create a separate EventHandler.)

[sinofseven]

In addition, the prototype implementation assumes asynchronous Lambda event sources whose types are defined in data_classes.

[sinofseven]

I have not yet confirmed operation, created tests, or written comments, but I have implemented the system and created a PR.

[leandrodamascena]

Commenting here for more visibility:

Hi @sinofseven, first of all, sorry for the late reply, but I was super busy these days. I was reading the code and it covers most of the aspects we expect from this utility, such as: routing per key, detail, object, event name and others. I must say we have a SUPER SUPER nice work here, thank you very much for that.

I have a proposal to follow with this PR and please let me know what do you think.

Creating a RFC

I'm going to start an RFC with what this implementation should cover. Powertools is available in 4 different languages and the idea is that this RFC can cover the design of this implementation, features that will be supported and resolvers that we will support in the first version. Don't worry we have people to implement in different languages. I need a week to have a v1 of this RFC and then you can contribute.

Keeping this PR open

I will keep the PR (#5799) open while we discuss the RFC, which will probably take something like 1 month, because we need to hear from the community, from you, from the other maintainers and we need to agree on the implementation. I will also run local tests with this PR to get more insight into the codebase, changes we need to make and other things.

Integrations

We have a utility called EventSource Data Class, where we map the event that comes from any service to Lambda, we can integrate this utility with this one and create an experience like:

from aws_lambda_powertools.async_resolver import S3Resolver

app = S3Resolver()

@app.resolve_by_name(object_name="powertools.txt")
def resolve_by_name():
    print(f"Event time {app.event_time}") # We can access the event properties

We also have a utility called Parser where we use Pydantic templates to do data parsing and validation against S3 Events. We can think of an idea how to integrate with that.

My personal opinion

While I like the idea of ​​having multiple resolvers like S3, EventBridge, SNS, SES, it might make sense to step back and agree to the following:

1 - Make the base router strong and consistent enough to support many types of resolvers
2 - Implement 2 initial resolvers like S3 and EventBridger, because I think events in this case are simpler to interpret and route
3 - Create tests that can be reused for many resolvers.
4 - As soon as we validate the 2 initial resolvers, we can add more.

I would love to hear your feedback and work together on this RFC to design this utility API and add this feature to Powertools.