Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use asyncio for async operations in Runners #4289

Open
merelcht opened this issue Nov 1, 2024 · 1 comment
Open

Use asyncio for async operations in Runners #4289

merelcht opened this issue Nov 1, 2024 · 1 comment

Comments

@merelcht
Copy link
Member

merelcht commented Nov 1, 2024

Description

We can refactor the code in Task that handles asynchronous loading and saving of data to use asyncio.

Context

#4206 (review)

@astrojuanlu
Copy link
Member

The idea is to refactor this method here:

kedro/kedro/runner/task.py

Lines 192 to 198 in a1fae50

def _run_node_async(
self,
node: Node,
catalog: CatalogProtocol,
hook_manager: PluginManager, # type: ignore[arg-type]
session_id: str | None = None,
) -> Node:

Possible inspiration:

We set an appetite of 1 sprint but we might need another one to wrap it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants