Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: Concurrency control #49

Closed
Iamcerba opened this issue Oct 30, 2023 · 1 comment
Closed

Question: Concurrency control #49

Iamcerba opened this issue Oct 30, 2023 · 1 comment

Comments

@Iamcerba
Copy link

Do you have real control over concurrency?

From what I see in the code there is "max_workers" parameter in config that is used to create custom ThreadPoolExecutor.
What I know from the documentation is that loop.set_default_executor works together with loop.run_in_executor and I'm not sure that it works with asyncio.gather.
Thus all catalog entries are synced in parallel in a single thread (using async io event loop) without any limit in concurrency.

Correct me if I'm wrong.

@Iamcerba Iamcerba changed the title Concurrency control Question: Concurrency control Oct 30, 2023
@Iamcerba
Copy link
Author

Iamcerba commented Oct 30, 2023

Answering my question:

I've found in the code call to loop.run_in_executor, thus each catalog entry is synchronized in a separate Thread and inside these threads "asyncio" isn't being used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

1 participant