Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move Threadpool class to separate module #151

Merged
merged 3 commits into from
Jan 27, 2025
Merged

Conversation

hoytak
Copy link
Collaborator

@hoytak hoytak commented Jan 24, 2025

Move the ThreadPool class to a separate module for organization purposes, and to allow adding functionality to mix async
and non-async parallel compute.

This PR is the first step of incorporating parts of #131, which proposed a model to achieve the larger and longer term goals of:

  • Allow us to isolate the async part of our codebase to only the portions that really need it.
  • Abstract away the specific async runtime (now tokio) to allow us to swap it out for lighter weight runtime down the road if desired.
  • Expose compute focused non-async primitives (e.g. par_for, JoinSet, JoinHandle) that we use frequently in the async world, but that could easily be done in the multithreaded sync world instead.
  • Expose reliable ways to call into async routines from the non-async runtime.
  • Lay the groundwork for better CTRL-C cancellations.

The XetRuntime class in this PR was the current ThreadPool class with added functions to allow spawning compute task; it was designed to allow all the current uses of tokio async primatives that would be better as compute primitives to be turned seamlessly into compute primitives.

@hoytak hoytak changed the title Moved Threadpool class to separate module to allow adding non-async compute features in the future. Move Threadpool class to separate module Jan 24, 2025
@hoytak hoytak merged commit 5f1176d into main Jan 27, 2025
2 checks passed
@hoytak hoytak deleted the hoytak/250122-move-xet-runtime branch January 27, 2025 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants