Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for virtual workflow threads #2297

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

Quinn-With-Two-Ns
Copy link
Contributor

@Quinn-With-Two-Ns Quinn-With-Two-Ns commented Oct 25, 2024

Add support for virtual workflow threads

Closes #1448

@Quinn-With-Two-Ns Quinn-With-Two-Ns marked this pull request as ready for review October 31, 2024 18:32
@Quinn-With-Two-Ns Quinn-With-Two-Ns requested a review from a team as a code owner October 31, 2024 18:32
Copy link
Member

@cretz cretz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Only main thing is if we can get the full existing test suite running against virtual thread-based workers/workflows to see if it could help us find interesting things.

// To test the virtual thread support we need to run a seprate test suite with Java 21
testing {
suites {
virtualThreadTests(JvmTestSuite) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if, in addition to this, it'd also be valuable in CI to run the whole existing test suite with virtual threads on for all workflow workers. If we have tests on thread interruption or deadlock detection, I think we want to confirm they still work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have tests on thread interruption or deadlock detection

Virtual threads don't behave any differently then platform threads here FWIW so I don't think it is providing a lot of value to double our CI time.

Copy link
Member

@cretz cretz Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a whole suite of different workflow logic/expectations, might as well confirm it works. I think we can run as a concurrent CI job to not double our time. I think the peace of mind is worth it. Should still have virtual-thread specific tests like these, but a whole CI run where virtual threads is enabled for everything (workflows and all 4 types of workers) would give some comfort.

Maybe even if it wasn't on in CI, even the ability to run it (e.g. some flag that changes all test workers and whatever supported plumbing needed) and confirmation that we did run it once and it works would be great.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that what this test suite is for if there is specific tests we can add them to this test site, but the differences between platforms and virtual threads are not enough to justify running all tests twice

Copy link
Contributor Author

@Quinn-With-Two-Ns Quinn-With-Two-Ns Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can revisit this though when we mark virtual threads as non experimental

edit: revisit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe even if it wasn't on in CI, even the ability to run it (e.g. some flag that changes all test workers and whatever supported plumbing needed) and confirmation that we did run it once and it works would be great.

Yep I can do that

* called for every thread created.
*/
@FunctionalInterface
public interface ThreadConfigurator {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure you need a whole new interface for what could just be a Consumer<Thread> (and maybe then remove justification for a new package), but not a big deal

* <p>Default is false
*/
@Experimental
public Builder setUseVirtualWorkflowThreads(boolean useVirtualWorkflowThreads) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrmm, useVirtualThreadsOnWorkflowWorker vs useVirtualWorkflowThreads is a tad confusing, but I can't think of a better way to represent the difference than what you've done

Copy link
Member

@cretz cretz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, but would like at least one run of the test suite done with all virtual threads in all 5 ways enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for virtual threads (project Loom)
2 participants