rt: overhaul task hooks

[Noah-Kennedy]

This change overhauls the entire task hooks system so that users can propagate arbitrary information between task hook invocations and pass context data between the hook "harnesses" for parent and child tasks at time of spawn.

This is intended to be significantly more extensible and long-term maintainable than the current task hooks system, and should ultimately be much easier to stabilize.

[Noah-Kennedy]

I'll fix CI tomorrow.

[Noah-Kennedy]

this is incorrect, and I will need to fix it

[rcoh]

Just reading the API, I really like this. I think its an API that allows us to expand in the future (especially if there isn't a huge perf hit by virtue of open-ended XYZContext objects).

I might consider keeping the current hook APIs (but implementing them behind the scenes with the new API). I think, ultimately, they may still be simpler for some use cases.

[Noah-Kennedy]

Just reading the API, I really like this. I think its an API that allows us to expand in the future (especially if there isn't a huge perf hit by virtue of open-ended XYZContext objects).

I might consider keeping the current hook APIs (but implementing them behind the scenes with the new API). I think, ultimately, they may still be simpler for some use cases.

Thank you for the feedback! Expandability was one of my main priorities with this changeset, and I'm delighted to hear that others are also excited about this.

Regarding keeping both sets of hooks around, I don't think this is a great idea due to the complexity of the systems involved, and I honestly just wanna get rid of the old APIs as they were never very good to begin with. That being said, I don't think it will be much work for folks to migrate over, and my hope is that we can leave this API completely unchanged even while we let it bake prior to stabilization, so hopefully no more migrations will be needed.

[Noah-Kennedy]

I'm running into some issues with loom, and I'm currently unsure if this is because of an actual issue or just false positives.

@Darksonn do you have any idea what might be causing the current failures with the multi-threaded scheduler?

[mox692]

Hmm, it looks like the loom failure log is not showing up on ci? ... I captured this locally anyway:

logs

running 1 test
test runtime::tests::loom_multi_thread::group_c::pool_shutdown has been running for over 60 seconds

thread 'runtime::tests::loom_multi_thread::group_c::pool_shutdown' panicked at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/loom-0.7.2/src/rt/location.rs:115:9:
Causality violation: Concurrent write accesses to `UnsafeCell`.

stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: loom::rt::location::PanicBuilder::fire
   3: loom::rt::cell::State::track_write
   4: scoped_tls::ScopedKey<T>::with
   5: loom::rt::cell::Cell::start_write
   6: loom::cell::unsafe_cell::UnsafeCell<T>::with_mut
   7: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
   8: tokio::runtime::scheduler::multi_thread::worker::Context::run
   9: tokio::runtime::context::scoped::Scoped<T>::set
  10: loom::thread::LocalKey<T>::try_with
  11: tokio::runtime::context::runtime::enter_runtime
  12: tokio::runtime::scheduler::multi_thread::worker::run
  13: loom::cell::unsafe_cell::UnsafeCell<T>::with_mut
  14: tokio::runtime::task::core::Core<T,S>::poll
  15: tokio::runtime::task::harness::Harness<T,S>::poll
  16: loom::cell::unsafe_cell::UnsafeCell<T>::with_mut
  17: tokio::runtime::task::UnownedTask<S>::run
  18: tokio::runtime::blocking::pool::Inner::run
  19: core::ops::function::FnOnce::call_once{{vtable.shim}}
  20: generator::stack::StackBox<F>::call_once
  21: generator::detail::gen::gen_init_impl
  22: generator::detail::asm::gen_init
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
worker thread panicking; aborting process
error: test failed, to rerun pass `--lib`

I haven't looked the entire code, but I suspect there's a data race caused by a concurrent write to an UnsafeCell somewhere.

[Noah-Kennedy]

Hmm, it looks like the loom failure log is not showing up on ci? ... I captured this locally anyway:
logs

I haven't looked the entire code, but I suspect there's a data race caused by a concurrent write to an UnsafeCell somewhere.

One of the loom tests (pool_shutdown) is timing out because it goes on too long, probably because somehow the number of branches is exploding.