Lack of asynchronous processing

[MilesCranmer]

For some reason it seems like the line

SymbolicRegression.jl/src/SymbolicRegression.jl

Line 889 in c5ed5d0

isready(state.channels[j][i])

is always returning true.

I really don't understand this. But it seems as though this would mean there isn't a chance for asynchronous processing to take place?

[MilesCranmer]

Okay it's not always true. But by the time the loop gets back to it, it is true.

Is this the Julia scheduler's fault maybe? Is it finishing tasks without giving the head node enough time to work?

[MilesCranmer]

Apparently quinnj has a workaround for this issue:

Just as an FYI, I made a very lightweight package that allows spawning multithreaded tasks on a non-main thread. i.e. it will keep a single main thread free to respond to other tasks/things, and only extra "worker" threads will be eligible to perform @spawn-ed tasks. It works well in the application/microservice setup where it's important to keep a "main" thread open to respond to health checks, heartbeats, etc. ref: https://github.com/JuliaServices/WorkerUtilities.jl