Repository navigation
Serial loops when the team is busy, a raw chunk loop, and idle workers that sleep - #68
Merged
Merged
Conversation
maleadt
force-pushed
the
tb/ka-host-scheduling
branch
from
October 25, 2026 22:10
5e77920 to
e2a75da
Compare
maleadt
changed the base branch from
vc/stack-seenidle
to
vc/stack-onehandler
October 26, 2026 05:29
Collaborator
Author
|
AI-generated summary of the rework: Reworked and force-pushed. It now sits on #62 rather than #65 (#65 is independent of it), and on main with #57.
|
This was referenced Oct 26, 2026
An idle worker restarted its spin time whenever it saw the pool busy, whether or not the busy region used it. With loops back to back the pool is busy nearly all the time, so a worker that the loops leave out never went to sleep: with the caller on the interactive thread (`-t N` on Julia >= 1.12) chunks go to workers 1 to N - 1, and worker N spins for as long as the loops go on, once something woke it (the start of the pool, `parallel_phase`). On N cores that is N + 1 busy threads and time slicing: giordano measured 505-905 involuntary context switches under `-t N` against 0-36 under `-t N,0`. Now only a worker that comes from a chunk restarts its spin time on a busy pool, until it sees the pool idle (its region ended); a worker that no region uses counts from the end of its last chunk or wake-up, and sleeps after `spin_time()`. A finished worker still keeps spinning while the rest of its loop runs, as documented; if it misses the short gap before the next loop, it takes that loop for its own too. Back-to-back loops of 4356 elements from the main task, with all workers woken first, `taskset -c 0-3 julia -t 4` on a 4-core Xeon E3-1245 v6, 4 processes of 2 s each: worker 4's thread took 2-69% of a core (above 25% in 3 of 4), now 0% in every process.
A loop started while another task holds the team runs on `Threads.@spawn`ed tasks. That keeps independent jobs that each run team loops from waiting on each other, but in code that is itself run in parallel (tasks, `Threads.@threads`) it fans the loop out over threads that are busy already: with N such callers, about 2N runnable tasks, and a few microseconds and some allocations per loop for the spawns. Inside `@threads`, the first iteration to arrive also takes the team and wakes workers whose threads are running the other iterations. `TeamScheduler(busy = :serial)` and `GreedyTeamScheduler(busy = :serial)` run such a loop serially on the calling task instead (Polyester.jl's rule, and libomp's `max_active_levels = 1`): when the team is held by another task, and while a `Threads.@threads` loop runs anywhere in the process (`jl_in_threaded_region`, a process-wide counter, checked before taking the team). `nchunks` then reports one chunk while the team is busy. The default stays `:tasks`; the check of the field is the only addition on its path.
…MyThreads `foreach_chunk(f, m, p)` calls `f(c)` for `c in 1:m` on at most `p` threads, the caller being one of them: with `m == p` as a `TeamScheduler` region (chunk `c` to participant `c`), with more chunks as a greedy one. It is meant for callers like KernelAbstractions' host kernels that dispatch one callable type of their own to the team many times and need neither OhMyThreads' iteration machinery nor its keyword arguments. It always runs serially when the team is busy or a `Threads.@threads` loop runs, as `busy = :serial` does, and serially on the caller for one chunk or one thread. The arguments are positional so that `precompile(ThreadTeams.foreach_chunk, (S, Int, Int))` covers a callable type `S`. For that, `foreach_chunk` creates the workers' closure with a static call before the region, where `job_closure` uses a dynamic one that a `precompile` statement does not follow; the first loop then costs about 5 ms (the closure's body) instead of 60 ms with the statement alone and 350 ms without it (Julia 1.13). `dispatch!` is unchanged. A mode in which the caller waits for the workers instead of running chunks will add a `wait` argument later.
maleadt
force-pushed
the
tb/ka-host-scheduling
branch
from
October 26, 2026 07:00
e2a75da to
02b1ea4
Compare
Benchmark Results (Julia v1)Time benchmarks
|
This was referenced Oct 26, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
KernelAbstractions.jl's CPU back-end is moving to run PoCL work-groups on ThreadTeams (JuliaGPU/KernelAbstractions.jl#906), with the aim of replacing Polyester
@batchand AcceleratedKernels' threads back-end. Review there turned up three things ThreadTeams has to do differently, and each makes sense for OhMyThreads users too.A worker that no running loop uses sleeps after its spin time. An idle worker restarted its spin time whenever the pool was busy, also during loops that don't use it. Under
-t Non Julia ≥ 1.12 the main task runs on the interactive thread, so its loops use the caller plus workers 1 to N − 1, and worker N spins for as long as loops keep coming. Back-to-back 4356-element loops from the main task, with the process allowed N + 1 CPUs: worker N's thread used 99–100% of a core on main (draco with-t 3, pegasus with-t 4, five more hosts at N = 2 and 4) and 0–4% with this PR. With exactly N CPUs the pool is oversubscribed and yields, and it is intermittent (0–42% on draco). This is what giordano's Oceananigans runs showed as 505–905 involuntary context switches under-t 8, against 0–36 under-t 8,0. A worker that ran a chunk of the running loop still spins until that loop ends, as documented.busy = :serialon both schedulers. A loop started while another task holds the team runs onThreads.@spawned tasks, which in code that is itself parallel (tasks,@threads) fans the loop out over threads that are busy already. Withbusy = :serialsuch a loop runs serially on the caller, as does one started while aThreads.@threadsloop runs. That's Polyester's rule and libomp'smax_active_levels = 1. The@threadscheck is process-wide, so this is opt-in and the default stays:tasks. It adds one field load and a branch torun_chunks.ThreadTeams.foreach_chunk(f, m, p = nworkers()): a loop over chunk indices, for callers that dispatch one callable type of their own many times and need neither OhMyThreads' iteration machinery nor keyword arguments. Withm == pit is a static region, with more chunks a greedy one, and it is always serial when the team is busy. The arguments are positional so thatprecompile(ThreadTeams.foreach_chunk, (S, Int, Int))compiles it for a callable typeS: the first loop then costs about 6 ms, against 440 ms without the statement (Julia 1.12). An emptyforeach_chunk(f, P, P)costs 15–40% less than the equivalent statictforeach.The static loop is unchanged: within ±10 ns on x86 across 5 interleaved rounds on taurus (Zen 3), pegasus (Zen 5), sagittarius (Tiger Lake) and draco (Kaby Lake), and within the timer resolution on Apple silicon.
Pkg.test()passes on Julia 1.10, 1.12 and 1.13 with 1 and 4 threads, apart from the 3 Enzyme errors that main also has with OhMyThreads 0.8.8 (JuliaFolds2/OhMyThreads.jl#177).