Repository navigation
Share the rescue clock of steal blocks between participants - #84
Merged
Merged
Conversation
This fixes a slowdown of `schedule = :steal` on loaded hosts. Thieves waited up to 2 µs in `steal_allowed` for an awake owner that had not claimed its command, but each call started its own clock: the wait was paid again for every victim, on every pass, in every block the dispatcher rescued, and also for blocks other thieves had already emptied. With descheduled workers, the waits added up to most of a launch, or until the startup deadline: on a 9950X with one CPU hog per worker thread, an empty-body launch took 0.7 ms with 16 threads and 1.5 ms with 32, against 3-4 µs with `:static`. Each steal block now keeps the first awake observation of its owner in the region (`owner_due`), shared by thieves and `rescue_ranges`, on the line of its range. `steal_allowed` no longer waits; participants rescan while an owner may still become due. Started and empty blocks need no protection. Owners must still be awake with their command pending RANGE_AWAKE_NS after that observation, and sleeping owners and claimed commands keep their protection until the deadline. Deliberately, the timestamp now survives intervening observations of the owner asleep for the rest of the region, where the dispatcher used to restart its clock, so waits never renew. Static mode keeps its dispatcher-only clocks. Same host, median per launch, 1 hog/CPU: axpy 1e5 747 -> 20 µs (16 threads) and 1516 -> 38 µs (32 threads), axpy 1e6 827 -> 34 µs and 1518 -> 87 µs (`:static`: 16-20 and 122-142 µs); 17-105x across the cases measured. Quiet hosts are unchanged.
Benchmark Results (Julia v1)Time benchmarks
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On a loaded host,
foreach_range(...; schedule = :steal)took a millisecond or more per launch, while:staticstayed in the microseconds: on a loaded 96-core Ampere Altra, axpy 1e5 at 48 threads took 517 µs with:stealvs 15 µs with:static. KernelAbstractions' CPU back-end (JuliaGPU/KernelAbstractions.jl#906) switches to:stealafter a launch of 50 µs or more, so once a launch was slow, every following one stayed slow. Found while working on #83.Cause
steal_allowedwaits up toRANGE_AWAKE_NS(2 µs) when an unstarted block's owner is awake but hasn't claimed its command. The first-seen time was local to the call, so every call started a new 2 µs wait: for each victim in turn, on every pass, in every block the dispatcher rescued, and also for blocks other thieves had already emptied, since an emptied block still looks unstarted. Under load most owners are in exactly that state. Traced on a 9950X with 16 threads and a busy-loop process on each of their CPUs, the caller spent 743 µs of a 773 µs launch insteal_allowed, over ~394 waits per launch, 96% of them on empty blocks. At 32 threads the waits exceed the startup deadline (1.5 ms), so every launch ended at the deadline. Static mode rescues the same chunks but waits once for all of them.Fix
RangeBlock.seen, on the line of its range word), reset byinit_blocks!. It records the first time anyone saw the owner awake with its command pending in this region; thieves andrescue_rangesshare it throughowner_due. A timestamp later than the observer's ownnowisn't due yet.steal_allowedno longer waits: it returnsYES(started, empty or due),SOON(owner awake, not yet due) orNO(protected). Participants rescan while any block isSOON;PROTECTED_PASSESstill applies toNO.RANGE_AWAKE_NSafter it was first seen so; sleeping owners, claimed commands and the caller's block stay protected until the deadline. One deliberate change: the timestamp now survives intervening observations of the owner asleep, so the wait can't start over within a region.rescue_rangesforStaticRangescompiles to the same LLVM IR as on main.Tests cover the clock on standalone blocks (
SOONturningYESwithout a reset, an observer reading before another's timestamp, future timestamps, protection of asleep and claimed owners, fresh clocks per region; sequential observations, not real CAS contention), plus the callers on a pool whose workers never run: the steal loop rescans while a block isSOONand then takes it, andrescue_rangesstarts and reuses the shared clock. Both fail on main's behaviour.Results
Median µs per launch, main / this PR. "Loaded" = one busy-loop process per used CPU, which ThreadTeams can't see (the pool isn't marked oversubscribed).
Ryzen 9950X, Julia 1.12, 3 processes per tree, alternating:
Ryzen 5950X, 16 threads, loaded, Julia 1.11, 4 repetitions;
kais KernelAbstractions#906's adaptive policy on top of ThreadTeams::stealkakalaunches using stealTwo direct
:stealcases there got slower (axpy 1e6 1.4×, axpy 1e7 1.15×); main happened to be fast on those, andkaon the same loops isn't affected.Quiet 2× Xeon Gold 6430 (32/64 threads,
:cores/:numa): within ±5% everywhere, ~2–5% cost at 64 threads:cores. NUMA SAXPY (2 × 1 GiB, fill kernel then SAXPY, KA policy) at 32 threads:numa: 288 → 347 GB/s median (pages local to their owner 81.5% → 87.9%;:static399), since fewer blocks of the cold fill are rescued or stolen.Tests: full suite on Julia 1.12 (
-t 16,1) and 1.10 (-t 8);foreach_range5/5 undertaskset -c 0-3 julia -t 4,1.