[vchuravy]

A script for reproducing the measurements of #46 on any machine and comparing machines:

benchmark/sweep/run.sh        # e.g. JULIA_CHANNELS="1.12 1.13" THREADS="2 4 8" REPEAT=3
julia benchmark/sweep/summarize.jl benchmark/sweep/results/*/results.csv

What a run does

• Placement: one hardware thread per physical core (from lscpu, within the shell's affinity), taskset, and each Julia thread pinned to its own CPU (pin.jl, loaded with julia -L); OpenMP gets the same CPUs through OMP_PLACES/OMP_PROC_BIND. Julia runs with -t N,0.
• Benchmarks, each in a process of its own:
  • overhead: benchmark/overhead/overhead.jl, all mechanisms;
  • syncbench: a Julia port of EPCC syncbench's PARALLEL FOR and REDUCTION (team, greedy team, Polyester, @threads :static/:dynamic, OhMyThreads);
  • epcc: the C EPCC syncbench v4.0 at a fixed commit, built with gcc (libgomp) and clang (libomp), each with the default and OMP_WAIT_POLICY=active;
  • wakeup: the loop after a busy gap of 0–10 ms, and for the team with set_spin_time!(0);
  • counters: LinuxPerf.jl counters per loop, skipped if perf_event_paranoid > 2;
  • steal: benchmark/overhead/steal.jl.
• Shared machines: a per-host lock file, and the load average recorded before every run.
• Output: results/<host>-<date>/ with machine.txt (CPU, topology, OS, compilers, versioninfo, governor), a log per run and one results.csv. summarize.jl prints tables in the layout of Benchmark report: loop overhead, wake-up and OpenMP comparison on x86, Apple M1 and four ARM boards #46: a row per host, Julia version and thread count, a column per mechanism.
• Several machines: remote.sh start|status|fetch|stop host… copies the checkout over ssh (including uncommitted and untracked files), runs the sweep in the background and collects the results.

Changes to the existing scripts

• benchmark/record.jl: appends CSV rows when BENCH_CSV is set, does nothing otherwise, so the scripts run as before on their own.
• counters.jl: also measures the greedy team and @threads :dynamic, counts task-clock, context switches and CPU migrations, and takes COUNTERS_EVENTS (LinuxPerf syntax) to choose other events. LinuxPerf only knows the generic perf list events; CPU-specific ones such as the RFO HITM counts of Benchmark report: loop overhead, wake-up and OpenMP comparison on x86, Apple M1 and four ARM boards #46 still need the perf CLI.
• overhead.jl: leaves out KernelAbstractions if it does not load (POCL is not built for every platform); OVERHEAD_SECONDS sets the time per mechanism.

Tested

• Full and QUICK=1 sweeps locally (Zen, Arch, gcc 16 + clang 23).
• remote.sh against a fresh depot (fake ssh), and against a real shared Xeon E5-2603 v4 host (2×6 cores, gcc only, juliaup not on the non-interactive PATH, stale registry).

Fixes found that way:

• Pkg.resolve neither installs nor updates the General registry, so the setup step now does both.
• libgomp 16 does not export omp_init_lock_with_hint, so EPCC v4.0 is built without the lock hints, with v3.1 as a fallback.

Not tested: macOS (no pinning there) and non-x86 hosts.

🤖 Generated with Claude Code

[github-actions]

Benchmark Results (Julia v1)

Time benchmarks

	main	23cd198...	main / 23cd198...
ad/forward tforeach team, 1000	4.63 ± 0.48 μs	7.28 ± 4.3 μs	0.636 ± 0.38
ad/forward tforeach team, 100000	0.148 ± 0.055 ms	0.129 ± 0.039 ms	1.15 ± 0.56
ad/reverse serial loop, 1000	17.1 ± 0.78 μs	17.1 ± 0.69 μs	1 ± 0.061
ad/reverse serial loop, 100000	1.61 ± 0.26 ms	1.62 ± 0.039 ms	0.996 ± 0.16
ad/reverse tforeach team, 1000	0.0327 ± 0.001 ms	0.0319 ± 0.0011 ms	1.02 ± 0.047
ad/reverse tforeach team, 100000	1.46 ± 0.54 ms	1.49 ± 0.37 ms	0.981 ± 0.44
ad/reverse tmapreduce team, 1000	20.2 ± 0.81 μs	22.5 ± 1.3 μs	0.894 ± 0.062
ad/reverse tmapreduce team, 100000	0.483 ± 0.081 ms	0.498 ± 0.045 ms	0.97 ± 0.18
fastbroadcast/9 arguments serial, 100x100x8	0.305 ± 0.02 ms	0.312 ± 0.022 ms	0.978 ± 0.094
fastbroadcast/9 arguments thread, 100x100x8	0.173 ± 0.016 ms	0.179 ± 0.016 ms	0.967 ± 0.13
fastbroadcast/log Base, 1000	7.2 ± 0.11 μs	7.22 ± 0.12 μs	0.998 ± 0.023
fastbroadcast/log Base, 100000	0.722 ± 0.017 ms	0.727 ± 0.017 ms	0.993 ± 0.033
fastbroadcast/log serial, 1000	7.07 ± 0.058 μs	7.07 ± 0.059 μs	1 ± 0.012
fastbroadcast/log serial, 100000	0.711 ± 0.017 ms	0.709 ± 0.016 ms	1 ± 0.033
fastbroadcast/log thread, 1000	4.54 ± 0.62 μs	4.59 ± 0.53 μs	0.989 ± 0.18
fastbroadcast/log thread, 100000	0.381 ± 0.012 ms	0.381 ± 0.0099 ms	0.999 ± 0.041
overhead/@tasks team	0.766 ± 0.065 μs	0.793 ± 0.06 μs	0.966 ± 0.11
overhead/OhMyThreads tforeach	6.23 ± 0.88 μs	6.4 ± 0.92 μs	0.974 ± 0.2
overhead/Threads.@threads :dynamic	4.53 ± 0.53 μs	4.67 ± 0.67 μs	0.97 ± 0.18
overhead/serial loop	26 ± 2 ns	27 ± 3 ns	0.963 ± 0.13
overhead/tforeach greedy team	1.03 ± 0.15 μs	1.03 ± 0.26 μs	1.01 ± 0.29
overhead/tforeach team	0.761 ± 0.066 μs	0.787 ± 0.068 μs	0.967 ± 0.12
overhead/tmapreduce team	0.793 ± 0.067 μs	0.918 ± 0.05 μs	0.864 ± 0.087
trixi/reset du Threads.@threads :static	0.119 ± 0.012 ms	0.122 ± 0.013 ms	0.976 ± 0.14
trixi/reset du serial	1.48 ± 0.11 μs	1.68 ± 0.1 μs	0.885 ± 0.085
trixi/reset du team	2.25 ± 0.13 μs	2.18 ± 0.14 μs	1.03 ± 0.089
trixi/reset du team, 10 in a row	19.5 ± 0.35 μs	19.5 ± 0.37 μs	0.999 ± 0.026
work/serial, 1000	1.31 ± 0.003 μs	1.31 ± 0.003 μs	1 ± 0.0032
work/serial, 100000	0.129 ± 0.0003 ms	0.129 ± 0.00033 ms	1 ± 0.0034
work/tforeach team, 1000	1.48 ± 0.11 μs	1.46 ± 0.088 μs	1.02 ± 0.095
work/tforeach team, 100000	0.067 ± 0.0027 ms	0.0671 ± 0.001 ms	0.999 ± 0.043
time_to_load	0.0902 ± 0.00041 s	0.0928 ± 0.0022 s	0.971 ± 0.023

[maleadt]

AI-generated review:

I used the sweep's approach for cross-host benchmarks of #62–#69 and needed these changes:

• a -t N mode next to -t N,0 (the interactive thread is where the Serial loops when the team is busy, a raw chunk loop, and idle workers that sleep #68 issues show);
• pin.jl pins 1:nthreads(), which under -t N pins the interactive thread and leaves the last default thread unpinned; pin by threadpooltids(:interactive)/(:default);
• pinning the calling thread before the pool starts makes uv_available_parallelism() return 1, so the pool comes up oversubscribed and workers sched_yield every 5 µs; run a loop first (or merge Detect oversubscription from every thread's affinity mask (thread pinning) #19 first);
• macOS: thread counts from hw.perflevel0.physicalcpu, not hw.physicalcpu; and remote.sh stop needs setsid to stop the run, not only the shell;
• a revision column in results.csv, so summarize.jl can compare branches.