perf(monorepo): release planning got up to 9x slower in 7.11.2, and the cost scales with packages times commits

[BryanFRD]

release --dry-run on the large monorepo fixture went from 75.9 ms to 663.2 ms between 7.11.1 and 7.11.2, and sits at 687.1 ms on 7.15.1. Nothing flagged it because the benchmark page is the only place it shows, and nobody reads it.

The measurement

Tracing benchmarks.json in FerrFlow-Cloud across twenty revisions, mono-large (200 packages, 10k commits) is flat at 116 to 131 ms from 7.0.5 through 7.11.0, dips to 75.9 ms on 7.11.1, then jumps to 663.2 ms on 7.11.2 and stays there.

Comparing 7.11.1 against 7.13.0, single-threaded release --dry-run, same runner shape throughout (4 cores, 30 runs, 3 warmups):

Fixture	7.11.1	7.13.0
single package	5.7 ms	7.0 ms	flat
10 packages × 100 commits	6.8 ms	10.6 ms	1.6x
50 × 500	10.4 ms	17.9 ms	1.7x
100 × 1k	20.8 ms	50.9 ms	2.4x
50 × 5k	65.6 ms	198.3 ms	3.0x
200 × 10k	75.9 ms	679.8 ms	9.0x
50 × 300, dependency graph	35.5 ms	22.8 ms	improved

The single-package fixture does not move, so this is not a slower runner. The cost tracks packages multiplied by commits, and the small graph fixture actually got faster.

Where it comes from

7.11.2 contains one functional change. The other two commits are dependency digest bumps.

c8d4a71 fix(monorepo): scope a package's commits to its own paths (#974)

That fix is correct and worth keeping: before it, a package's changelog listed another package's commits. The shape of how it filters is what costs.

commits_for_package (src/monorepo/run/plan.rs:269) resolves the commit range for a package, then hands the whole list to scope_commits_to_package, which filters it commit by commit:

commits
    .into_iter()
    .filter(|c| {
        let files = files_for_commit_cached(repo, id, inputs.commit_files_cache);
        files.is_empty() || pkg.is_touched_by(&files, true)
    })
    .collect()

Two costs stack up, and I have not profiled which dominates:

• get_changed_files_for_commit runs once per distinct commit. The cache is created once per run (src/monorepo/run/mod.rs:214) and shared across packages, so that part is O(commits) rather than O(packages × commits). Still, on a 10k-commit history it is 10k tree diffs in a path that previously did none.
• The filter itself runs per package. On a fixture with no tags yet, every package resolves the full history, so mono-large does 200 × 10k iterations, each taking a Mutex lock on the shared cache and running is_touched_by over that commit's file list. Two million lock acquisitions on a map that is read-only after the first pass.

Direction

The loop is inside out. Walking the history once and attributing each commit to the packages its files touch turns packages × commits into commits plus the number of real attributions, which on these fixtures is far smaller. A path prefix index built from the package list would make the attribution a lookup rather than a scan per package.

The Mutex is also worth removing from the hot path. Once the walk is single pass the map can be built up front and read without locking.

Parallelism will not cover for this. The benchmark's parallel set measures 1.0x on mono-large and 0.8x on the graph fixture, so the work is not currently spread across cores anyway.

Reproducing

The fixtures come from FerrLabs/Fixtures and the harness from FerrLabs/Benchmarks. The competitive set pins --jobs 1, so compare like for like:

ferrflow release --dry-run --jobs 1

against a 200 package, 10k commit fixture with no existing tags, on 7.11.1 and on main.

[BryanFRD]

Reproduced it locally, and the measurement moves the target: the cost is per commit, not per package times commits, so the direction I proposed in this issue is wrong.

Reproduction

Built a fixture of 200 packages and timed release --dry-run --jobs 1 --timing, reading the per-package compute stage. At 800 commits there is no regression at all, 7.11.2 is faster than 7.11.1. Growing the same fixture to 4000 commits, with the package count untouched:

	per-package compute
7.11.1	14 ms
7.11.2	107 ms

7.6x on the same input, from the same two binaries, with only the commit count changed between the two experiments. So #974 is confirmed as the cause, and the scaling axis is commits alone. The "packages times commits" framing in the description is wrong.

Where the time goes

scope_commits_to_package calls get_changed_files_for_commit once per distinct commit, memoised for the run in CommitFilesCache. That is one gix tree diff against the first parent per commit, roughly 18 µs each here, and it did not exist before #974.

Cold and warm runs are identical (85 ms against 84 ms), so the on-disk cache under .git/ferrflow-cache does not cover these diffs. Every invocation redoes all of them.

What I tried, and why it is not the fix

I implemented the inversion this issue proposes: a prefix index over package paths, one bitmap of touched packages per commit, memoised, so the per-package filter becomes an index rather than a path scan. It works and it is asymptotically better, turning packages times commits into commits, and I verified the output is byte-identical and the suite stays green.

It bought 5%: 92 ms to 87 ms, against a 14 ms baseline. Not worth the code, so I have dropped it rather than open a pull request that carries a hundred lines and a new cache for noise.

The reason is arithmetic I should have done before proposing it. The per-pair matching was never the expensive half. The tree diffs are, and they are already O(commits) with a cache in front of them.

What would actually fix it

The diffs have to get cheaper or go away, not be attributed more cleverly. The obvious candidate is replacing N per-commit tree diffs with one traversal, git log --name-only --format=%H over the range, parsed once into the cache. One process and one walk instead of 4000 diffs.

That needs care rather than enthusiasm: merges are the awkward part, since get_changed_files_for_commit diffs against the first parent while git log --name-only shows nothing for a merge by default. Getting that wrong changes which commits land in which package's changelog, which is exactly the correctness #974 was fixing. Worth doing, worth doing carefully, and worth a benchmark on the real Fixtures mono-large rather than my synthetic one.

I have not done it. Say the word if you want it, or leave this open with the analysis and pick it up alongside the wider Fixtures work.