Skip to content

Latest commit

 

History

History
571 lines (451 loc) · 26.5 KB

benchmarks.md

File metadata and controls

571 lines (451 loc) · 26.5 KB

Benchmark report

The latest working-tree measurements are in the 2.16.x competitor round below. Historical product, component, and in-process graph measurements remain after it so changes in methodology stay visible rather than silently replacing old numbers.

2.16.x working-tree competitor round

Measured 2026-09-17 on the same Windows workstation as the 2.1.0 gate. The committed tag at measurement was 720afcd (2.16.0); the release binary and repository_suite harness included the unpublished session-Arc, census, and parse-batch speed work on that tree. Fat-LTO release, CARGO_INCREMENTAL=0.

These tools are not equivalent products. Wall-clock is reported so the cost of each contract is visible. ripgrep and tokei win at listing and counting. Weavatrix is the only row that emits a typed evidence graph with provenance. Serena, Aider RepoMap, CodeQL, GitNexus, and Gitingest were not timed: they are not a small local CLI on this machine, and installing them would not produce a comparable artifact.

In-process engine gate

Same harness as 2.0.0–2.1.0: three cold Weavatrix::open samples and 1,000 hot graph_stats calls in one process.

cargo bench --locked --all-features --bench repository_suite -- .
Measurement 2.1.0 (2026-08-03) This working tree Read
Repository view 212 files / 1,693 nodes / 8,203 edges 454 files / 4,805 nodes / 21,621 edges Corpus and extractors grew (n8n, Dify, Agent, Mermaid, Web3).
Cold build median 78.55 ms 223.6 ms Absolute time tracks size. Per node both sit at ~46 µs.
graph_stats, 1,000 hot calls 656 µs/call 102 µs/call Census cache: ~6.4× faster on a 2.8× larger graph.
Unchanged refresh 7.09 ms/call 3.38 ms/call Cheap Arc scan reuse.
Literal search (TODO) 7.20 ms/call 14.83 ms/call Larger tree; not the target of this pass.

Cold open is not claimed faster than 2.1.0. The repo more than doubled, and the new domain extractors run on that tree. The speed work removes the session-clone tax and the per-call graph walk.

The in-process graph is one file larger than the CLI graph_stats sample (454 vs 453 files) because benchmark-results/*.json landed before repository_suite ran.

Raw evidence:

  • benchmark-results/rust-engine-2.16.1-speed.json (SHA-256 16ABD8A696DC13C610E3572BD75D62BD00B7C70E44F6A0F78383E689B8519D05).

Process-level wall clock on this tree

Three samples after one discarded npx warmup. Medians. target/ and .git/ excluded where the tool has a flag.

Tool Version Median What the process produced
ripgrep --files 15.1.0 41.3 ms 458 paths
tokei . -e target 15.0.0 95.6 ms 455 files; Rust 376 files / 50,767 code lines
ripgrep -n "fn " 15.1.0 107.2 ms 2,367 matching lines
git ls-files --cached 2.54.0 148.3 ms 466 tracked names
weavatrix-rust Weavatrix::open working tree 223.6 ms In-process typed graph (harness above)
weavatrix-rust tool graph_stats 2.16.0 CLI 668.7 ms Fresh process + compact JSON; 4,804 nodes / 21,620 edges
weavatrix-rust analyze 2.16.0 CLI 724.9 ms Full snapshot JSON on stdout
ast-grep fn $NAME($$$) 0.45.3 via npx 814.7 ms 5 structural matches (strict pattern, not the rg count)
Repomix --quiet 1.18.0 3,284 ms 3.8 MB packed XML, then deleted
scc / Gitingest / Serena / CodeQL — — Not installed; skipped

A second consecutive graph_stats CLI process was 705 ms. That is not in-process reuse; each invocation rebuilds. The 102 µs figure is the in-engine hot path after open.

Raw evidence:

  • benchmark-results/competitor-round-2026-09-17.json (SHA-256 DAF8B8547DED9B0EB238B5A586AAB8CFD928A745DC834B52810D5ED3DDEB3079).

Current release evidence from 2026-08-03 remains below for the published 2.1.0 gate. Historical product, component, and in-process graph measurements follow so methodology drift stays visible.

2.1.0 minor release basic and short-load gate

The minor release was measured at 2b4ac4ba4eb52df1ae50ec134107fdf0a1e20898. It repeats the bounded three-cold-build and 1,000-hot-call harness after adding map_stacktrace, select_tests, build_graph, token budgets, and dependency-injection type references through weavatrix-parse 0.3.0. No long soak or fresh competitor run is part of this gate; the historical competitor tables below still apply to the unchanged components.

Measurement Result
Repository view 212 files / 1,693 nodes / 8,203 edges
Cold build 75.85 ms min / 78.55 ms median / 367.77 ms max
graph_stats, 1,000 hot calls 0.656 ms/call
Unchanged refresh, 3 calls 7.091 ms/call
Literal search, 5 calls 7.197 ms/call

Read against 2.0.2 (192 files / 1,531 nodes / 7,287 edges): the repository itself grew about 13% in edges with the five new capability modules, the hot-call and unchanged-refresh medians are unchanged (0.656 vs 0.661 ms and 7.09 vs 7.44 ms), and the cold-build median moved from 73.21 ms to 78.55 ms - proportional to the larger tree, not a per-unit regression. The cold-build maximum caught one straggler sample on a freshly settled machine and is reported rather than discarded. A first measurement taken while a fat-LTO build ran concurrently produced 4-12x worse numbers and was discarded as machine contention, consistent with the methodology note below.

The same code passed fmt, strict Clippy in all-features and no-default-features modes, 39 all-feature integration binaries, the no-default-feature suite, rustdoc, and the repository's own architecture contract at the 300-line file and 100-line function budgets.

Raw evidence:

  • benchmark-results/rust-engine-2.1.0-basic.json (SHA-256 C0A7877EF8A5C7F22FA19B7C582DF32293D4F4EDB11E11964E6B87352B9AEECA).

2.0.2 duplicate-integrity patch basic and short-load gate

The patch release was measured at 19b37608a01af8aaaec1d3fca7784f1b5c2d2da4. It repeats the bounded three-cold-build and 1,000-hot-call harness after fixing filtered clone-family integrity. No long soak or competitor run is part of this patch gate.

Measurement Result
Repository view 192 files / 1,531 nodes / 7,287 edges
Cold build 62.80 ms min / 73.21 ms median / 91.29 ms max
graph_stats, 1,000 hot calls 0.661 ms/call
Unchanged refresh, 3 calls 7.436 ms/call
Literal search, 5 calls 5.504 ms/call
Measured line coverage 87.71%
Measured function / region coverage 80.57% / 85.30%

The same code passed 122 all-feature tests, 85 no-default-feature tests, strict Clippy in both feature modes, rustdoc, dependency audit, package dry run, the enforceable architecture contract, and a regression against the Semantic repository. With tests excluded, every returned clone-family member is visible and every family pair identifier resolves to a returned pair.

Raw evidence:

  • benchmark-results/rust-engine-2.0.2-basic.json (SHA-256 E49C5DCAD32B2C2502AE09E67087238B261435804851BBC4DD13C701426FF77A).

2.0.1 patch engine basic and short-load gate

The patch release was measured at 2ee6052bed059a086763d5728a4026ff5f9a9319. It uses the same bounded three-cold-build and 1,000-hot-call harness as 2.0.0; no long soak or competitor run is part of this patch gate.

Measurement Result
Repository view 187 files / 1,516 nodes / 7,223 edges
Cold build 127.85 ms min / 168.35 ms median / 170.81 ms max
graph_stats, 1,000 hot calls 1.134 ms/call
Unchanged refresh, 3 calls 11.620 ms/call
Literal search, 5 calls 7.542 ms/call
Measured line coverage 87.69%
Measured function / region coverage 80.47% / 85.27%

The same revision passed the enforceable architecture contract and the production self-health gates: zero architecture violations, dependency or runtime findings, cycles, dead-code candidates, and duplicate families at the 50-token / 92% threshold.

Raw evidence:

  • benchmark-results/rust-engine-2.0.1-basic.json (SHA-256 C25D39627049683F41B8E71EA6DBD04563AD4807AB0D70C45D5E05F5F60DE458).

2.0.0 engine basic and short-load gate

This bounded release check analyzes the committed weavatrix-rust repository at 4b4773ff1c3a8b25ff4319451115cab35c1b4ad8 with the fat-LTO release profile. It intentionally replaces a long soak with three cold builds and a short 1,000-call hot loop:

cargo bench --locked --all-features --bench repository_suite -- .
Measurement Result
Repository view 185 files / 1,503 nodes / 7,067 edges
Cold build 118.13 ms min / 124.65 ms median / 137.06 ms max
graph_stats, 1,000 hot calls 1.373 ms/call
Unchanged refresh, 3 calls 14.919 ms/call
Literal search, 5 calls 9.867 ms/call
Measured line coverage 87.81%
Measured function / region coverage 80.64% / 85.43%

The same revision passed the native architecture contract, repository audit, dead-code review, and production duplicate scan: zero violations, runtime cycles, audit findings, dead-code candidates, or clone families at the 50-token / 92% threshold. The benchmark is a basic regression and short-load gate, not a claim about long-running throughput or every repository shape.

Raw evidence:

  • benchmark-results/rust-engine-2.0.0-basic.json (SHA-256 50769A645BFA8668E0643D593215F7E51CF9DD5582B6FC3CE3803E8E28D2B260).

Historical installed npm MCP boundary: npm 1.0.0 / Rust engine 1.0.1 vs JavaScript 0.3.15

This is the release gate. Both local source trees were packed, installed into isolated npm roots, and invoked through their installed package bins. Each of the four tools used three paired fresh processes. Pair order alternated between Rust-first and JavaScript-first; every process received an empty HOME, graph home, and XDG/AppData cache. Package/native/initialize versions, MCP protocol responses, advertised tools, successful results, and process cleanup are mandatory invariants.

Tool Rust cold median JavaScript cold median Paired speedup
graph_stats 249.87 ms 7,561.94 ms 30.73x
list_endpoints 321.07 ms 7,400.17 ms 26.15x
find_dead_code 310.45 ms 9,298.27 ms 31.81x
run_audit 378.47 ms 11,583.83 ms 35.62x

The median of all 12 paired cold-boundary ratios is 30.34x, above the 24x release threshold, and every selected tool is faster than JavaScript. Warm-call medians are 3.17 ms for Rust and 494.83 ms for JavaScript, a 156.10x speedup and above the independent 30x warm-call threshold.

The boundary is spawn-to-first-successful-tool-response. It includes the npm launcher, MCP initialization, catalog listing, graph construction, and the requested tool. It excludes npm installation, which is recorded separately. The Windows PowerShell RSS sampler could not attach before the short Rust processes exited, so Rust RSS is explicitly unavailable; it is not recorded as zero. JavaScript median peak process-tree RSS was 782,077,952 bytes. Cleanup uses PID plus process creation time so unrelated processes that later reuse a PID cannot be mistaken for leaked children.

Raw evidence:

  • benchmark-results/npm-mcp-boundary-mcport-0.3.0-vs-js-0.3.15.json (SHA-256 8224CACEA4F10B6B09BB525FCC1E4FFA0A7AF1292CD1C4EC63515A2CF99D7F5A);
  • harness: scripts/benchmark-npm-mcp.mjs;
  • Rust package: weavatrix 1.0.0, native weavatrix-rust 1.0.1 Windows GNU fat-LTO binary;
  • JavaScript package: weavatrix-js 0.3.15 on Node v24.15.0.

Immutable graph parity against JavaScript 0.3.14

Both engines analyzed the same clean 502-file checkout at 66ddec80f29ec53bd501f89c5516402b7b80d474. Paths, symbols, and relations were normalized before comparison.

Relation Rust JavaScript Common Result
imports 2,320 1,126 1,126 100% JS coverage
method 63 4 4 100% JS coverage
re_exports 80 75 75 100% JS coverage
calls 3,403 2,323 2,024 exact 299 owner-only; 0 missing; 0 wrong

All 299 non-exact JavaScript call edges were audited. Rust resolved the same source-line target and additionally attached the containing symbol, so they are owner-model differences rather than missing or incorrect targets. Rust-only edges are retained as evidence but are not automatically called correct merely because there are more of them.

Raw evidence:

  • benchmark-results/graph-parity-rust-1.0.0-vs-js-0.3.14.json (SHA-256 BF7393AAA83764A432DADD522C86FDDA03646F8A2006DBB08F8B325EAEF492DA);
  • benchmark-results/call-audit-rust-1.0.0-vs-js-0.3.14.json (SHA-256 B4164FF769B79986282E311390BB7E3C34737FEE68992AE62CBA23D3421D382D).

Historical in-process graph builds (2026-07-27)

Both engines were measured back-to-back on the same checkouts; every row compares identical Git revisions (verified per artifact). Both harnesses take the median of three in-process cold builds with a warm filesystem cache (frontend JS: a single sample; its first three-sample attempt exceeded a 30-minute process budget). The Rust timing includes endpoint extraction; the JavaScript timing excludes it, which biases every row in JavaScript's favor.

Repository Revision Rust cold build JS cold build Speedup Rust/JS endpoints
frontend 8b39a8ad 760.4 ms 14,284.1 ms 18.8x 1 / 0
analytics 38c32aba 222.9 ms 3,151.6 ms 14.1x 73 / 67
automation 8ab859ac 434.9 ms 23,110.7 ms 53.1x 0 / 0
bgp-speaker b1121fd6 29.7 ms 533.3 ms 18.0x 0 / 0
warroom 6a887f0e 261.9 ms 2,988.3 ms 11.4x 9 / 8
AI-Dev-System 81e7e9a1 155.4 ms 3,002.0 ms 19.3x 20 / 18
grpc-server a9376fd7 11.8 ms 2,491.3 ms 210.8x 0 / 0
radiochron 6093530c 72.9 ms 1,126.0 ms 15.5x 0 / 0

Geometric mean over these eight repositories: 25.6x, and Rust is faster on every one. controller-rest-api left the speedup table because its checkout moved to a different revision after the JavaScript side was measured; it remains in the corpus artifact.

Two things must be said about this round rather than hidden. It was measured while other applications were saturating the machine, so the ratios are a floor, not a best case. And the Rust graphs are now larger than in earlier rounds - scope-aware reference resolution and the module resolver add real edges (analytics 13,317 -> 15,266, frontend 35,472 -> 38,871) - so the engine does more work per build than the round that reported higher ratios. The two smallest repositories still cost JavaScript 0.5-2.5 s, so the ratio stays largest exactly where an MCP server restarts most often.

Endpoint counts describe different but compatible evidence models and are not treated as a universal precision score - controller-rest-api is the clearest example, where the two engines count route surfaces differently in both directions.

Historical multi-repository graph comparison (2026-07-27)

Both graphs were normalized to shared shapes (file paths, file#symbol pairs, source -> target relation strings) and compared per category on the same checkouts; the raw sweep is committed as benchmark-results/accuracy-sweep-vs-js-0.3.14.txt. "Coverage" below is the share of the JavaScript engine's evidence that the Rust engine also finds.

  • Symbols/contains: Rust covers 90-100% of the JavaScript symbols on every repository and is a strict or near superset on seven of eight (frontend 99.6%, grpc-server and bgp-speaker 100%, warroom 93.2% while finding more total). JavaScript finds only 50-92% of Rust's symbols.
  • Imports: Rust covers 98-100% of JavaScript's import evidence on Python, Go, Java, and Rust repositories while finding up to 2.6x more (workspace-aware crate::/super::/module-path resolution, go-module paths, classpath imports, CommonJS require). On JavaScript/TypeScript home turf the engines overlap 55-78% in both directions: the JavaScript engine still resolves re-export chains and directory mains that the Rust lexical tier does not.
  • Inheritance: Python is exact parity (430/430 on automation). JavaScript reports near-zero inheritance on TypeScript and Rust code that Rust extracts (frontend 117 vs 1).
  • Calls: mixed by language - Rust leads on Go, Rust, Python, and mixed repositories; the Java engines disagree in both directions and neither is a superset. Call-name resolution remains name-based in both engines.

Endpoint ground truth (Express)

The generated swagger of controller-rest-api (463 path+method operations) is the one corpus repository with an authoritative route list:

Engine Reported Full-path matches Precision Recall
Rust 1,243 396 31.9% 85.5%
JavaScript 0.3.14 968 396 40.9% 85.5%

The Rust engine resolves Express mount chains across files - including middleware-separated arguments, destructured requires, and multi-line route declarations - and matches the JavaScript engine exactly: both find the same 396 of 463 swagger operations. All 67 operations missed by both engines are edgeAnalytics/* paths merged into the swagger from a separate proxied service; their routes do not exist in this repository's source, so 85.5% is the static-analysis ceiling here. Rust additionally keeps the router-local declarations as explicit evidence, which lowers its precision figure without losing any real endpoint.

End-to-end graph competitors

madge 8.x and dependency-cruiser 17.x (no-config, JSON output) build import graphs only - no symbols, calls, endpoints, or domain evidence - yet on the same checkouts:

Repository Weavatrix Rust (full graph) madge (imports only) dependency-cruiser (imports only)
analytics 105.1 ms 21,298 ms (203x) 45,831 ms (436x)
frontend 402.6 ms 18,960 ms (47x) aborted after 707,116 ms (>1,750x)

dependency-cruiser did not finish frontend: the process aborted (exit 134, out of memory) after roughly 11.8 minutes. The abort is reported as measured, not extrapolated.

Import edges compared with madge (analytics)

Speed is only half the comparison; the edges themselves were diffed against madge's dependency tree on the same checkout, counting only file-to-file imports so the two contracts are comparable.

Count
edges both engines agree on 1,389
madge only 9
Weavatrix only 31

Agreement is 99.4% of madge's tree. Every divergence was traced to source:

  • 7 of the 9 madge-only edges are re-exports (export { x } from './y'). Weavatrix records those as re_exports edges, which this comparison deliberately excluded, so they are a representation difference rather than a miss - and the distinction is what lets an architecture rule separate a forwarding barrel from a real dependency.
  • 2 are JSON imports (import map from '../version.map.json'). Weavatrix does not scan .json as source, so the target file has no node to point at. This is a genuine gap.
  • The Weavatrix-only edges are re-export chain expansions: where madge stops at the barrel, Weavatrix additionally records the module that actually defines the imported symbol, which is what a blast-radius question needs.

Phase profile is exposed via WEAVATRIX_PHASE_TIMING=1; on frontend the cold build splits across parse/integrate/resolve/snapshot at roughly 71/82/66/46 ms after pre-sized graph capacity and hash-indexed symbol lookups. The remaining lever is node/edge validation and canonicalization inside the weavatrix-graph crate itself.

Component competitors

Each row keeps an equivalent contract where possible. A narrower competitor contract is called out explicitly.

Repository scan

Contract Weavatrix Scan Competitor Result
6k raw parallel walk 10.2 ms jwalk 10.8 ms 5.6% less time
6k ignore-aware manifest 20.7 ms ignore 37.4 ms 44.7% less time
1m raw parallel walk 264.7 ms jwalk 313.1 ms 15.5% less time
1m compact manifest 1,019.4 ms ignore 2,106.7 ms 51.6% less time

The manifest contract includes deterministic ordering, ignore semantics, skip evidence, hashes, and incremental inputs. Raw walking is measured separately.

Graph build path (2026-07-28)

Edge canonicalization was rewritten to group edges with a counting sort and order each group by sorting a permutation of indices, so an edge is moved once instead of three times and no per-source bucket is allocated. Topology construction now reserves from the iterator size hint. The canonical order and every validation error are unchanged and pinned by a test that compares the canonicalizing path against the already-sorted path.

Paired A/B, same binary and corpus, only the two crate files swapped, six runs each on the frontend repository (15.7k nodes, 39.2k edges), medians:

Phase Before After Change
snapshot (contains GraphBuilder::build) 51.6 ms 38.9 ms -25%
integrate 67.5 ms 56.5 ms -16%
parse (unaffected by this change) 86.2 ms 81.0 ms -6%

The parse row is the noise floor of this machine: the change cannot affect parsing, so only differences comfortably above 6% are attributable. The crate's own builder bench agrees on the size of the win: median build of 10k nodes and 30k edges moved from 30.8 ms to 25.5 ms.

A binary search over the sorted node list was tried in place of the hash position index and measured worse (40.9 ms), so the hash index stays and the measurement is recorded next to it.

The improvement lives in weavatrix-graph 0.6.1 and reaches this engine when that version is published: weavatrix-graph is a released external crate by design, and a boundary test rejects a path dependency on it, so the A/B above was taken by swapping the two crate files in place rather than by wiring a local override into this manifest.

Competitor rows below and the end-to-end table above were measured on a quiet machine. The corpus re-run after this change landed on a heavily contended machine (repeat medians moved by 3-5x on unrelated repositories, and the petgraph competitor itself swung from 19 ms to 52 ms between runs), so those numbers were discarded rather than published; the end-to-end table therefore still reflects the previous measurement round and predates the scope-aware reference resolution that adds real edges.

Graph

200k nodes and 1m edges:

Contract Weavatrix Graph petgraph Result
validated dual CSR build 14.391 ms 78.735 ms 5.47x
BFS 13.026 ms 49.606 ms 3.81x
SCC 92.304 ms 316.340 ms 3.43x
Dijkstra 69.190 ms 102.050 ms 1.47x
rich canonical snapshot 623.691 ms 644.840 ms 1.03x

For a narrower pre-sorted topology-only build, petgraph measured 12.895 ms versus 14.391 ms. Weavatrix keeps validation and reverse CSR in that row; the remaining 11.6% difference is recorded rather than hidden.

Text search

Contract Weavatrix Search ripgrep Result
Windows 20k files 292.5 ms 399.7 ms 26.8% less time
Windows 200k files 3,181.4 ms 7,507.9 ms 57.6% less time
Ubuntu 200k one-shot 606.2 ms 448.5 ms ripgrep 26.0% less time
200k resident-index query 24.4 ms 4,927.2 ms 202x

Weavatrix wins repeated indexed queries and the measured Windows corpus. It does not claim to beat ripgrep for every one-shot Linux workload.

Vector search

50k vectors, 384 dimensions, target recall at least 99.9%:

Engine Build Query suite Total Recall Memory
Weavatrix Vector 1,792.15 ms 1,176.33 ms 2,968.48 ms 99.975% 90.44 MiB
usearch 17,754.55 ms 1,825.52 ms 19,580.07 ms 100% 144.74 MiB
hnsw_rs 16,645.54 ms 20,004.26 ms 35,801.95 ms 99.988% reported by harness

Clone detection

Corpus Weavatrix Clone Competitor
Rust 20.2 ms jscpd 41.1 ms
Go 47.2 ms jscpd 80.3 ms
Python 175.4 ms jscpd 344.2 ms
JavaScript 233.1 ms jscpd 390.6 ms
TypeScript 220.8 ms jscpd 329.2 ms
Java 104.5 ms jscpd 136.3 ms

The accuracy gate separately covers Type-1/2/3 fixtures and a BigCloneBench oracle; runtime alone is not used as an accuracy claim.

Git and memory

Contract Weavatrix Competitor Result
Git history 1k warm 0.355 ms gix 0.884 ms / git2 1.552 ms 2.49x / 4.37x
Git reopen 2.521 ms gix 3.940 ms / git2 10.483 ms 1.56x / 4.16x
Memory append+load 100k 97.313 ms cqrs-es 144.972 ms 32.9% less time
Memory context 100k/300k 0.193 ms agentic-memory 5.042 ms 26.1x
Validated memory projection 71.076 ms agentic-memory 42.191 ms competitor 1.68x

The last memory row is intentionally retained: the Weavatrix path validates dangling references while the compared from_parts path accepts them.

Cross-repository Git

Five real local repositories (analytics, automation, warroom, AI-Dev-System, radiochron), 1,232 commits and 3,635 tree entries total, 20 iterations, p50. The competitor rows are hand-rolled loops over gix and libgit2 whose results are asserted byte-identical to RepositorySet before timing starts.

Operation weavatrix-git gix loop libgit2 loop
set history, serial 2.099 ms 2.602 ms 1.859 ms
set history, parallel 2.115 ms 1.558 ms 1.938 ms
set reopen + histories 143.334 ms 32.123 ms 38.913 ms
shared-commit correlation 5.793 ms 5.096 ms 5.616 ms
set snapshot manifests 4.887 ms 3.449 ms 3.851 ms

Read this table plainly: on warm cross-repository operations at this corpus size the three engines are within about 35% of each other and no engine wins categorically; on cold set reopen weavatrix-git is currently 4.5x slower than a gix loop. The value of RepositorySet is a single parity-verified API for histories, shared objects, and cross-repository diff - not a speed claim. Reopen cost is recorded here as the next optimization target.

Reproduce from the weavatrix-git repository:

tools/competitor-bench/target/release/weavatrix-git-competitor-bench.exe `
  --cross-repo 1000 20 <repo> <repo> ...

Reproduce

$env:WEAVATRIX_BENCH_OUTPUT = "benchmark-results/rust-real.json"
cargo bench --bench repository_suite -- <same-revision repositories...>

$env:WEAVATRIX_JS = "C:\path\to\weavatrix"  # JavaScript 0.3.14 checkout
node scripts/bench-js-fair.mjs benchmark-results/js-real-0.3.14.json <same repositories...>

Component benchmark commands and generated artifacts live in their independent crate repositories. Wall-clock numbers vary by hardware, filesystem cache, and corpus; ratios should be refreshed before making release claims on a new machine.