bench: add experimental node:bench module

[jasnell]

A new node:bench module, modeled closely after the node:test architecture.

import { bench, suite } from 'node:bench';

suite('URL', () => {
  const input = 'https://example.com/a?b=c';

  bench('construct', {
    samples: 30,
    params: { input: 'short' },
  }, (b) => {
    const operations = 10_000;

    b.start();
    for (let i = 0; i < operations; i++) {
      new URL(input);
    }
    b.end(operations);
  });
});

node --bench benchmark.mjs

The core functionality is here but there are still improvements that are needed. Specifically, there's a fair amount of overhead interjected by the harness itself that can impact the results. The focus was on minimal functionality, not absolutely optimal functionality. But this gives a solid starting point for improvements.

[nodejs-github-bot]

Review requested:

• @nodejs/loaders
• @nodejs/performance
• @nodejs/startup
• @nodejs/tsc

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/76609/

[codecov]

Codecov Report

❌ Patch coverage is 92.38845% with 174 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.09%. Comparing base (05a8e91) to head (9176490).
⚠️ Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/bench_runner/cli.js	88.18%	72 Missing and 3 partials ⚠️
lib/internal/bench_runner/harness.js	93.90%	42 Missing and 1 partial ⚠️
lib/internal/bench_runner/benchmark.js	93.91%	25 Missing ⚠️
lib/internal/bench_runner/reporter/spec.js	92.61%	10 Missing and 1 partial ⚠️
src/node_options.cc	75.00%	7 Missing and 3 partials ⚠️
lib/internal/bench_runner/reporter/json.js	92.18%	4 Missing and 1 partial ⚠️
src/histogram.cc	93.61%	0 Missing and 3 partials ⚠️
src/node_options-inl.h	85.71%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #65606      +/-   ##
==========================================
+ Coverage   90.06%   90.09%   +0.02%     
==========================================
  Files         751      761      +10     
  Lines      254919   257177    +2258     
  Branches    48124    48613     +489     
==========================================
+ Hits       229603   231697    +2094     
- Misses      16492    16649     +157     
- Partials     8824     8831       +7     

Files with missing lines	Coverage Δ
lib/bench.js	100.00% <100.00%> (ø)
lib/bench/reporters.js	100.00% <100.00%> (ø)
lib/internal/bench_runner/benchmarks_stream.js	100.00% <100.00%> (ø)
lib/internal/bench_runner/runner.js	100.00% <100.00%> (ø)
lib/internal/bootstrap/realm.js	96.31% <100.00%> (+0.01%)	⬆️
lib/internal/histogram.js	96.22% <100.00%> (+0.11%)	⬆️
lib/internal/main/bench_runner.js	100.00% <100.00%> (ø)
src/histogram.h	72.72% <ø> (ø)
src/node.cc	76.61% <100.00%> (+0.06%)	⬆️
src/node_config_file.cc	83.68% <ø> (ø)
... and 9 more

... and 44 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[RafaelGSS]

How is it different from what we (@H4ad) proposed in #50768?

[jsumners-nr]

@RafaelGSS probably nothing except that hopefully now it can progress. My question would be: why not vendor https://www.npmjs.com/package/bench-node?

[jasnell]

... why not vendor ...

I initially considered vendoring one of the existing packages and opted against it in favor of a light weight, minimal, no-new-dependency approach that a more feature-rich tool like bench-node can build/iterate on. I don't consider these either/or options.

Node.js' own benchmarks demonstrate that the full set of features provided by bench-node aren't necessary for a minimal bench suite. node:bench is intended to meet the low end, minimal requirement. These don't need to compete.

	node:bench	bench-node
Role	Minimal built-in benchmark kernel	Full-featured userland benchmark tool
Core features	Samples, warmup, suites, hooks, raw results, reporters, CLI isolation	Automatic warmup, V8 plugins, workers, DCE checks, baselines
Reporting	Structured event stream with spec and JSON reporters	Text, chart, pretty, HTML, JSON, and CSV
Dependencies	No new runtime dependencies	Piscina (installed but not used??)

We can provide stable measurement and structured result primitives without needing to ship the larger user-facing feature set and these existing tools can continue serving their current users while optionally adopting the built-in foundation. Just like node:test hasn't replaced the more sophisticated test suites like vitest.

That said, this is just a starting point. I've never been one to say "It must be done this specific way!" so all options are on the table. What I want is a built-in node --bench ... mechanism, I don't necessarily care how we get there as long as it works.

[jsumners-nr]

Just like node:test hasn't replaced the more sophisticated test suites like vitest.

If I can't do it with node:test I just don't worry about it.

That said, this is just a starting point. I've never been one to say "It must be done this specific way!" so all options are on the table. What I want is a built-in node --bench ... mechanism, I don't necessarily care how we get there as long as it works.

I'm with you. I just think it's easier to deliver what folks would actually want out of the module by bringing something like bench-node in. Look at the logger attempt. Instead of bringing in pino (yes, Matteo and I were/are both against that) a whole new logger was written while trying to keep the learnings from pino. That failed. But the parts undici is used for seem to be working. I think the same approach can be used.