Parallelize bounded HNSW graph post-processing

[nvzm123]

Summary

This isolates the bounded post-ingest HNSW optimizations previously developed alongside #2481 and #2594. GPU CAGRA construction remains serialized; writerThreads now also controls bounded CPU work after construction.

Changes

• Bounds effective workers by the requested count, available processors, and available work.
• Materializes eligible device-backed adjacency layers from one temporary host copy when the layer has at least 65,536 nodes and the copy is at most 4 GiB; larger device layers retain the serial path.
• Serializes level-zero neighbors in parallel 64 MiB waves while committing encoded bytes in deterministic node order.
• Preserves serial upper-layer writing and the existing on-disk format.
• Propagates writerThreads through ordinary, binary-quantized, and scalar-quantized accelerated writers.
• Preserves task-order failure suppression, interruption status, executor termination, and native-host cleanup.
• Documents the expanded writerThreads behavior and memory bounds.

Validation

• mvn -o spotless:check install: 352 tests, 0 failures/errors, 30 skipped
• Deep1B 1M one-segment sanity run with writerThreads=1: 1,000,000 documents/vectors and 32,000,000 validated arcs
• Deep1B 1M one-segment sanity run with writerThreads=16: 1,000,000 documents/vectors and 32,000,000 validated arcs
• Deep1B 1M four-segment sanity run with writerThreads=16: 1,000,000 documents/vectors, four leaves, and 32,000,000 validated arcs
• All three retained indexes passed assertion-enabled CheckIndex -level 2 and completed 980 measured queries after 20 warmups
• API reference generation was idempotent; git diff --check passed

The Deep1B runs are functional sanity checks, not controlled performance measurements.

Integration note

This branch is based directly on main and contains only the post-ingest optimization. If #2476 lands first, this change needs a semantic rebase so that #2476's newer ownership, validation, and upper-layer fixes are retained.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.