Skip to content

Repair quantized Lucene format adapters - #2627

Draft
nvzm123 wants to merge 3 commits into
NVIDIA:mainfrom
nvzm123:zackm_cuvs_issue_2615
Draft

nvzm123 wants to merge 3 commits into
NVIDIA:mainfrom
nvzm123:zackm_cuvs_issue_2615

Conversation

@nvzm123

@nvzm123 nvzm123 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Important

Depends on #2625. This draft also tightens the CPU-fallback dimension contract: without cuVS, the quantized formats now report and enforce Lucene's current 1024-dimension limit instead of advertising the GPU path's 4096 limit.

Closes #2615.

Caches LuceneProvider instances by format version, adds correctly typed quantized HNSW factories, and preserves the released method descriptors as deprecated compatibility entry points with actionable failures. Reflection now preserves IOException, runtime exceptions, and errors.

The binary CPU fallback records its Lucene 10.2 flat-vector layout so it can be read correctly, while the released GPU-written Lucene 9.9 layout remains unchanged. Tests cover concurrent provider construction, legacy descriptors, parameter order, compound segments, CPU dimension boundaries, and live binary and scalar GPU indexing/search at 4096 dimensions.

The Fern API generator now handles leading annotations and multiline inline Javadocs, and the generated pages are refreshed.

Testing

  • python3 -m unittest fern.scripts.test_generate_api_reference (2 passed)
  • python3 fern/scripts/generate_api_reference.py --quiet (second run produced no changes)
  • bash -n fern/build_docs.sh
  • mvn -Dtest=TestBackCompat,TestQuantizedVectorsCpuFallback test (14 passed)
  • mvn -Dtest=TestLuceneProviderConcurrency test (1 passed)
  • mvn spotless:check verify (351 run; 321 passed, 30 skipped)

The complete GPU run still emits existing cuVS graph-clamping, dataset-dimension, GDS fallback, Java native-access, Vector API, and Maven Javadoc-plugin warnings.

@copy-pr-bot

copy-pr-bot Bot commented Sep 16, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Lucene: isolate versioned providers and repair quantized format factories

1 participant