Conversation
imotov
left a comment
imotov
left a comment
There was a problem hiding this comment.
Looks like a good start to me. However, I'd like to be able to run it to check a few things, and I think either the environment is missing some dependencies or the README is skipping over some important steps. It also seems that since these dependencies aren't installed in our cuVS environment, all integration tests are being skipped in CI.
| The Lucene backend is opt-in because its runtime is not provisioned by the | ||
| ordinary cuVS Bench installation. Provisioning the required custom PyLucene | ||
| build is currently external to cuVS Bench. Every algorithm requires PyLucene | ||
| 10.2.0 and JDK 22. CAGRA additionally requires matching `cuvs-java` and thin |
There was a problem hiding this comment.
Do we need to install PyLucene manually for that to work?
|
|
||
| def _java_vector(self, vector: np.ndarray) -> Any: | ||
| return self.lucene.JArray("float")( | ||
| tuple(float(value) for value in vector) |
There was a problem hiding this comment.
I think this might skew the benchmark due to tuple conversion and cost of Python -> Java marshaling for all these floats.
| ) | ||
| hits = [] | ||
| for score_doc in searcher.search(query, k).scoreDocs: | ||
| stored_id = stored_fields.document(score_doc.doc).get(_ID_FIELD) |
There was a problem hiding this comment.
I don't think this is fair treatment for Lucene. Stored fields are pretty expensive option here. It might be better to at least replace it with doc values.
| for prefix in _python_prefixes(): | ||
| yield ( | ||
| prefix / "lib", | ||
| prefix / "targets" / "x86_64-linux" / "lib", |
…e_branch # Conflicts: # dependencies.yaml # python/cuvs_bench/pyproject.toml
imotov
added
improvement
|
/ok to test 196f888 |
2 participants
Summary
lucenecuVS Bench backend for Lucene CPU HNSW and cuvs-lucene CAGRA build/search.--force.This draft isolates the non-Java integration work previously explored in #2385 and #2475. The external PyLucene and Maven artifacts are runtime prerequisites and are not yet provisioned automatically by the cuVS Bench package.
The live
lucene_e2ecases are manual-only in current CI. Automated provisioning and artifact handoff are tracked in #2635.Validation
Performance validation
The data-path changes were compared against the preceding commit on the same EC2 A10G, using the same PyLucene 10.2 runtime, JDK, native environment, and exact cuvs-java and cuvs-lucene JAR hashes. Each side built and searched an isolated Deep1B 1M-vector, 96-dimensional index over 10,000 queries.
top_kRecall remained comparable in every paired run. Result materialization decreased by approximately 22-61%, while the separately measured JVM search phase remained broadly unchanged. These are single paired measurements and should be treated as directional rather than a statistical performance claim.