Skip to content

[BUG] NNTest: GTEST_SKIP() in compute_1nn() doesn't skip the test, so cuTile-backend cases fail when cuTile is unavailable #2660

Description

@zbrad

Related: #2552 (introduced the cuTile top-1 NN backend and these tests)

Summary

In cpp/tests/neighbors/distance_nn.cu, the cuTile availability check calls
GTEST_SKIP() from inside the helper compute_1nn():

void compute_1nn()                                    // line 120
{
  ...
  if (backend == cuvs::distance::detail::Top1nnBackend::Cutile &&
      !cuvs::distance::detail::is_top_1_nn_backend_available(
        backend, x.data_handle(), y.data_handle(), m, n, k, metric)) {
    GTEST_SKIP() << "cuTile is not available for this device/input";   // line 132
  }

GTEST_SKIP() only returns from the function it is written in. Every TEST_P
body is this->compute_1nn(); this->compare(); (lines 294-395), so after the
"skip" the test carries on into compare(), which then checks output buffers
that were never written, and fails. The result is a [ SKIPPED ]-style message
immediately followed by a spurious [ FAILED ] for the same test.

This only shows up where the cuTile backend is compiled in but reports
unavailable at runtime, so it is likely invisible on devices where cuTile works.

Environment

  • NVIDIA GB10 (DGX Spark, aarch64), compute capability 12.1, driver 580.173.02
  • CUDA 13.4, CUVS_CUTILE_ENABLED
  • cuvs e0f8a4eb; upstream main 1943ca8d has identical code at the lines above
  • On this machine is_top_1_nn_backend_available(Cutile, ...) is false for every
    cuTile-backend case (why it is unavailable here is not relevant to the bug)

Reproduce

./NEIGHBORS_TEST --gtest_filter='NNTest/NNTest_fp32_fused.test/16'
[ RUN      ] NNTest/NNTest_fp32_fused.test/16
.../distance_nn.cu:132: Skipped
cuTile is not available for this device/input

.../distance_nn.cu:205: Failure
Value of: cuvs::devArrMatch(ref_dist.data_handle(), selected_dist.data_handle(), ...)
  Actual: false (actual=55.788799285888672 != expected=22.509374618530273 @0)
[  FAILED  ] NNTest/NNTest_fp32_fused.test/16

12 cases fail this way: NNTest_fp32_fused.test/16..24 (9), NNTest_fp16_fused.test/0..1
(2) and NNTest_fp32_fused_i64.test/0 (1). That is exactly the number of "cuTile is
not available" messages in the run (12). The mismatching values look like
"wrong results" but are just unwritten outputs: different shapes report the same
value, and no cuTile kernel ran.

Suggested fix

Return early from compare() when the test was skipped:

void compare()
{
  // GTEST_SKIP() in compute_1nn() only returns from that helper, not the test body.
  if (::testing::Test::IsSkipped()) { return; }
  ...

With this change on the same machine, NNTest/* gives 42 passed, 12 skipped,
0 failed (previously 42 passed, 12 failed). We have not tested alternatives such
as returning early in each TEST_P body or moving the availability check into
SetUp() (where GTEST_SKIP() does skip the whole test).


Assisted-by: Claude

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions