Skip to content

[Feature]: DRA-aware gpu-allocatable readiness criterion in lifecycle-manager for GPUCluster mode #1899

Description

@tanishagoyal2

Prerequisites

  • I searched existing issues

Code of Conduct

  • I agree to follow NVSentinel's Code of Conduct

Feature Summary

Make the lifecycle-manager's default gpu-allocatable node readiness criterion recognize GPU nodes on GPU Operator GPUCluster (DRA) clusters, so ValidationRequests proceed there without a per-cluster override of the ValidationConfiguration.

Problem/Use Case

The default gpu-allocatable criterion requires nvidia.com/gpu in node.status.allocatable with a count above zero. In GPU Operator 26.7's GPUCluster mode there is no device plugin, GPUs are exposed through DRA ResourceSlices, and that resource is always zero. Every ValidationRequest, including the automatic new-node validation, stays Pending forever with Node not ready, waiting criterion=gpu-allocatable, and no dcgm-diag-test Job is created.

Proposed Solution

The readiness CEL only sees the node object, update the CEL expression to check for resourceSlice presence to confirm the GPU availability

Component

Health Monitor

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions