Skip to content

[vllm][torch 2.15] DeepEPv2 NCCL GIN capability probe fails on B200 #8808

Description

@github-actions

Summary

Both B200 DeepEP clusters abort while probing NCCL GIN / DeepEPv2 communicator properties, at two different call sites hitting the same probe. The nightly wheel bundles its own NCCL and a version change would plausibly break this query, but the underlying NCCL error code is not surfaced in either log, so the proximate cause is inferred rather than observed. Separately, the test's skip guard greps for the substring 'NCCL GIN' while the library raises a message lacking it, so the intended graceful skip never fires.

Signature

RuntimeError: DeepEPv2 communicator properties query failed; networking capability could not be determined.

Affected job clusters

  • :nvidia: (B200) Distributed
  • :nvidia: (B200) FusedMoE Layer Kernels

Evidence it is torch-nightly specific

Fails on torch-nightly build #89071 and passes on the same-commit baseline #89028 (commit 073f883b56a5).

Representative jobs

Suggested routing

pytorch/pytorch (agent confidence: classification medium, new-failure medium)


Filed automatically by the vLLM torch-nightly triage workflow. The root cause above was produced by an automated analysis of the Buildkite log tail and has not been human-verified.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    vllm-torch-nightlyvLLM CI regression attributed to a PyTorch nightly

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions