Skip to content

fix: resolve node ID collisions made by a memoized resubmit - #17028

Open
Joibel wants to merge 3 commits into
argoproj:mainfrom
Joibel:fix/resubmit-node-id-collision
Open

Joibel wants to merge 3 commits into
argoproj:mainfrom
Joibel:fix/resubmit-node-id-collision

Conversation

@Joibel

@Joibel Joibel commented Sep 20, 2026

Copy link
Copy Markdown
Member
  • Ran make pre-commit -B
  • Signed-off commits with Conventional Commit messages
  • PR title is a conventional commit message (it becomes the release notes entry)
  • Unit or e2e tests cover the change
  • For features: an associated issue and a feature description file (make feature-new)
  • Opened as draft; will mark "Ready for review" once builds are green

Follow-up to #16883, which fixed the controller side of the same problem.

Stacked on #17027, which unbreaks main; the first commit here is that PR. Rebasing once it merges leaves one commit.

Motivation

#16883 made the controller tolerate node ID hash collisions: a name that loses the 32-bit slot to another name is stored under the widened NodeID64 instead, and ResolveNode looks in both places. Memoized resubmit was left alone, and it is the one place that rehashes an entire existing graph.

Resubmit gives the workflow a new name. A node ID is a hash of the node name, which starts with the workflow name, so every ID changes: a collision the original workflow had dissolves, and one it never had can appear. convertNodeID returned newWf.NodeID(name) unconditionally, so under a new collision:

Rare, but a resubmit picks a fresh random name each time, so it is reachable by chance rather than by anything the user did.

Modifications

convertNodeID is replaced by newNodeIDs, which works the whole mapping out before the plan is applied:

  • nodes are assigned in node name order, so the result does not depend on map iteration order, which applyResetPlan has;
  • a name whose 32-bit slot has already gone to another name is widened with NodeID64, matching what the controller does when it creates a node, so ResolveNode finds it afterwards;
  • nodes the plan deletes are mapped too, so a reference to one cannot alias a node that is kept;
  • a name that collides at both widths is refused with an error rather than corrupting the graph, as initializeNode refuses it.

Verification

Two unit tests. TestNewNodeIDsWidensACollisionMadeByTheRename uses a pair of withItems sibling names that do not collide under the old workflow name but do under the new one (found by search, since the hash is fixed), and checks that nothing is lost and both nodes are resolvable by name afterwards. TestNewNodeIDsKeepsDeletedNodesOutOfTheWay checks every node is mapped to a distinct ID. An end-to-end test through FormulateResubmitWorkflow is not possible: it picks the new name at random, so a collision cannot be forced.

go test ./workflow/util/... and the resubmit/retry tests in workflow/controller and server/workflow pass; golangci-lint run ./workflow/util/... is clean. Full make pre-commit -B not run — codegen needs the checkout at its $GOPATH path and this was built in a worktree; nothing here is generated.

Documentation

Not needed, no user-visible behaviour or configuration changes.

AI

Claude Code (Claude Opus 5) found the gap while analysing the CI failure fixed in #17027, and wrote the change, the tests and the commit message. The colliding name pair was found by brute-force search over FNV-1a 32.

Summary by CodeRabbit

  • Bug Fixes
    • Improved workflow resubmission and retry handling when renamed nodes produce identifier collisions.
    • Ensured node identifiers remain unique, including when reset plans remove nodes.
    • Added fallback identifier widening for collisions and clearer handling when both identifier sizes conflict.

Joibel and others added 2 commits September 20, 2026 10:45
The resubmit half of the node ID collision test pinned the behaviour
memoized resubmit had in argoproj#16883, where a failed pod node was copied into
the new workflow as Pending. argoproj#16938 rewrote memoized resubmit to plan
the same reset as a retry, which deletes a failed execution node so the
controller creates it afresh. Neither branch contained the other, so
both were green on their own and main went red on the merge.

Keep the coverage of carrying a widened node ID across the rename by
memoizing a workflow whose colliding leaf succeeded, which resubmit
still keeps, and cover the delete path in its own test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHgquNL1W9UC2xz3R5cmXN
Signed-off-by: Alan Clucas <alan@clucas.org>
Resubmitting gives the workflow a new name, and a node ID is a hash of
the node name, which starts with the workflow name, so the rename
rehashes the whole graph. Collisions the original workflow did not have
can appear, and the mapping used the 32-bit ID unconditionally: the node
that lost the race was dropped from the new status, or, worse, a
reference to a node the reset plan deleted could land on the ID of one
that was kept and survive pruning as a false edge.

Work the IDs out up front, in node name order so the result does not
depend on map iteration order, widening a name whose 32-bit slot has
gone to another one just as the controller does when it creates a node.
Deleted nodes are mapped too, so their references cannot alias a node
that is kept.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHgquNL1W9UC2xz3R5cmXN
Signed-off-by: Alan Clucas <alan@clucas.org>
@codecov

codecov Bot commented Sep 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.66667% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.77%. Comparing base (5642521) to head (bc08dfa).

Files with missing lines Patch % Lines
workflow/util/util.go 91.66% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #17028      +/-   ##
==========================================
+ Coverage   44.75%   44.77%   +0.02%     
==========================================
  Files         575      575              
  Lines       39591    39610      +19     
==========================================
+ Hits        17717    17734      +17     
- Misses      21872    21874       +2     
  Partials        2        2              
Flag Coverage Δ
unit-tests 44.70% <91.66%> (+0.02%) ⬆️
unit-tests-windows 31.71% <91.66%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Joibel
Joibel marked this pull request as ready for review September 20, 2026 11:13
@Joibel
Joibel requested a review from a team as a code owner September 20, 2026 11:13
@coderabbitai

coderabbitai Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: argoproj/argo-workflows/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: cd403793-83f9-42d4-b8d0-3b3f30da769b

📥 Commits

Reviewing files that changed from the base of the PR and between 5642521 and bc08dfa.

📒 Files selected for processing (2)
  • workflow/util/node_id_collision_test.go
  • workflow/util/util.go

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

Memoized resubmission now precomputes unique node IDs after renaming. It widens colliding IDs to 64-bit values, rejects collisions at both widths, and drops references to nodes absent from status. New tests cover rename collisions and reset-plan deletions.

Changes

Memoized resubmit node ID mapping

Layer / File(s) Summary
Precomputed collision-safe node IDs
workflow/util/util.go
FormulateResubmitWorkflow computes renamed node IDs before memoization. mapID uses the computed values, and missing status nodes produce empty IDs for reset-plan removal.
Collision and reset-plan coverage
workflow/util/node_id_collision_test.go
Tests verify 32-bit collision widening, unique mappings when nodes are deleted, and successful node name resolution.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Suggested reviewers: isubasinghe

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the primary change: fixing node ID collisions during memoized resubmission. It also follows the Conventional Commit format.
Description check ✅ Passed The description covers the motivation, modifications, verification, documentation impact, AI use, related issues, tests, and lint results. It notes that the full pre-commit check was not run and inclu…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Joibel Joibel added cherry-pick/4.0 Cherry-pick this to release-4.0 cherry-pick/4.1 Cherry pick this to release-4.1 labels Sep 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cherry-pick/4.0 Cherry-pick this to release-4.0 cherry-pick/4.1 Cherry pick this to release-4.1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant