Skip to content

fix(runtime): coordinate graceful API shutdown - #373

Merged
VarshithReddy2006 merged 1 commit into
mainfrom
codex/issue-362-graceful-shutdown
Nov 16, 2026
Merged

VarshithReddy2006 merged 1 commit into
mainfrom
codex/issue-362-graceful-shutdown

Conversation

@vbtgongithub

Copy link
Copy Markdown
Collaborator

Closes #362

I added one API shutdown path for SIGTERM and SIGINT. It closes the HTTP listener, notifies connected SSE clients to reconnect after a randomized 2–5 second delay, and gives local BullMQ workers up to five seconds to finish active jobs. I kept the drain local so workers on other replicas can continue processing.

I added deadlines for each phase. The coordinator flushes pending runtime state and audit writes, then closes Redis and MongoDB. It continues cleanup after a phase fails and reports the failures on exit.

Validation

I ran 54 worker lifecycle tests, 10 SSE resilience tests, 19 frontend reconnect tests, and 13 integration tests. The integration tests covered an open HTTP/SSE stream and disposable MongoDB. Both backend and frontend typechecks passed.

I did not exercise the BullMQ drain against a live Redis instance. The API watchdog is 15 seconds. A host with a shorter termination grace period can interrupt cleanup.

Signed-off-by: VBTG <vuppubhanu18@gmail.com>
@vbtgongithub

Copy link
Copy Markdown
Collaborator Author

@VarshithReddy2006 I have implemented the changes for this issue. Please review my changes and merge this PR if everything looks good.

@VarshithReddy2006 VarshithReddy2006 added enhancement New feature or request help wanted Extra attention is needed nsoc26 level3 10 points - core logic, database changes, and complex features. labels Nov 16, 2026

@VarshithReddy2006 VarshithReddy2006 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Verified graceful shutdown coordinator draining BullMQ workers, notifying SSE clients, and closing Redis and MongoDB connections with passing CI.

@VarshithReddy2006
VarshithReddy2006 merged commit f31b9a4 into main Nov 16, 2026
5 checks passed
@VarshithReddy2006
VarshithReddy2006 deleted the codex/issue-362-graceful-shutdown branch November 16, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request help wanted Extra attention is needed level3 10 points - core logic, database changes, and complex features. nsoc26

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(runtime,m0): implement graceful shutdown coordinator draining BullMQ workers, Redis streams, and Mongoose connections

2 participants