Skip to content

Enforce deadlines and cancel expired Gun streams - #591

Open
aej wants to merge 6 commits into
elixir-grpc:masterfrom
fanatics-live:fix/gun-call-deadline-cleanup
Open

Enforce deadlines and cancel expired Gun streams#591
aej wants to merge 6 commits into
elixir-grpc:masterfrom
fanatics-live:fix/gun-call-deadline-cleanup

Conversation

@aej

@aej aej commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix the Gun client adapter so a finite RPC timeout is enforced as one absolute deadline across the whole call.

When that deadline expires, the adapter now cancels the Gun stream, stops its response process, and removes the connection-process bookkeeping entry. The shared HTTP/2 connection remains available for later RPCs.

Background

The Gun adapter previously doubled the configured timeout on every response wait. That was originally intended to give the server time to enforce the wire-level grpc-timeout and return its own DEADLINE_EXCEEDED response.

Because each headers, data, and trailers wait started a fresh doubled timeout, however, a finite RPC could run well beyond its configured deadline. When a local wait finally timed out, the response process, Gun stream, and bookkeeping entry were also left behind.

Changes

  • Carry one private monotonic deadline through the Gun request lifecycle.
  • Keep the existing relative grpc-timeout wire header unchanged.
  • Arm one deadline timer in the per-stream response process, including while a lazy response stream is not being consumed.
  • Route expiry through the existing Gun cancellation and bookkeeping paths.
  • Give response frames already in flight a bounded 100 ms grace period before resetting an expired Gun stream, avoiding Gun 2.4's connection error for late HEADERS.
  • Skip the reset when the server has already closed the stream during that grace period.
  • Make timeout, terminal-response, late-message, and repeated-cancellation cleanup idempotent.
  • Preserve the public API and shared connection lifecycle.

The first terminal event wins: a response handled before expiry succeeds; expiry handled first returns DEADLINE_EXCEEDED. Local expiry may therefore cancel the request before the server logs its own deadline.

Validation

  • Added a test-only commit that reproduces the leaked response process before the fix.
  • Added focused coverage for no headers, stalled response bodies, unary and streaming calls, lazy streams, terminal/deadline ordering, repeated timeouts, late messages, and explicit cancellation.
  • Added an interop regression that makes a real unary RPC on the same channel immediately after a deadline.
  • mix run script/run.exs --rounds 20 --concurrency 2 passes.
  • mix test --warnings-as-errors --max-cases 1
    • grpc_core: 122 tests/doctests, 0 failures
    • grpc_server: 213 tests/doctests, 0 failures
    • grpc: 377 tests/doctests, 0 failures, 2 skipped

@aej aej changed the title Fix/gun call deadline cleanup Fix Gun call deadline enforcement and stream cleanup Sep 9, 2026
@aej aej changed the title Fix Gun call deadline enforcement and stream cleanup Enforce deadlines and cancel expired Gun streams Sep 10, 2026
@aej
aej marked this pull request as ready for review September 11, 2026 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant