Skip to content

protocol: keep upstream H2 fallback fail-closed until non-idempotent replay is release-qualified #116

Description

@seonghobae

Refs #51, #58, cloudflare/pingora#979.

Finding / current safety boundary

The current gateway is not presently exposed to this supplier defect. Foundation #1@f181d0464abad56cf4d24a8061c4cc4642b356be deliberately builds upstream peers with peer.options.alpn = ALPN::H1, and its public delivery contract states that HTTP/1.1 is the only accepted upstream protocol in the initial gateway contract. Preserve that fail-closed boundary.

A future upstream-H2 / H2→H1 fallback capability cannot be promoted from Pingora's current public supplier identity without resolving a request-replay defect:

  • protected cloudflare/pingora/main@4487f7b2ab50f159e4a2cf4f6a6b813f61bb6e19 still maps a library-generated HTTP/2 GOAWAY(PROTOCOL_ERROR) while reading response headers to InvalidH2 and unconditionally sets err.retry = true.into();
  • pingora-proxy still treats InvalidH2 as a reason to prefer HTTP/1 when the peer permits H1 fallback, so a request already delivered to an H2 origin can be replayed on a new H1 attempt;
  • cloudflare/pingora#979 is open and labelled bug + Accepted (Accepted: merged to Cloudflare's internal repository), but that internal disposition is not a public, immutable, release-qualified dependency identity;
  • latest public Pingora remains 0.9.0, published 2026-09-09, immutable=false, with no later public release containing a verifiable repair.

The upstream reproducer demonstrates one POST reaching the origin over H2, then being transparently replayed over H1 after an invalid H2 response. This is a transport correctness and safety defect with buyer-visible consequences such as duplicate orders, payments, notifications or other non-idempotent operations. Product-specific idempotency keys, payment semantics, authorization and business recovery remain product-owned; they must not be copied into the shared gateway as a workaround.

Standards boundary

RFC 9110 §9.2.2 states that a proxy MUST NOT automatically retry non-idempotent requests. RFC 9113 §8.7 separately defines the HTTP/2 cases that provide positive evidence a request was not processed (for example a GOAWAY last-stream boundary or REFUSED_STREAM). A generic library-generated PROTOCOL_ERROR after the request may have reached the application is not such evidence.

Primary authority:

RED acceptance

  1. Current shared-gateway upstream delivery remains H1-only; no released/versioned configuration enables H2 upstream or H2→H1 fallback.
  2. Public protected Pingora main still contains the unconditional InvalidH2 retry path and public 0.9.0 is not repaired.
  3. Accepted on upstream #979 is supplier intent/internal disposition only, not public release authority.
  4. No CWL released-supplier fixture proves that a POST whose H2 request has already been delivered is executed exactly once when the origin induces the InvalidH2/downgrade path.
  5. No mutable contributor/internal branch, downstream supplier fork, local patch, scanner exception, or business-level idempotency logic is admissible as transport repair authority.
  6. Do not infer retry safety from "no response received": after request transmission, absence of a response does not prove that the origin did not process a non-idempotent request.

GREEN acceptance

Only after a maintainer-owned repair is present in a public release-qualified Pingora identity:

  1. run an unchanged, local real-wire TLS/H2 origin fixture against the exact selected supplier identity. It must receive a complete POST request/body, induce the library-generated PROTOCOL_ERROR/InvalidH2 path, permit H1 fallback at the peer, and prove the origin observes the non-idempotent operation exactly once;
  2. prove the same no-replay invariant for at least POST and PATCH when the request may already have been applied. Keep RFC-idempotent methods such as PUT/DELETE and positively-unprocessed H2 cases as separate semantics rather than labelling every state-changing method non-idempotent;
  3. include controls for a safe/idempotent request and for HTTP/2 evidence that positively establishes non-processing (REFUSED_STREAM or an applicable GOAWAY last-stream boundary), so the repair does not disable legitimate retry indiscriminately;
  4. test both fresh and reused upstream-connection conditions where the supplier path can differ, request-body completion, downgrade/fallback state, connection-pool cleanup and bounded retry count. No duplicate origin execution, silent second connection carrying the same non-idempotent request, or retry loop is allowed;
  5. preserve H1-only behavior by default. Upstream H2/H1 fallback becomes a versioned capability only when a concrete consumer contract requires it; omission remains fail-closed;
  6. keep product idempotency keys, transaction state, payment/order semantics and recovery logic outside this gateway. The gateway owns only reusable HTTP retry/failover transport semantics;
  7. require exact-head fmt/compile/test/strict Clippy/warning-denied rustdoc/100% owned-production line+region coverage, realistic concurrency/failure traffic, OCI, Supply/Security and current-range review on the selected supplier identity;
  8. after immutable gateway release, the consumer owner still proves its own parity → shadow/canary → observed rollback → cutover. No release or cutover credit comes from this characterization alone.

Promotion order

current H1-only fail-closed guard → public maintainer integration of #979 repair → release-qualified Pingora identity → unchanged released-supplier real-wire replay RED→GREEN → explicit versioned upstream-H2 capability only if consumer evidence requires it → immutable pingora-gateway release → consumer parity/shadow/canary/rollback/cutover.

This issue is a supplier/protocol gate. It is not authority to enable upstream H2 early, to vendor an internal Cloudflare fix, or to move product transaction/idempotency semantics into pingora-gateway.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions