Skip to content

feat: add transparent Warp StreamBus transport - #77

Merged
mirkobrombin merged 3 commits into
mainfrom
feat/warp-streambus
Sep 4, 2026
Merged

mirkobrombin merged 3 commits into
mainfrom
feat/warp-streambus

Conversation

@mirkobrombin

Copy link
Copy Markdown
Member

Summary

This PR adds Warp StreamBus as an alternative transport to WebSocket in RFW.

StreamBus uses WebTransport over HTTP/3 while preserving the existing SSC protocol and public APIs. Applications can enable it without changing components, events, calls, hydration, broadcasts, or session-resumption logic.

WebSocket remains available as the default transport and transparent fallback when WebTransport is unavailable.

Configuration

Enable StreamBus in rfw.json:

{
  "transport": "streambus"
}

Or through an environment variable:

RFW_TRANSPORT=streambus

Supported values:

  • websocket
  • streambus
  • auto

Implementation

  • Added the /streambus WebTransport endpoint backed by Warp StreamBus.
  • Reused the existing JSON SSC protocol over reliable, length-prefixed QUIC streams.
  • Added bounded queues, message priorities, and transport-level backpressure.
  • Kept /ws available as a transparent fallback.
  • Added configuration precedence between RFW_TRANSPORT and rfw.json.
  • Exposed the selected transport through the generated rfw_config.js.
  • Added development certificate-hash discovery for self-signed HTTP/3 certificates.
  • Added direct HTTP/3 endpoint discovery when the development reverse proxy uses a different UDP port.
  • Upgraded Warp to v2.1.0.
  • Fixed a pre-existing synchronization race in the HTTP hook.
  • Updated the README, SSC guide, changelog, templates, examples, and transport documentation.

Compatibility

No application-level API changes are required.

The following features work through both transports:

  • Component updates
  • Send
  • Call
  • Hydration
  • Broadcasts
  • Session resumption
  • Event replay

Performance

Producer-side fanout benchmark using distinct SSC JSON serialization and one publication per target session:

Subscribers StreamBus WebSocket Improvement
1 910 ns/op 3,930 ns/op 4.3x
10 10,957 ns/op 42,859 ns/op 3.9x
100 127,513 ns/op 465,366 ns/op 3.6x

At 100 subscribers:

Metric StreamBus WebSocket Improvement
Allocated memory 27,200 B/op 107,295 B/op 3.9x less
Allocations 500 allocs/op 1,600 allocs/op 3.2x fewer

Environment: Go 1.25.0, AMD EPYC 9V74, median of five one-second benchmark runs.

These results measure server-side publication and fanout overhead. They do not imply that every WebTransport network round trip will always be faster than WebSocket.

Validation

  • go test ./...
  • go test -race ./...
  • go vet ./...
  • Go WASM compilation and JavaScript runner tests
  • Live WebTransport/QUIC integration test
  • WebSocket fallback tests
  • Manifest and environment-variable precedence tests

@mirkobrombin
mirkobrombin requested a review from a team as a code owner September 1, 2026 15:49
@mirkobrombin
mirkobrombin merged commit 6c9cca3 into main Sep 4, 2026
8 checks passed
@pietrodicaprio

Copy link
Copy Markdown
Member

Post-merge review: the transport direction is sound, but I would keep WebSocket as the production default and address the following before advertising StreamBus as production-ready.

1. P1 — outbound backpressure can break SSC ordering

sendStreamBusSession calls session.PrepareOutbound(out) before transport delivery and then discards the error returned by connection.Send:

  • func sendStreamBusSession(connection *streamBusConnection, session *Session, out Outbound) {
    if session == nil {
    return
    }
    session.outboundMu.Lock()
    defer session.outboundMu.Unlock()
    if session.streamConnection != connection {
    return
    }
    _ = connection.Send(session.PrepareOutbound(out))
    }
  • func (c *streamBusConnection) Send(out Outbound) error {
    payload, err := json.Marshal(out)
    if err != nil {
    return err
    }
    ctx, cancel := context.WithTimeout(c.transport.Context(), 5*time.Second)
    defer cancel()
    _, err = c.bus.Publish(ctx, streambus.Frame{
    Topic: c.topic, Payload: payload, Reliability: streambus.Reliable,
    Priority: streambus.PriorityInteractive,
    })
    return err

The subscription uses streambus.Block. When its bounded queue remains full, Publish can block for five seconds and return a context error. The caller has already consumed an SSC sequence number and retained the frame for replay, but the live connection remains attached and the frame was not delivered. A later frame exposes a sequence gap; if this was the last frame, the client can remain silently stale. Because Broadcast still walks StreamBus targets serially, one slow target can also delay every later StreamBus target by up to five seconds per publication.

There is a related invalid-payload failure: StreamBus marshals only after PrepareOutbound. A non-JSON payload is therefore retained and consumes a sequence number before marshal fails; that unmarshalable replay entry can poison every later resume attempt.

Recommended direction: reuse the WebSocket writer invariants from #71. Validate/marshal before sequencing, perform a truly non-blocking enqueue per connection, apply a write deadline, and on queue overflow or write failure close that connection with a best-effort resync_required. Do not ignore transport errors.

Required regression: two StreamBus subscribers, one that stops reading and one healthy. Broadcast must return promptly, the healthy subscriber must receive every frame in order, the slow subscriber must be disconnected, and resume must replay the complete sequence. Also prove that an invalid outbound payload does not consume a sequence number or poison replay.

2. P1 — the second replay journal grows with historical connections

Every connection gets a unique topic, while the endpoint enables a 256-frame Warp replay journal:

  • func newStreamBusEndpoint(runtime *WSRuntime) *streamBusEndpoint {
    maximum := DefaultSSCLimits().MaxMessageBytes
    if runtime != nil && runtime.limits.MaxMessageBytes > 0 {
    maximum = runtime.limits.MaxMessageBytes
    }
    return &streamBusEndpoint{
    runtime: runtime,
    bus: streambus.NewInMemory(streambus.Config{
    DefaultBuffer: 256, MaxBuffer: 4096, ReplayCapacity: 256,
    MaxPayloadBytes: maximum,
    }),
    }
  • func newStreamBusConnection(ctx context.Context, request *http.Request, transport *wt.Session, stream *wt.Stream, bus *streambus.InMemory, maximum int) (*streamBusConnection, error) {
    topic := fmt.Sprintf("rfw/connection/%d", streamID.Add(1))
    subscription, err := bus.Subscribe(ctx, streambus.SubscribeOptions{Topic: topic, Buffer: 256, Overflow: streambus.Block})
    if err != nil {
    return nil, err
    }
    connection := &streamBusConnection{
    request: request, transport: transport, stream: stream, reader: bufio.NewReader(stream),
    bus: bus, topic: topic, subscription: subscription, maximum: maximum, done: make(chan struct{}),
    }
    go connection.writeLoop()
    return connection, nil
    }

In Warp v2.1.0, publishing creates a replay entry per topic, while closing/removing a subscription removes only the subscriber and not the topic journal:

RFW already owns the authoritative sequenced replay in Session, so this transport-level journal is redundant and retains payloads for every historical connection. I would set Warp ReplayCapacity to zero for these per-connection topics, or add explicit topic lifecycle cleanup if the second journal is intentionally required.

Required regression: repeated connect/send/disconnect cycles must return subscription/topic/journal counts to their baseline and retain no per-connection payloads.

3. P1 release gate — merged main is currently red

The post-merge workflow failed EUProvGuard on golang.org/x/crypto v0.55.0:

https://github.com/rfwlab/rfw/actions/runs/33912255934

It reports GO-2026-6354 / CVE-2026-78662 and GO-2026-6355 / CVE-2026-56855, both fixed in x/crypto v0.56.0:

The affected symbols are in x/crypto/ssh, which is not in RFW's compiled dependency closure here (QUIC currently reaches chacha20/chacha20poly1305), so I do not see evidence that this specific SSH path is exploitable through RFW. The release gate is nevertheless correctly blocked and the dependency needs updating and rescanning.

4. P2 — production lifecycle and configuration need a complete contract

  • AcceptStream has no handshake deadline.
  • writeFrame has no QUIC-stream write deadline.
  • StreamBus uses hard-coded buffer/replay/publish timeout values instead of SSCLimits.
  • the HTTP/3 server starts in a background goroutine whose bind failure is only logged and which has no exposed shutdown lifecycle;
  • the built-in certificate is valid for 13 days with no rotation, while there is no public caller-provided TLS/HTTP3 server path matching the production guidance.
  • streambus and auto currently behave identically, while invalid runtime configuration silently becomes WebSocket even though build-time configuration fails closed.

This should either be documented as an experimental/dev transport for now or exposed through an owned server configuration/lifecycle API with caller-provided TLS, bounded handshake/write timeouts, clean shutdown, and consistent validation.

5. P2 — current tests and benchmark do not prove the compatibility claims

The native WebTransport integration test proves one handshake, framing, and generic ack. The WASM test proves URL/config selection only. I did not find end-to-end StreamBus coverage for component initialization/hydration, Send, typed Call, broadcast targeting, resume authorization, replay, queue overflow, or WebSocket fallback under failure.

The benchmark also compares StreamBus in-memory queue publication against an unconfigured synchronous WebSocket socket write. Production WebSocket delivery uses its own outbound writer queue, so the reported 4x result is not an equivalent producer-path comparison. It should measure equal boundaries: queue admission versus queue admission, or complete network delivery versus complete network delivery.

What looks good

Reusing the existing SSC envelope, session initializer/resume authorizer, message authorizer, sequence/ack model, and component API is the right architecture. Frame-size limits and keeping WebSocket available also give this a migration-safe foundation. I would fix the delivery/lifecycle invariants incrementally rather than replace the design.

Local verification on the merged tree: go test ./..., go test -race ./host, and WASM vet for host/hostclient pass. Those green tests do not exercise the failure modes above.

@mirkobrombin

Copy link
Copy Markdown
Member Author

Thanks. I split the release-blocking items into #83 and opened #84.

Some findings were worth fixing, but the framing was off:

  • feat: add transparent Warp StreamBus transport #77 did not merge with failing PR checks. The PR head 7ecf377 had green Test, js/wasm, iOS native SSC, native/wasm golangci, secret scan, and EUProvGuard on Sep 2. The EUProvGuard failure is on the Sep 4 post-merge main run for 6c9cca3.
  • The CVE gate is valid. fix[closes #83]: harden StreamBus release blockers #84 bumps golang.org/x/crypto to v0.56.0, which also raises the module and CI baseline to Go 1.26 because that x/crypto release requires it.
  • The StreamBus outbound sequencing point is valid enough to fix now. fix[closes #83]: harden StreamBus release blockers #84 validates payloads before sequencing, closes a StreamBus connection that cannot accept a frame, and stops keeping a second Warp replay journal for per-connection topics.
  • HTTP/3 lifecycle/configuration work, write deadlines, and broader end-to-end StreamBus coverage are second-PR material. They are useful release hardening work, but they are not evidence that feat: add transparent Warp StreamBus transport #77 merged with a broken PR check or that WebSocket must remain the production default forever.
  • The benchmark doc already scopes the numbers as producer-side dispatch, not network latency or throughput. A stricter comparison should be a dedicated benchmark PR with equal boundaries.

Please review #84.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants