Skip to content

dogfood: de-flake the #489 streaming-RPC e2e (mid<final race) #973

Description

@vivek7405

Problem

The blog e2e test a streaming-RPC action renders tokens incrementally (#489) (test/e2e/e2e.test.mjs, around L506-536) is timing-flaky on a loaded/fast runner. It catches the token stream "partway" to prove incremental arrival: it waits until >= 2 .ts-item elements exist, snapshots that count as mid, waits for all 8, then asserts mid < final. When the browser is starved under CI load, the waitForFunction(>=2) poll can resolve LATE (after all 8 tokens have already rendered), so mid = 8 and the assertion fails with the count climbed incrementally (mid=8 < final=8).

Observed: failed once on the E2E (Puppeteer against the blog example) job on PR #970, passed on an immediate re-run of the same commit, and passed on the E2E (blog served on Bun) job with identical code. So it is a pre-existing test-robustness race, NOT a product bug (PR #970 changed only Tailwind dev-CSS handling, which has no path to RPC-stream timing).

Design / approach

Two robust fixes, either or both:

  • Slow the source a touch so the stream is reliably catchable mid-flight regardless of runner load. The action already sleeps 60ms per token (8 tokens ~= 480ms); a larger gap (e.g. 120ms) widens the window the snapshot must land in. Keep it modest so the test stays fast.
  • Make the assertion race-free instead of snapshot-based: assert the count is STRICTLY INCREASING across two mid samples (or assert mid >= 2 && mid < 8 was ever observed by polling), rather than a single mid < final snapshot that a fast completion defeats. The intent is "tokens arrive incrementally, not one buffered batch", which a monotonic-increase check captures without depending on catching an exact instant.

The content-type assertion (application/vnd.webjs+stream) is not flaky and should stay.

Implementation notes (for the implementing agent)

  • Test: test/e2e/e2e.test.mjs, the #489 test at ~L506-536 (the mid/final/waitForFunction(.ts-item >= 2) block).
  • Source action: examples/blog/modules/verbdemo/actions/stream-tokens.server.ts (streamTokens, the await new Promise((r) => setTimeout(r, 60)) per-token delay). If you bump the delay, keep the total well under the test's 8s waitForFunction timeouts.
  • Component: examples/blog/components/token-stream.ts (renders .ts-item / .ts-start); page examples/blog/app/rpc-stream/page.ts.
  • Landmine: the e2e is gated behind WEBJS_E2E=1 and does NOT run under a plain npm test; run WEBJS_E2E=1 node --test test/e2e/e2e.test.mjs to reproduce, ideally under CPU load to surface the race. The same test runs on Bun (E2E (blog served on Bun)), so any source-side change (the delay) must stay cross-runtime-safe.
  • Invariant: do not weaken the test's real assertion (that arrival is incremental, not buffered); make it robust, not lax.

Acceptance criteria

  • The #489 test no longer depends on catching an exact mid-stream instant (no single mid < final snapshot that a fast completion defeats)
  • The test still fails if the stream is served buffered-all-at-once (counterfactual: force the action to yield everything before the first flush and confirm the test goes red)
  • Passes reliably on both the Puppeteer and Bun e2e jobs across repeated runs

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions