Skip to content

test(browser): fix six journey sync flakes from the 2026-11-04 sweep - #2566

Merged
matt2e merged 5 commits into
mainfrom
mtoohey/flaky-journey-sync
Nov 5, 2026
Merged

test(browser): fix six journey sync flakes from the 2026-11-04 sweep#2566
matt2e merged 5 commits into
mainfrom
mtoohey/flaky-journey-sync

Conversation

@matt2e

@matt2e matt2e commented Nov 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Six browser journeys each acted before a boundary they depended on: a debounced search landing, a background recovery read (in two journeys), a message reveal taking focus, a cold page load that fetched the fixture module by module, and a pause-then-seek in the same tick. Each commit adds the missing barrier, or removes the race at its source. All six changes are test-only: no assertions are loosened, no retries, sleeps or timeouts change, no browser cases are added or removed, and no product code changes.

test(channels): let the member search land before the final pointer expansion

Problem

WebKit's channel-members-focus.spec.mjs:826 "final invitation expansion continues keyboard focus at the first new row" failed 5 times between 2026-11-02 23:00Z and 2026-11-04 11:00Z, on 4 unrelated branches including main. Every failure was in the pointer iteration at :853: "Your other agents" held 8 "Add Helper" rows, never 10.

Cause

Typing "Helper" shows the local agent rows at once. The relay search runs after SEARCH_DEBOUNCE_MS (300 ms) and adds a "People not in this channel" group above "Your other agents". In all five traces, the pointer click started 185–260 ms after the fill, while the list was still aria-busy. The click passed Playwright's hit-target check inside Show more's box (y 523–547). The People group then landed and moved Show more to y 543–567, so the press hit Show more and the release hit the row below it, and no click reached the button.

Fix

After filling the search, wait for the Member list's data-refreshing="false", as this file's large-search test already does. The search keeps loading true from the moment the text changes until that text's results arrive, so the barrier can't pass early.

Validation

  • Adversarial ordering, with a fixture gate that held the relay search until pointerdown on Show more and the debounce lowered to 100 ms: without the fix 20/20 failed at :853 with "Received: 8", and with the fix 20/20 passed.
  • Natural contention, with 4 CPUs shared with 4 busy loops: without the fix 1/24 failed with the CI signature, and with the fix 48/48 passed.
  • Both engines, --repeat-each=30: 60/60 passed. The full file, both engines: 36/36 passed.

test(channels): deny the failed preference read so recovery can't race Retry

Problem

WebKit's sidebar-section-creation.spec.mjs:9 "Channels offers empty-section creation with keyboard focus and confirmed persistence" failed at :167 clicking "Retry preferences". The call log said "element is not enabled", then "element was detached from the DOM".

Cause

The journey failed the channel-sections writer read with a 502. A 502 counts as transient, so relay recovery reads again after 1 s. That read disables Retry preferences, and its success removes the button. In the trace, WebKit's first actionability check for the Retry click returned 100 ms after recovery's read reached the route. Any click that takes about a second to land loses the Retry to recovery.

Fix

Inject a 403 instead. A denial isn't transient, so recovery leaves it alone, and the manual Retry is the only way out, which is the path this journey exists to exercise. The failure is recorded under app.report.relayFailures. The check that the alert doesn't show the status now looks for "403".

Note on main's #2150

#2150 merged to main after this commit was written (as 21f6b97a99). It makes the same 403 and relayFailures change in this hunk, with its own one-line comment, so the cherry-pick conflicted there. The conflict was resolved by keeping this commit's side. On this branch, the commit therefore only replaces that comment with the longer explanation and changes the alert check from "502" to "403". On main that check passes no matter what, because the injected status is already 403. The commit body still refers to #2150 as an open PR. No other adjustment was needed.

Validation

  • Adversarial ordering, with the Retry click delayed 1.2 s and the first read after the failure held for 600 ms:
    • without the fix: 10/10 failed with the CI signature
    • with the fix: 10/10 passed, and 6/6 passed with a 3.5 s delay
  • Both engines, --repeat-each=20: 40/40 passed. The full file, both engines: 6/6 passed.

test(inbox): wait for the entry reveal before filling an opened conversation

Problem

WebKit's inbox-archive.spec.mjs:658 "reply and fresh mention reopen their filters without replacing composer draft" failed on attempt 1, and the same SHA passed on rerun. The composer was empty: Expected: "Unsent mention retention draft", Received: "".

Cause

Opening a conversation reveals its entry reply. useMessageReveal focuses the row two frames after the thread target is ready. The journey filled the composer as soon as it appeared, so on a slow runner the reveal landed during the fill. Playwright focuses the editor and selects its contents, and then inserts text at the selection. In WebKit, the row taking focus clears that selection, so nothing is inserted. A probe that focused the row between the two steps reproduced it in WebKit but not in Chromium. The trace shows the only DOM change during the fill was the entry row gaining tabindex=-1. Three more journeys in the file share the race. The existing in-view wait in one of them isn't a barrier, because the row is in view before the reveal runs.

Fix

Add a revealed() helper. It waits for the conversation list's aria-busy to be false, for the entry row to be focused, and then two frames. All four journeys that fill after Open now call it. The product's focus policy is addressed separately in #2563.

Validation

  • Adversarial ordering, with the reveal held 800 ms and run as soon as a fill placed its selection: without the fix 17/24 failed, and with the fix 24/24 passed.
  • The four journeys, both engines, --repeat-each=10: 80/80 passed. The full file, both engines, --repeat-each=2: 40/40 passed.

test(voice-notes): serve the fixture as a build so WebKit's first load fits

Problem

The "WebKit recording (macOS)" lane failed in the first case of voice-notes.spec.mjs (:138), in the shared beforeEach. page.goto(.../tests/fixtures/voice-notes.html) exceeded the 15 s navigation timeout.

Cause

The fixture runs the real PluginRuntime, ConversationService, relay session and Voice Notes plugin, and the Vite dev server served that graph module by module. The traces show about 816–818 requests. Vite's median wait was about 50 ms, but the median receive was about 1.0–1.14 s, and the first request to the last response took 13.3–14.5 s. On a cold 3-core macOS runner, WebKit's per-module round trips alone nearly fill the budget. #1971 only warmed the server's transforms.

Fix

  • Build the fixture once in beforeAll and serve it with build and preview from vite-server.mjs, as four other specs already do. The page now loads in 4 requests instead of 838.
  • The build keeps development React (import.meta.env.DEV and process.env.NODE_ENV are defined as build.mjs does), so StrictMode's double effects still apply.
  • The last case used to import /src/features/messages/attachment-draft.ts from the dev server. The fixture now exposes attachmentDraft bound to its session instead.

Validation

Cold page.goto to load, WebKit (WPE), one new browser per run:

CPUs, busy loops dev server build
3, none 1.15–1.23 s 0.53–0.65 s
3, 3 2.0–2.85 s 0.44–1.21 s
2, 8 3.7–6.8 s 1.1–2.55 s
1, 3 5.1–6.1 s 2.0–2.3 s
  • CI config, Chromium (all five cases) and Linux WebKit (the untagged case), --repeat-each=10: 60/60 passed. Pinned to 3 CPUs with 3 busy loops, --repeat-each=5: 30/30 passed.
  • The macOS lane couldn't be validated here. Its selection (--project webkit --grep @media-recorder) can't pass on Linux, because Playwright's Linux WebKit has no MediaRecorder. Before and after this change, those cases fail at the same step locally. Hosted macOS CI is the first real check of the fix.
  • tsc --noEmit is clean, and the flake-check listing test in tests/integration/flake-check.test.mjs that reads this spec passes.

test(github): seek the fullscreen PR video without pausing it in the same tick

Problem

WebKit's github-body.spec.mjs:35 "PR media plays under the packaged media policy in a narrow GitHub panel" failed at :189. The fullscreen video's seeked fired, but currentTime was about 0.5 instead of 15.

Cause

The fullscreen VideoPlayer autoplays and, on loadedmetadata, seeks to the inline video's position. The journey then paused the playing video and set currentTime in the same evaluate. There were two candidate mechanisms:

  • (a) the restore lands after the journey's seek and overwrites it
  • (b) WebKit completes a seek issued in the same tick as pause() on a playing video without updating currentTime

Local probes support (b). About 0.2% of reloaded samples were unmoved: seeked fired, but currentTime didn't change. In those samples the restore always came before the journey's seek, and requestVideoFrameCallback showed a frame at 15.2 s while currentTime still read under 0.2. Across 1,800 reopen samples, loadedmetadata never landed after the journey's seek.

Fix

Before the journey opens the viewer, a MutationObserver pauses the fullscreen video as soon as it mounts, so it never plays. The observer also resolves on the video's first seeked, which is the restore's. The journey waits for that before its own seek, and no longer calls pause() in the seek evaluate. This covers both candidate mechanisms: the video has never played (b), and the restore has already landed (a). The assertion stays exact (toBeCloseTo(…, 1)), and the two inline videos still cover playback.

Validation

  • Reload A/B, WebKit, 30 repeats × 50 samples per mode, 4 CPUs shared with 2 busy loops:

    • old: 3/1,500 unmoved, all with the CI signature
    • new: 0/1,500 unmoved

    That fits (b), but it isn't statistically conclusive (one-sided p ≈ 0.12).

  • Reopen A/B: 0/900 unmoved in both modes, and the restore never landed late (0/1,800).

  • "PR media plays", both engines, --repeat-each=10: 20/20 passed. WebKit pinned to 4 CPUs with 4 busy loops, --repeat-each=16: 16/16 passed.

test(channels): deny the retried preference read so recovery can't race the final Retry

Problem

Found in code review, following up the sidebar-section-creation commit above. navigation-groups.spec.mjs "Create new retains its draft when preferences fail before submission and recovers in place" injects 502s on sidebar-preferences and clicks "Retry preferences" twice, the second time right after page.unroute. Recovery's next background read can succeed and remove the alert before that click lands. No CI failure of this journey was looked up. The race was found by reading the code and confirmed with a forced delay.

Cause

  • The first 502 starts recovery, which retries after 1 s and again about 2 s later, and the route fails both.
  • The dialog's Retry calls preferences.reload. That cancels recovery's timer but keeps its attempt count, so the next background read is scheduled 2 s or 4 s after the manual Retry fails.
  • After the unroute, that background read succeeds, and the alert unmounts with its button before a slow final click lands.

Fix

Keep the first failure a 502, so recovery still owns it and shows the "Reconnecting to relay…" notice the journey asserts. From the first manual Retry on, the route answers 403, so recovery stays idle and the final click is the only thing that can recover preferences. The 403s go under app.report.relayFailures, and the 502 stays under sidebarPreferenceFailures.

Validation

  • A forced wait between page.unroute and the final click, both engines:
    • without the fix, 4.5 s: 4/4 failed with the click timing out
    • without the fix, 1.5 s: 6/6 passed, consistent with the 2–4 s window
    • with the fix, 4.5 s: 6/6 passed
  • Both engines, --repeat-each=10: 20/20 passed. The full file, both engines: 20/20 passed.
  • Rebased on main f4805605f5: tsc --noEmit, biome check --error-on-warnings on the seven changed files and git diff --check are clean. Biome's one info-level useLiteralKeys note is on an untouched line that is the same on main. node --test tests/integration/flake-check.test.mjs passes 10/10. In WebKit and Chromium, --workers=4 --retries=0, channel-members-focus, sidebar-section-creation, inbox-archive, github-body and navigation-groups in full passed 84/84. voice-notes.spec.mjs under playwright.ci.config.mjs, Chromium and Linux WebKit: 6/6 passed. The macOS recording lane was not run, as noted above.

A green run doesn't prove the flakes are gone. Hosted CI covers broad validation.

Coordination

  • Draft test: settle member search before invitation expansion #2532 adds the same data-refreshing wait in channel-members-focus.spec.mjs and conflicts textually with this branch. It duplicates the first commit, so only one of the two should land.
  • feat(inbox): rename to Activity and simplify the filter bar #2543 (open, head c1dcf443) deletes inbox-archive.spec.mjs and moves its draft journeys into a new inbox-mark-read.spec.mjs. There it works around the same reveal-vs-fill race with a typeDraft helper that refills until the text lands (expect(...).toPass()). Git reports a modify/delete conflict with this branch. A resolver who just takes the delete drops the revealed() fix. Porting the helper into inbox-mark-read.spec.mjs would replace the retry loop with a real barrier.
  • Run native Codex through plugin execution targets #2150 is merged. Its overlap in sidebar-section-creation.spec.mjs was resolved as described in the second commit's section.

🤖 Generated with Claude Code

@matt2e
matt2e requested a review from a team as a code owner November 5, 2026 02:48
This was referenced Nov 5, 2026
Open
Merged
matt2e added a commit that referenced this pull request Nov 5, 2026
@matt2e @claude
test(sidebar): wait for the rolled-back row to leave Starred before c…
6b697fa 
…ollapsing Work (#2574)

## Summary

`navigation-groups.spec.mjs` "dismissing a failed move restores keyboard
focus to its placement or a surviving row" clicked Work's header before
the rolled-back row had left Starred. This adds the missing barrier to
the journey's `failMove` helper. The change is test-only: no assertions
are loosened, no retries, sleeps or timeouts change, and no product code
changes.

## Problem

WebKit's `navigation-groups.spec.mjs:650` failed once, at `await
expect(beta).toBeHidden()` after the click on Work's summary, on
`workflow-ui-issues` (#2362), a branch that doesn't touch the sidebar.

- run 41946923047:
https://github.com/block/buzz-app/actions/runs/41946923047/job/126981730084

In a scan of about 311 failed browser-journey jobs over about 25 hours,
this journey failed only this once.

## Cause

The trace (artifact `browser-journeys-webkit-11`) shows the following:

- The 502 lands and the alert appears with the rollback render. The
optimistic Starred copy of the row is then an `AnimatePresence` exit: it
stays in the DOM and in layout (`inert`, `aria-hidden`, fading), and the
row re-enters Work at once.
- The journey's barrier, Work's beta row being visible, passes as soon
as that row mounts at about 5% opacity (snapshot opacity 0.046), while
the Starred copy is still there.
- The click on Work's summary measured its box with the Starred copy
still laid out (y 393–425). Removing the copy moved the header up by a
row, to y ≈ 379 in the final screenshot.
- Playwright's hit-target interceptor decides on the first event of a
click and doesn't recheck later events. The press landed on the header,
and the release landed on the Beta row that slid under it. The browser
fired no click on the header, so `toggle("group:work", false)` never
ran.
- The trace is consistent with this. Work's `<details>` stays `open` and
its content never gets `hidden` or `inert` in any snapshot after the
click, and the call log shows no retry. The final screenshot has the
pointer resting on the Beta row, which shows its hover highlight, and
Alpha is still the current channel.

This is the same press-and-release mismatch that #2566 describes for
Show more in `channel-members-focus`. Here the layout shift is an exit
animation completing rather than a search result landing.

## Fix

After the alert, `failMove` waits until the sidebar holds exactly one
`beta` row. Only the rolled-back placement in Work remains, so the
Starred exit has finished before the journey acts on the layout. The
helper runs for both of the journey's failed moves.

## Validation

- Adversarial ordering, with `requestAnimationFrame` held from the 502
until the header press and a 300 ms press-to-release delay on the header
click:
- without the fix: 20/20 failed (10 Chromium, 10 WebKit) with the CI
signature. The press landed on the Work summary with two beta rows in
the DOM, the release landed on the Beta row, and the click fired on a
common ancestor.
  - with the fix: 20/20 passed.
- Whole `navigation-groups.spec.mjs`, `--workers=4 --retries=0
--repeat-each=10`: WebKit 100/100 passed (3.8 min) and Chromium 100/100
passed (3.9 min). The machine was heavily loaded (load average about
70), and the WebKit here is Playwright's macOS build, not the Linux
build CI uses.
- `tsc --noEmit`, `biome check --error-on-warnings` on the changed file
and `git diff --check` are clean. `node --test
tests/integration/flake-check.test.mjs` passes 10/10.

This changes the same file as #2566 (a different journey, so the hunks
don't overlap). A green run doesn't prove the flake is gone, because it
fired once in about 311 jobs. Hosted CI covers broad validation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Matt Toohey <contact@matttoohey.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
matt2e and others added 5 commits November 5, 2026 15:45
@matt2e @claude
test(channels): deny the failed preference read so recovery can't rac…
bd7c9bd 
…e Retry

## Problem

WebKit's `sidebar-section-creation.spec.mjs:9` "Channels offers
empty-section creation with keyboard focus and confirmed persistence"
failed at line 167 when clicking "Retry preferences". The click timed out
after 10 s. Its call log says "element is not enabled", then "element was
detached from the DOM".

- clay/bestie-empty-state: run 41547288213,
  https://github.com/block/buzz-app/actions/runs/41547288213/job/125626193138
- kylez_NEXUS-4082_channel_unread_boundary: run 41553877476 attempt 1,
  https://github.com/block/buzz-app/actions/runs/41553877476/job/125648251816
  (attempt 2 passed)

## Cause

The journey's second creation fails the strong channel-sections writer
read with an injected 502. `isTransientFailure` counts a 502 as
transient, so the preference store publishes the error with `transient`,
and relay recovery reads again after `RECOVERY_DELAYS_MS[0]` (1000 ms).
That read disables Retry preferences (`disabled={refreshing}`), and its
success removes the button. This is intended: recovery owns transient
background reads.

The trace from run 41547288213 shows the race. The 502 was fulfilled at
153289 ms and the click started at 153482. WebKit's first "visible,
enabled and stable" check did not return until 154493. Recovery's read
reached the route at 154393, so that check found the button disabled.
The button detached at 155188, and the click timed out. Any click that
takes more than about a second to land loses the Retry to recovery.

## Fix

Inject a 403 instead. A denial is not transient, so `loadState` reports
it as settled and recovery leaves it alone. The manual Retry is then the
only way out, which is the path this journey exists to exercise. The
fixture already uses this reasoning for activity reads ("Activity reads
retry a 502 quietly, so their journeys inject a 403").

The failure is recorded under `app.report.relayFailures`, the fixture's
allowance for injected relay 403s, rather than
`sidebarPreferenceFailures`, which only matches 502s. The check that the
alert doesn't show the status now looks for "403". A comment explains
why a transient status would race recovery. The existing `writerReads`
of 3 after the click now also confirms that no background read ran
first.

Only the test changes (+7/-4). No other assertions change, and no
product code changes or browser cases are added or removed.

Open PR #2150 ("Run native Pi agents through plugin execution targets")
makes the same 403 and `relayFailures` change in this hunk as part of a
feature change. It leaves the alert check on "502". When #2150 rebases,
it can take this side of the hunk.

## Validation

Local Linux workstation, Playwright 1.63.0, `--no-deps --workers=1
--retries=0`:

- Adversarial ordering, using temporary, uncommitted changes: the Retry
  click was delayed 1.2 s, modelling the slow actionability check, and
  the first writer read after the injected failure was held for 600 ms.
  Without the fix: 10/10 failed (5 WebKit, 5 Chromium), all at the Retry
  click with "element is not enabled" followed by "element was detached
  from the DOM", the CI signature. Holding the read without delaying the
  click: 6/6 passed. With the fix: 10/10 passed, and 6/6 passed with a
  3.5 s delay, past recovery's first two retries.
- Natural contention: WebKit pinned to 4 CPUs shared with 4 busy loops,
  2 workers, 24 repeats. Without the fix: 24/24 passed, so load alone did
  not open the window locally. With the fix: 24/24 passed.
- `bin/pnpm test:browser tests/browser/sidebar-section-creation.spec.mjs
  --project webkit --project chromium --no-deps --workers=1
  --grep "empty-section creation" --repeat-each=20`: 40/40 passed.
- Full `sidebar-section-creation.spec.mjs`, both engines: 6/6 passed.
- `biome check` and `git diff --check` are clean.

A green run doesn't prove the flake is gone. Hosted CI covers broad
validation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: mtoohey <mtoohey@squareup.com>
(cherry picked from commit 69eae17b94b702a9c74c154e37ea147441c7ab11)
@matt2e @claude
test(inbox): wait for the entry reveal before filling an opened conve…
c020a70 
…rsation

## Problem

WebKit's `inbox-archive.spec.mjs:658` "reply and fresh mention reopen
their filters without replacing composer draft" failed on attempt 1 on
branch peon/people-relationships-cache, and the same SHA passed on
rerun:

- run 41752428902 (webkit 14/14),
  https://github.com/block/buzz-app/actions/runs/41752428902/job/126320899490

    Error: expect(locator).toHaveText(expected) failed
    Expected: "Unsent mention retention draft"
    Received: ""

## Cause

Opening a conversation reveals its entry reply: `useMessageReveal`
scrolls to the row, sets `tabindex=-1` and focuses it once the thread
target is ready, two frames after the commit. The journey filled the
composer as soon as it appeared, so on a slow runner the reveal landed
during the fill. Playwright fills a contenteditable in two steps: it
focuses the editor and selects its contents, then inserts the text at
the selection. In WebKit, focusing the row clears the editor's
selection, so the insert had nowhere to go. A temporary probe that
focused the row between the two steps reproduced it: WebKit inserted
nothing, while Chromium kept the selection and inserted the text.

The trace shows the same thing. Between the fill's before and after
snapshots, the only DOM change is the entry row gaining `tabindex=-1`.
The editor's paragraph and trailing break are the same unchanged nodes,
so the text never landed. A reset that cleared it would have replaced
them.

A user typing as the thread finishes loading doesn't lose text to the
seed reset. That reset only applies to an untouched draft, and
`valueRef` updates synchronously on input. Probes logged no editor
reset while text was typed. Characters typed before the reveal stay in
the composer.

The reveal does take focus from the composer, though. Its first focus
for a request doesn't check where focus is now, so keys typed after it
go to the row. That focus policy is shared by every reveal, so this
change doesn't touch it.

Three more journeys in this file open a conversation and fill it, and
they share the race. The "unselected row Archive" journey already
waited for the entry row to be in viewport plus two frames, but that
isn't a barrier: the row is in view before the reveal runs.

## Fix

Add a `revealed()` helper. It waits for the conversation list's
aria-busy to be false, for the entry row to be focused, and for two
frames via the shared `frames()` helper. A focused row is the reveal's
own observable effect, so filling after it can't overlap the reveal.
All four journeys that fill after Open now call it, replacing the
inline in-view wait.

Only the test changes (+22/-12). No assertions are loosened, and no
product code changes or browser cases are added or removed.

## Validation

Local Linux workstation, Playwright 1.63.0, `--no-deps --workers=1
--retries=0`:

- Adversarial ordering, using a temporary, uncommitted model in
  `useMessageReveal`: the entry reveal was held for 800 ms. If a fill
  placed its selection in a textbox meanwhile, the model ran the reveal
  at that moment and cleared the selection when the row took focus, as
  WebKit did in CI. The four journeys, both engines, `--repeat-each=3`:
  without the fix 17/24 failed, with the empty editor (or a disabled
  Send) in every journey, the existing in-view wait included. With the
  fix: 24/24 passed.
- The four journeys, both engines, `--repeat-each=10`: 80/80 passed.
- The same four journeys with four busy loops pinned to the
  workstation's CPUs, `--repeat-each=4`: 32/32 passed.
- Full `inbox-archive.spec.mjs`, both engines, `--repeat-each=2`:
  40/40 passed.
- `biome check` and `git diff --check` are clean.

A green run doesn't prove the flake is gone. Hosted CI covers broad
validation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: mtoohey <mtoohey@squareup.com>
(cherry picked from commit c715b9eb7603aeca65fb85629fc210189df576c3)
@matt2e @claude
test(voice-notes): serve the fixture as a build so WebKit's first loa…
8c941af 
…d fits

## Problem

The "WebKit recording (macOS)" lane failed in the first case of
`voice-notes.spec.mjs`, "record, preview, send and play the chat card
with speed and seeking" (:138). Both failures were in the shared
beforeEach: `page.goto(.../tests/fixtures/voice-notes.html)` exceeded
the 15 s navigation timeout waiting for load. The other three cases in
the job passed.

- renovate/all-minor-patch: run 41755043277,
  https://github.com/block/buzz-app/actions/runs/41755043277/job/126329063633
- peon/dnd-stable-transform: run 41757728323 attempt 1,
  https://github.com/block/buzz-app/actions/runs/41757728323/job/126337712196
  (attempt 2 passed)

## Cause

The fixture runs the real PluginRuntime, ConversationService, relay
session and Voice Notes plugin, and a Vite dev server serves that graph
module by module. Load waits until every module has run.

Both traces show the client, not Vite, setting the pace:

- Run 41755043277 made 818 requests: 314 .js, 263 .ts, 123 .tsx,
  82 inline .png and 29 .css. Vite's median wait was 48 ms, but the
  median receive took 1.14 s and the slowest 2.6 s, for responses of
  1-3 KB. The last response ended 13.3 s after the first request
  started. For over 3 s in the middle, no new request started.
- Run 41757728323 made 816 requests: median wait 53 ms, median receive
  1.0 s, and 14.5 s from first request to last response. Its
  `@vite/client` request also waited 6.1 s.

Each module costs WebKit a round trip through its network and web
processes. On a 3-core macOS runner with a cold browser, the 800 round
trips by themselves nearly fill the budget. Later cases reuse the warm
browser and pass. #1971 warmed Vite's transforms in beforeAll, which
fixed only the server's side.

## Fix

Build the fixture once in beforeAll and serve the bundle with `build`
and `preview` from `vite-server.mjs`, as `agent-share-card`,
`channel-members-focus`, `conversation-name` and `goose-model-row`
already do. The page now loads in 4 requests instead of 838, and the
warmup goes away because a build has nothing left to transform.

The dev server served development React. The build defines
`import.meta.env.DEV` and `process.env.NODE_ENV` as `build.mjs` does for
`developmentReact`, so StrictMode's double effects and React's
development warnings still apply. The first case asserts no unexplained
page errors, and the microphone cases count releases. The bundle
contains React's development warnings.

The last case read the composer's draft by importing
`/src/features/messages/attachment-draft.ts` from the dev server inside
`page.evaluate`. A bundle has no such URL. The fixture now exposes
`attachmentDraft` bound to its session instead. Drafts are cached per
session in that module, so this reads the composer's own draft. The
manual dev-server preview described in `docs/plugin-architecture.md`
still works.

No assertions, timeouts or retries change, and no browser cases are
added or removed.

## Validation

Local Linux workstation, Playwright 1.63.0, WebKit (WPE), `--no-deps
--workers=1 --retries=0`. Each run started a new browser, so each run's
first case was cold. Times are `page.goto` until load, with the run and
busy loops pinned by `taskset`:

| CPUs, busy loops | dev server (838 requests) | build (4 requests) |
| ---------------- | ------------------------- | ------------------ |
| 3, none          | 1.15-1.23 s (5)           | 0.53-0.65 s (5)    |
| 3, 3             | 2.0-2.85 s (20)           | 0.44-1.21 s (20)   |
| 2, 8             | 3.7-6.8 s (10)            | 1.1-2.55 s (10)    |
| 1, 3             | 5.1-6.1 s (10)            | 2.0-2.3 s (10)     |

The first case alone, cold, with 3 CPUs and 3 loops: 2.3-2.8 s before,
0.7-1.1 s after. Linux WebKit spends much less per request than the
macOS runner did (a median receive of about 1 s there), so these local
gains are 2-3x. In CI, where the per-request cost dominated, the gain
should be larger. The build itself takes 2.5 s on 3 CPUs and 3.9 s on
1 CPU, inside beforeAll's 120 s.

- `bin/pnpm test:browser --config tests/browser/playwright.ci.config.mjs
  tests/browser/voice-notes.spec.mjs --project chromium --project webkit
  --no-deps --retries=0 --repeat-each=10` (Chromium runs all five cases,
  Linux WebKit the untagged one): 60/60 passed. The same command pinned
  to 3 CPUs shared with 3 busy loops, `--repeat-each=5`: 30/30 passed.
- The macOS lane's selection (`--project webkit --grep @media-recorder`)
  can't pass on Linux, because Playwright's Linux WebKit has no
  MediaRecorder. Before and after the change, all four cases failed at
  the same step: Stop recording never appears. Hosted macOS CI runs them.
- `biome check --error-on-warnings`, `tsc --noEmit` and
  `git diff --check` are clean. The flake-check listing test in
  `tests/integration/flake-check.test.mjs` that reads this spec passes.

A green run doesn't prove the flake is gone. Hosted CI covers broad
validation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: mtoohey <mtoohey@squareup.com>
(cherry picked from commit eedec5a9b58fb97afef64f378f1cccdca928a573)
@matt2e @claude
test(github): seek the fullscreen PR video without pausing it in the …
b109800 
…same tick

## Problem

WebKit's `github-body.spec.mjs:35` "PR media plays under the packaged
media policy in a narrow GitHub panel" failed at :189,
`expect(seek.actual).toBeCloseTo(seek.expected, 1)`. The fullscreen
video's `seeked` fired, but `currentTime` was still near half a second
instead of 15 (half the duration).

- peon/federated-session-bundle: run 41567557720 attempt 1 (Received
  0.533400327),
  https://github.com/block/buzz-app/actions/runs/41567557720/job/125692945945
  (the rerun passed)
- renovate/all-minor-patch: run 41683322444 (Received 0.48949321),
  https://github.com/block/buzz-app/actions/runs/41683322444/job/126083327417

## Cause

The fullscreen `VideoPlayer` autoplays, and its `onLoadedMetadata`
handler seeks to the inline video's position (`initialTime`). The
journey waited for `readyState` and `seekable`. Then, in one evaluate,
it called `pause()` on the playing video and immediately set
`currentTime` to half the duration.

There were two possible explanations:

(a) The restore lands after the journey's seek and overwrites it, so
    the `seeked` the journey awaits is the restore's.
(b) WebKit completes a seek issued in the same tick as `pause()` on a
    playing video without updating `currentTime`. `seeked` fires, but
    `currentTime` stays where playback paused.

This change fixes (b), the only one the evidence supports. Local probes
(WebKit WPE, Playwright 1.63.0) repeated the journey's pause-then-seek
many times per test. Some also logged every media event, every
`currentTime` write and the journey's own seek:

- Reloading the fullscreen video before each sample, so it autoplays
  and restores again, left about 0.2% of samples unmoved: `seeked`
  fired, and `currentTime` still equalled its value before the seek
  300 ms later. That was 3 of 1,500 in the A/B below and 5 of 1,810
  in earlier runs.
- In all three unmoved samples whose event log was kept,
  `loadedmetadata` and the restore's write came before the journey's
  seek. Across 1,800 samples that reopened the viewer,
  `loadedmetadata` never landed after the journey's seek. Nothing
  showed (a) happening.
- In the two unmoved samples that checked, `requestVideoFrameCallback`
  reported a presented frame at 15.2 s while `currentTime` still read
  0.062 and 0.157. The pipeline had moved, but the position WebKit
  reported had not.

The CI values sit near the inline video's paused position. By
themselves they would fit either explanation. The local evidence
decides between them.

## Fix

Only the journey changes. Before it opens the viewer, a
MutationObserver pauses the fullscreen video as soon as it mounts.
Pausing clears the autoplay flag, so the video never plays. The
observer also resolves `window.fullscreenRestored` on the video's first
`seeked`, which is the restore's. The journey awaits that before its
own seek and no longer calls `pause()` inside the seek evaluate.

The seek now runs on a video that has never played, after the restore
has finished. That avoids (b). It also rules out (a), because the
restore has already landed when the journey seeks. The assertion stays
exact (`toBeCloseTo(…, 1)`). The two inline videos still cover
playback: each must reach 0.2 s and decode a frame. The fullscreen
video's autoplay was never asserted.

There is no product change and no `initialTime` guard in `VideoPlayer`,
because nothing showed the restore landing late. No retries, sleeps or
timeouts change, and no browser cases are added or removed.

## Validation

Local Linux workstation, Playwright 1.63.0, `--no-deps --retries=0`:

- Reload A/B, using a temporary, uncommitted spec: the journey up to
  the fullscreen viewer, then 50 samples per test. Each sample calls
  `load()` on the fullscreen video, waits for `readyState` and
  `seekable` as the journey does, and seeks to half the duration.
  WebKit, 30 repeats, 3 workers per mode, each mode pinned to 4 CPUs
  shared with 2 busy loops:
  - old (autoplay, then `pause()` and the seek in one tick): 3/1,500
    unmoved (`currentTime` 0.091-0.112 against 15), all with the CI
    signature, and 1,500/1,500 playing before the seek.
  - new (`pause()` right after `load()`, then seek after the restore's
    `seeked`): 0/1,500 unmoved, 0/1,500 playing before the seek.
  Three against zero fits (b) but isn't statistically conclusive
  (one-sided p ≈ 0.12).
- Reopen A/B, same setup: the viewer was closed and reopened before
  each of 30 samples per test. The new mode used this change's
  observer. Old: 0/900 unmoved. New: 0/900 unmoved, 0/900 playing
  before the seek. Neither mode reproduced the problem, so this run
  only shows the observer holds the video paused and the restore never
  landed late (0/1,800).
- `bin/pnpm test:browser tests/browser/github-body.spec.mjs --project
  webkit --project chromium --no-deps --workers=1 --repeat-each=10
  --grep "PR media plays"`: 20/20 passed.
- The same journey, WebKit, pinned to 4 CPUs shared with 4 busy loops,
  `--workers=2 --repeat-each=16`: 16/16 passed.
- Full `github-body.spec.mjs`, both engines: 2/2 passed.
- `biome check` and `git diff --check` are clean.

A green run doesn't prove the flake is gone. Hosted CI covers broad
validation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: mtoohey <mtoohey@squareup.com>
(cherry picked from commit c03a0a08deddd973dd29fe6728f9c56f93643f84)
@matt2e @claude
test(channels): deny the retried preference read so recovery can't ra…
63c808b 
…ce the final Retry

## Problem

Code review follow-up to "test(channels): deny the failed preference
read so recovery can't race Retry" (suggestion,
`tests/browser/navigation-groups.spec.mjs` "Create new retains its draft
when preferences fail before submission and recovers in place"). The
journey injects 502s on `sidebar-preferences`, which relay recovery
treats as transient. It clicks "Retry preferences" twice, and the second
click comes right after `page.unroute`. Recovery's next background read
can then succeed and remove the alert before that click lands. That is
the failure mode the earlier commit fixed in
`sidebar-section-creation.spec.mjs`.

## Cause

The timing, from `src/features/relay/recovery.ts` and the journey:

- The first 502 starts recovery. It retries after 1 s and again about
  2 s later (`RECOVERY_DELAYS_MS`), and the still-installed route fails
  both.
- The inline "Reconnecting to relay…" notice the journey waits for
  renders only once `shown` is set, `RECOVERY_NOTICE_DELAY_MS` (3 s)
  after the first failure. So the first manual Retry comes after one or
  two background retries.
- The dialog's Retry calls `preferences.reload`, not
  `recovery.retryNow`. Its read cancels recovery's pending timer but
  keeps `attempt`. When it fails with a 502, recovery schedules its next
  read at `RECOVERY_DELAYS_MS[attempt]`, 2 s or 4 s later.
- The journey then asserts, unroutes and clicks Retry again. If that
  click hasn't landed when recovery's read runs, the read succeeds
  because the route is gone. The alert unmounts with its button, and
  the click times out waiting for it.

The window is 2-4 s rather than the about 1 s of the earlier case, so
this needs a slower click, but nothing in the journey orders it.

## Fix

Keep the first failure a 502, so recovery still owns it and shows the
notice the journey asserts. From the first manual Retry on, the route
answers 403. A denial is not transient: `loadState` reports it as
settled, recovery cancels its timer and stays idle, and the dialog's
Retry is the only read left. After the unroute, the final click is the
only thing that can recover preferences.

The journey still exercises a failed retry that keeps the draft and
Retry preferences. A 403 leaves the dialog unwritable the same way.
The 403s are recorded under `app.report.relayFailures`, the fixture's
allowance for injected relay 403s, and the 502 stays under
`sidebarPreferenceFailures`. The manual retry's response wait now
matches 403. Only the test changes (+14/-4). No assertions are
loosened, and no product code changes or browser cases are added or
removed.

## Validation

Local Linux workstation, Playwright 1.63.0, `--no-deps --retries=0`:

- Adversarial ordering, using a temporary, uncommitted copy of the spec
  that waited between `page.unroute` and the final Retry click to model
  a slow click. Both engines, `--workers=2`:
  - Without the fix, with 4.5 s: 4/4 failed (2 WebKit, 2 Chromium),
    all at the final click with "locator.click: Timeout 10000ms
    exceeded ... waiting for ... Retry preferences".
  - Without the fix, with 1.5 s: 6/6 passed, consistent with the 2-4 s
    window.
  - With the fix, with 4.5 s: 6/6 passed.
- `bin/pnpm test:browser tests/browser/navigation-groups.spec.mjs
  --project webkit --project chromium --workers=4 --repeat-each=10
  --grep "Create new retains"`: 20/20 passed.
- Full `navigation-groups.spec.mjs`, both engines: 20/20 passed.
- `biome check --error-on-warnings` and `git diff --check` are clean.

No CI failure of this journey was looked up. The race was found by
reading the code and confirmed with the forced delay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: mtoohey <mtoohey@squareup.com>
(cherry picked from commit b1eb5a07ec427209e45cfbc51528ce643cd3adfb)
@matt2e
matt2e force-pushed the mtoohey/flaky-journey-sync branch from d5fa4ef to 63c808b Compare November 5, 2026 05:09
kazemek21
kazemek21 approved these changes Nov 5, 2026
View reviewed changes

@kazemek21 kazemek21 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kyle reviewed this PR and the Nexus review-tracker notes and approved with no comments.

Posted by Codex (gpt-6.1-sol) on behalf of kylez

@matt2e
matt2e merged commit 890ca11 into main Nov 5, 2026
47 checks passed
@matt2e
matt2e deleted the mtoohey/flaky-journey-sync branch November 5, 2026 06:17
@matt2e matt2e mentioned this pull request Nov 5, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kazemek21 kazemek21 kazemek21 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@matt2e @kazemek21