Skip to content

feat(unread): restore the Inbox's evidence from a device record on launch - #2407

Open
matt2e wants to merge 2 commits into
inbox-1-channel-windowsfrom
inbox-2-device-record
Open

matt2e wants to merge 2 commits into
inbox-1-channel-windowsfrom
inbox-2-device-record

Conversation

@matt2e

@matt2e matt2e commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Stack

This stack splits #1697, the frozen draft umbrella at 34fc40668, into three layers. Each PR is based on the one before and merges in order. #2408's tree equals #1697's, and #1697 closes when #2408 merges.

  1. fix(unread): keep the viewer's threads in the Inbox under busier channels #2401 (PR 1, inbox-1-channel-windows): reads each channel on its own and remembers your threads, so busier channels no longer push a thread out of the Inbox.
  2. feat(unread): restore the Inbox's evidence from a device record on launch #2407 (PR 2, inbox-2-device-record): saves the Inbox's evidence on the device and restores it on launch. ← this PR
  3. perf(unread): catch the Inbox up from a saved mark instead of re-reading every repair #2408 (PR 3, inbox-3-catch-up): catches up from a saved mark instead of re-reading everything on every repair.

Before and after

#2401 47e9a8031, this PR's base, is on the left and this PR 1c7968418 on the right. Both run the web app in Chromium on a fresh device against one local fixture relay with the same made-up team. The box at each pane's bottom right shows the time since the pane loaded, its relay queries and the bytes downloaded.

Saved Inbox rows within a second of a relaunch, while message reads are slow

After a normal launch, the relay holds message reads for 15 s and both panes reload. This PR shows the rows it saved on the device right away and keeps them through the read error; #2401 shows none until the reads return.

pr2407-s5b.mp4

What this PR adds

This is PR 2 of three. It saves the evidence behind the Inbox on the device and restores it on launch. It does not catch up from a mark: as in PR 1, every repair (launch, reconnect, the 15-minute backstop, and recovery while a repair is owed) reads in full.

  • The record. After a repair, unread saves the signed events the next repair would read of what it holds. It also saves every unread or undecided row and what those rows depend on: roots, parents, the viewer's own messages in their threads, and edits and deletions. The limit is 2,048 events and 4 MiB; a selection over either keeps no events. Roots and parents fetched only to group replies are saved as structure and never count. The record goes in buzz-unread-v1's events store, which PR 1 created empty, so there is no IndexedDB version bump. A PR 1 record has no structure; it reads as its threads alone.
  • Restore. Once a session, the launch verifies the saved events again beside the repair, not ahead of it. It checks 8 events a batch and yields between batches (CPU metric verify.restore). It admits them all once every event has passed. One event failing verification discards them all, keeping the remembered threads. Rows then show from disk before the relay answers, and with the relay down.
  • Reconcile. Restored events only add to what the repair reads:
    • A restored event that a read should have returned and did not, because it was deleted while the device was closed, is dropped.
    • One whose channel membership is not confirmed yet waits for it and is saved meanwhile.
    • Known limit: the repair's reads carry no deletions. So a restored event that no read reaches stays until its deletion arrives. That is an unread message older than its channel's window and outside the roster sample. This is documented in docs/inbox.md.
  • Wipe and fence.
    • Clearing the cache wipes the record, including one the launch was still reading, and the session's clear waits for that wipe.
    • A partition purge records itself with a time:token fence. A session that read before the purge cannot save the left community back, even after reading again. A session started after it, as when rejoining, saves as usual.
  • relayMatches (own commit): reconcile judges a page as the relay matched it. The relay matches #h by the channel it filed an event under, so an event with no h tag matches by its target's channel.

Left for PR 3: complete and the bookkeeping that only serves it: the list of evicted content, and the ungrouped and lost checks. Only PR 3's route() reads it. The test "a selection over the record's bounds, or one retention dropped, saves an incomplete record" moves with it.

One side effect of the deferral: retention no longer holds a filter slot for content evicted during the repair. A selection can therefore keep the next older held event in that slot. That event is real evidence, and on relaunch it sits below the read's cut, so reconcile does not judge it.

Details and limits: docs/inbox.md.

Commits

  1. e72cad769 reapplies git diff 4d3639704 de2d02efb without the hunks PR 1 already carries: keepRoot, the structure-cache order and exact bound, request shapes, storage names, and two docs/inbox.md lines.
    • unread-storage.ts and thread-participation.ts start from de2d02efb's versions.
    • complete is deferred, as above.
    • Tests are the layer-2 tests of 34fc40668, fix(unread): keep participated threads in the Inbox and persist its evidence across launches #1697's test trim:
      • unread-storage.test.ts: 9 tests, with the in-memory IndexedDB kept inline.
      • unread-restore.test.ts: 6 tests, and de2d02efb's harness with floor: () => 0.
      • The deleted-events fixture in unread-repair.test.ts.
      • The reload, refused-write and purge parts of unread-storage.spec.mjs.
    • The layer-3 lines of those tests stay with PR 3: complete, the mark and its entry, roster, whole, through, absent, sameMeta and the delta.
  2. 1c7968418 adds relayMatches, moved from da8ed62cc. In this layer, every read reconcile judges asks for content kinds alone (kinds 9, 40002 and 40008, confirmed by the trace check below), so no response yet holds an event with no h tag. The test that shows the drop reads the catch-up delta and stays in PR 3.

Behaviour and cost against PR 1

PR 1 this PR measured at
Online reads per launch — identical 1c7968418 vs 47e9a8031: scripted reader in Vitest, 7 launches, 31 requests and 73 filters each side (see Checks). Not a live relay.
Relay bytes and DB time as PR 1 unchanged Follows from the identical reads; not measured separately.
Warm or offline launch rows only once the relay answers rows from disk, before the relay answers and with it down Behaviour, covered by unread-restore.test.ts. Time to first row: not measured.
Verifying a 2,048-event record — ~0.56 s CPU in 256 batches of ~2.2 ms; Chromium 561 ms (566 ms wall), WebKit 546 ms (601 ms wall) 1c7968418. Restore's own loop (eventDto over a JSON clone, 8 a batch, yieldToHost between). Headless Playwright Chromium and WebKit on an Apple M3 Max, macOS 27.2. 2,048 synthetic signed kind-9 events of 573 B each. Medians of 5. Not the desktop WKWebView.
One save of that record — 1.17 MB of events plus a 5.6 KB meta. A first save into an empty partition takes 38 ms (Chromium) or 66 ms (WebKit). A save that adds one event and drops one takes 3.9 or 18 ms. Reading it back takes 11 or 23 ms Same build, setup and medians, with real IndexedDB through browserUnreadStorage.
Record size on a real roster — not measured —

A save writes only what changed against the partition's record: the meta, a put for each event the partition lacks, and a delete for each it no longer holds. A save identical to the last issued one is skipped. Restore runs once a session.

FOUNDATION

src/features/relay/session.ts is FOUNDATION. This PR changes 3 lines (+5/−2): const unreadWiped = unread.clear();, and unreadWiped joins channels.clearCache() in the clear's Promise.allSettled. A cache clear therefore settles once the record is wiped, and rejects if the wipe fails. These are #1697's lines from 9815e47a0, moved here as part of the split. Matt Toohey approved them before the change. Nothing else in session.ts changes.

Checks

All of these were run on 1c7968418:

  • Diff against de2d02efb, git diff de2d02efb HEAD -- src tests over the branch's files:
    • Production:
      • fold.ts +4/−2: PR 1's per-channel fold.
      • unread-storage.ts +6/−17: the complete deferral only.
      • unread.ts +87/−60: the per-channel fold (byTarget, foldable), owing = unchecked, the exact structure-cache bound (heldRoots, rememberedRoot), relayMatches, and the complete deferral.
      • Identical to de2d02efb: session.ts, partition-purge.ts, thread-participation.ts, reader.ts, inbox.ts, unread-window-batch.ts, errors.ts, client-metrics.ts and InboxPage.tsx.
    • Tests are carved from 34fc40668, so the diff against de2d02efb shows its trim. Every test title here exists in 34fc40668, except:
      • PR 1's three: the two keepRoot tests and the window-drop test.
      • Four layer-2 halves of 34fc40668 tests whose layer-3 clauses are removed. They take de2d02efb's titles or close to them.
    • Files outside the branch's set differ only because main changed them.
  • Request-trace equality with PR 1. The probe was a throwaway Vitest file, not committed:
    • Setup: the unread-restore.test.ts harness with fixed keys.
    • Stores: three from those tests. They are the relaunch store (a lost thread, a mention, a follow, a quiet channel, a DM, R1 and 500 busy events), a lookup-fetched root, and a deleted reply.
    • Launches: 7 online ones (fresh, warm, warm again; lookup first and warm; deletion first and warm). Each records every reader.read filter list.
    • Waiting: on layer 2 it waits until restore has verified the whole record (512, 513, 502 and 12 events) and the reads have gone quiet.
    • Comparison: it ran at HEAD, then again with unread.ts and unread-storage.ts checked out from 47e9a8031. The recorded calls are identical, 31 requests and 73 filters each, all kinds 9, 40002 and 40008.
    • Not compared: the whole app's /query trace in a browser, and a live relay.
  • bin/pnpm typecheck: clean.
  • biome check --error-on-warnings on the 8 changed code files: clean.
  • git diff --check 47e9a8031 HEAD: clean.
  • Vitest on src/features/relay, src/bundled/inbox, src/features/notifications, src/features/messages and src/features/communities: 274 files passed and 1 skipped; 6,146 tests passed and 4 skipped.
  • Playwright --no-deps on unread-storage and PR 1's eight (sidebar-unread, message-navigation, scroll, navigation-thread-history, inbox, inbox-edit, thread-unread, channel-lifecycle), one engine after the other: Chromium 77 passed (2.5 min) and WebKit 77 passed (3.2 min), with no timeouts or retries.
  • Pre-commit hooks passed on both commits. The push used --no-verify.

Browser cases. One added, unread-storage.spec.mjs: "unread record survives reload, keeps the last record over a refused write, and is purged with only its partition".

  • Why a browser test: it needs real IndexedDB across a reload, two windows' connections to one partition, and a transaction aborted by an uncloneable value. The in-memory IndexedDB in unread-storage.test.ts stands in for all three.
  • Fail-then-pass: with the purge fence's refusal removed from save(), it fails at unread-storage.spec.mjs:129 ("purged by another window"). On PR 1's unread-storage.ts it fails at the first record read.
  • None removed.

Not verified: the desktop app (restore timing in WKWebView, the native host), a live relay, record sizes on a real roster, and just scan. Agent review has not run, nothing was exercised in the running app, and there has been no human test, so this stays a draft.

🤖 Generated with Claude Code

matt2e and others added 2 commits November 5, 2026 17:24
…unch

Reapply layer 2 of the Inbox split (de2d02e over 4d36397) on top of
the channel windows, leaving out what layer 1 already carries.

The unread record in buzz-unread-v1 now also holds the signed evidence
behind the Inbox's rows and the structure that groups it. On launch,
restore verifies it again in batches of 8 (verify.restore) and admits
it whole, or discards it all if any event fails. Rows then show before
the network answers, and with the relay down. Reads that should have
returned a restored event and did not drop it. A cache clear wipes the
record, and a partition purge fences any session that read before it,
so a stale window cannot save a left community back.

The complete flag, and the bookkeeping that only serves it (the dropped
list, the ungrouped and lost checks), stay with the catch-up layer,
which is its only reader.

Tests are the layer-2 tests of the trimmed head (34fc406):
unread-storage and unread-restore, the deleted-events fixture in
unread-repair, and the reload, refused-write and purge journey in
unread-storage.spec.mjs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
…ed them

The relay matches `#h` against the channel it filed an event under. For
an event with no `h` tag, such as a reaction taken off (an h-less kind
5), that is its target's channel, which the client cannot see.
Reconcile now counts such an event in a channel-scoped filter's page.
Otherwise it leaves a full page looking short, and restored evidence
below the page's cut is dropped though the relay still holds it.

Every read reconcile judges in this layer asks for content kinds (9,
40002, 40008) alone, so no response holds such an event yet. The test
that shows the drop (da8ed62) reads the catch-up delta and stays with
that layer. This moves here from da8ed62 because it belongs to
reconcile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
@matt2e
matt2e force-pushed the inbox-2-device-record branch from f8e0f0e to ca55105 Compare November 5, 2026 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant