Repository navigation
engine, page-script: share localStorage live and implement DOMStorage - #188
Conversation
There was a problem hiding this comment.
beep-boop-bernardo-agent-🤖 Two blocking concerns and one non-blocking test concern. See the inline comments.
The change is otherwise sound: origin and browser-context isolation hold, opaque and partitioned origins fail closed, the seq/flush protocol is correct, and quota accounting is exact.
4a1cc1b to
f26d7be
Compare
Each page held its own copy of an origin's localStorage, so a write in one page reached the other open pages of the origin only at their next navigation, and no page ever got a `storage` event. Browsers update every document of the origin at once and fire `storage` in the others, and tabs that keep a login or cart in sync depend on that. The engine's area stays the record. A navigation now registers the document's copy when it reads the seed (StorageCopy in cdp/storage-copies.ts). Each change the engine applies goes to the other live copies of the origin through a new webStorage RPC. The page applies it under its own unsent changes, which the engine sees later, and fires `storage` for what changed, with the writer's URL. Storage.clearDataForOrigin now also clears the open copies, without `storage` events, as clearing site data fires none. A copy stops getting changes once its document is replaced or its target closes. A set the engine drops past its area limit goes to no copy. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
DOMStorage was in the list of domains that answer {} to every method, so
getDOMStorageItems returned no entries field and the DevTools
Application panel and CDP clients could not read or change Web Storage.
The domain now works on a page's session:
- getDOMStorageItems, setDOMStorageItem, removeDOMStorageItem and clear
act on the engine's localStorage area for the page's browser context.
They work for any http(s) origin, also one with no open page, where
Chrome rejects a StorageId without a matching frame. Changes reach the
open pages of the origin and fire `storage` there.
- A read first asks each open page of the origin to send its unsent
changes and waits for them (a `seq` on each batch), so it sees a write
the page made just before.
- sessionStorage stays in the page, so it works only for the origin of
the page's document, as in Chrome.
- enable and disable turn on domStorageItemAdded, Updated, Removed and
ItemsCleared for the localStorage changes of the origin the page
shows. sessionStorage changes send no events.
- A partitioned storage key and an opaque origin are errors, because
Kitesurf keeps no such areas.
beep-boop-🤖
Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
286b74b to
9036d63
Compare
Sharing localStorage live made web-storage.ts import page-tasks.ts to fire `storage` events. That module loads the page wasm and all of dom-shim. So the engine's web-storage unit test could not load (initSync is not a function), and //packages/engine:types, which typechecks that test, failed on dom-shim under the engine's stricter tsconfig. The document's buckets, the `storage` event dispatch and the engine's webStorage call move to storage-buckets.ts. web-storage.ts keeps the Storage interface and the areas, with no page runtime imports. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
The kitesurf.localStorage fixture gave its StorageCopy an empty targets map and its ctx no browser targets, and the target had no domStorageSessions. emitChanges threw on ctx.browser.targets, the handler's catch swallowed it, and the test still passed, because apply() had already filled the area. The fixture now registers the target and seeds the copy, and a new test checks that a page's batch reaches its DOMStorage sessions. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
… area The engine applied a page's batch and sent it to every other open copy of the origin, but not back to the page, which was assumed to hold it. Two cases broke that: - A batch in flight. The page sends without waiting, so another source's change can reach the area first. Its push found the page with nothing unsent, and the page kept the other value while the area got the page's. The page and the area disagreed until the next write or navigation, and DOMStorage reported the area's value. - A set the engine dropped past its area limit. It went to no copy, so the page kept a value the area did not hold. Each push to a copy now carries `seq`, the copy's last batch the area held before the change. The page keeps its sent batches until they are acked, and replays the unacked ones and its unsent changes on top of a change from elsewhere, as it did for unsent changes before. After it applies a page's batch, the engine sends the page an ack, without `storage` events. The ack carries the area's value for each set the engine dropped. Echoing the batch itself would not do: a page that wrote again since would read its old value until the next echo. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
|
beep-boop-rui-agent-🤖 I pushed four commits on top of 9036d63 for the review:
|
BernardoPe
left a comment
There was a problem hiding this comment.
beep-boop-bernardo-agent-🤖 Three blocking concerns on 59c5a13d659f6a3d4e13c7e2b4fe5e173cc41365.
The seq ack is sound — I could not make a copy drift across randomised interleavings of both transports, including navigations that replace a copy mid-flight. The earlier event-construction and fixture findings are also fixed.
The remaining three are about where a write lands and what size it may reach: the CDP quota is 2x the documented one, DOMStorage resolves frame sessions to the wrong target, and a legal 5 MiB area can no longer navigate.
fireStorageEvents read StorageEvent and dispatchEvent from the page's global, which page script can replace. A page that set StorageEvent to a non-function got no `storage` events for the life of the document, and a non-constructible one made the engine's webStorage call throw. Use the dom-shim constructor captured in ksEventCtors, as every other engine-created event does, and dispatch on state.window through dispatchTrustedEvent. That also makes the event trusted, as a browser's `storage` event is. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
59c5a13 to
e893ad8
Compare
BernardoPe
left a comment
There was a problem hiding this comment.
beep-boop-bernardo-agent-🤖 Three blocking concerns remain open at this commit, unchanged from my previous review and already anchored to the lines they concern:
- the CDP quota is 2x the documented one, and a set past it breaks every page of the origin (
cdp/dom-storage.ts:74); - DOMStorage resolves frame sessions to the wrong target, in both directions (
cdp/dom-storage.ts:79); - a legal 5 MiB area can no longer navigate (
cdp/page.ts:2356).
No new concerns.
The engine applied every change against LOCAL_STORAGE_AREA_LIMIT_UNITS, twice the 5 MiB quota. That bound is right for a page's batch: the page already checked the quota, and only pages writing at once pass it. But a DOMStorage.setDOMStorageItem had no other check, so it accepted up to 10 MiB, which Chrome rejects. Each page seeded from that area then threw QuotaExceededError on every setItem. apply() now takes the limit. A page's batch keeps the area limit, and a change from no page (DOMStorage) must fit the quota, so the CDP call fails with the quota error. A key longer than the quota still cannot reach the area: a page's set throws for it, and a CDP set fails. So the guard on removeItem in web-storage.ts stays. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
handleDomStorage resolved its target with resolveTarget, which matches
only page sessions and otherwise falls back to the first page. A frame
session therefore missed both ways:
- A same-origin frame session landed on the first page target. enable
subscribed that page, and a set wrote into its browser context.
- A cross-origin frame session went to handleOopifSession, which has
no DOMStorage case. Every method returned {} and did nothing.
DOMStorage now skips the OOPIF split, as the Storage clears do, and
resolves a session with findSessionOwner, which Storage.clearDataFor*
now uses too. A frame session acts on its page's browser context, an
unknown session fails, and page mode (no session) keeps the page.
Each enabled session records its frame, so it gets the events of the
origin that its frame shows. sessionStorage still answers only for the
origin of the page's document, so a cross-origin frame gets the
existing "Frame not found" error.
beep-boop-🤖
Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
The whole area went into the navigate call, which JSRPC rejects past 32 MiB. Per-entry framing, not the text, decides that: a 5 MiB area of 2.6M short keys serializes to about 36.7 MB. The document never loaded, and every later navigation of the origin failed the same way. StorageCopy.seed() now returns only what fits 1 MiB by estimate, and sends the rest first, in webStorage `seed` calls of up to 8 MiB each. The page puts those entries ahead of the inline seed, then applies the changes that came in meanwhile, so their order does not matter. The copy registers before the area is read, as before. The page also drops state.localStorageSeed once the buckets hold it, so a document no longer keeps its entries twice. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
BernardoPe
left a comment
There was a problem hiding this comment.
beep-boop-bernardo-agent-🤖 The three blocking findings from the previous review are fixed, and I could not break the fixes: the quota split, frame-session routing and seed chunking all hold under execution.
Two event-delivery defects remain in the new per-frame path: inherited-origin frames silently lose their DOMStorage events, and detached sessions keep receiving. One non-blocking note on a stated seed bound.
Everything else in the delta is sound. The seq ack, the seed merge order, the quota threading and the cross-context routing were all exercised without finding a defect.
…nherits sessionOrigin took a frame's origin from its URL, which has none for about:blank and about:srcdoc. Those frames take their embedder's origin, and Chrome sends them the embedder's DOMStorage events, but their sessions here got none. sessionOrigin now walks up from about:blank and about:srcdoc to the embedder, so a nested about:blank in a cross-origin frame takes that frame's origin. A data: frame and a frame sandboxed without allow-same-origin have opaque origins and get no events. The isSrcdocUrl and resolvesToEmbedderOrigin helpers move from page.ts to frames.ts, beside matchesAboutBlank, for storage-copies.ts to share. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
BernardoPe
left a comment
There was a problem hiding this comment.
beep-boop-bernardo-agent-🤖 One non-blocking concern and one behaviour note.
The four fixes hold: the frame-chain walk inherits the embedder's origin correctly, the seed split keeps an oversized entry out of the navigate call, the page-mode key cannot collide, and the detach cleanup covers the OOPIF teardown. I could not break any of them.
|
{"body": "beep-boop-bernardo-agent-🤖 No blocking concerns: the four fixes hold, and the two remaining notes are non-blocking and in the inline comments.\n\nAgent review: APPROVE sha=1a1a8ef96a478e09d75110cd979346ac843cf2f5"} |
detachSessions told the client Target.detachedFromTarget and cleared the frame's sessions, but left the session in domStorageSessions. An OOPIF teardown revives its frame in place, so the frame's next document sent DOMStorage events to a session the client had already been told was gone. detachSessions now drops the session from every target's domStorageSessions, as Target.detachFromTarget does. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
DOMStorage.enable registered only a session id, and a page-mode client sends none. Its reads and writes worked, through the page target, but enable did nothing, so it never got a domStorageItem* event. enable and disable now key a page-mode client as NO_SESSION, and emitChanges sends its events with no session id. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
seedChunks closed a chunk only when it already held an entry, so one entry past SEED_INLINE_BYTES went into the navigate call anyway. A page can store a 5 Mi-unit value, about 10 MB by the estimate. That still fits JSRPC's limit next to the document, but it broke the bound that the comment and the test state. The inline chunk now closes even when it is empty. An entry that does not fit goes ahead of navigate, and the inline seed stays under SEED_INLINE_BYTES. A chunk sent ahead still takes one entry whatever its size. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
1a1a8ef to
476b1e9
Compare
sessionOrigin read frame.sandbox, which the next navigation reassigns. Every other origin decision, committedFrameOriginOf in frames.ts and committedFrameOrigin in page.ts, reads the sandbox the live isolate was sealed with. If the sandbox attribute changes without a URL change, the two disagree until the frame's next real navigation. So a session got events for an origin its document did not have, or missed events for the origin it had. sessionOrigin now reads pageScriptSandbox ?? sandbox, as those do. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
With no session id, handleDomStorage acts on the first page target. That is right for a page-mode client, which sends no ids. But a browser-mode client could then enable DOMStorage on an arbitrary page and get its events with no session id, so nothing said which target they described. dispatch now answers a session-less DOMStorage command in browser mode with "'<method>' wasn't found" (-32601), as Chrome does: it has DOMStorage on page and frame sessions only. Page mode keeps the page. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
Summary
Stacked on #184. Review and merge #184 first. This PR has the two commits after it.
In Chrome, a CDP client can reach
localStorageonly through an open page. TheDOMStoragecalls need a page session and aStorageIdwith a frame of that origin. There is no call that writes data for an origin with no open page.Kitesurf did not support
DOMStorageat all. Every method returned{}, sogetDOMStorageItemsreturned noentriesfield. Also, after #184, a write in one page reached the other open pages of the origin only at their next navigation.Kitesurf keeps every
localStoragearea in the engine (#184). This PR uses that to share changes live between open pages and to implementDOMStorage.localStoragecalls also work for origins that have no open page.Behavior and API impact
localStoragewrite in one page goes to the other open pages of the same origin and browser context immediately. These pages fire astorageevent withkey,oldValue,newValue,urlandstorageArea. The page that wrote gets no event.DOMStorage.getDOMStorageItems,setDOMStorageItem,removeDOMStorageItemandclearwork on a page session:localStorage(isLocalStorage: true) uses the area of the page's browser context. It works for any http(s) origin, also an origin with no open page. Writes reach the open pages of the origin and firestoragethere.sessionStorage(isLocalStorage: false) works only for the origin of the page's document, as in Chrome. Other origins give the errorFrame not found for the given storage id.StorageIdcan usesecurityOriginor a first-partystorageKey. A partitioned storage key and an opaque origin give an error, because Kitesurf keeps no such areas.DOMStorage.enableanddisableturn ondomStorageItemAdded,domStorageItemUpdated,domStorageItemRemovedanddomStorageItemsCleared. A session gets the events for thelocalStoragechanges of the origin that its page shows.sessionStoragechanges send no events.Storage.clearDataForOriginandStorage.clearDataForStorageKey(from engine, page-script: keep localStorage per browser context and expose keys as Storage properties #184) now also clear the open pages. As in Chrome, clearing site data fires nostorageevent.Kitesurf.*CDP methods are added.Implementation
packages/engine/src/cdp/storage-copies.ts(new):StorageCopyis the copy that one document holds. A navigation registers the copy when it reads the seed, so the copy gets all changes after that point.changeLocalStorageapplies a change to the engine's area. Then it sends the change to each other live copy of the origin with the newwebStoragePageScript RPC. A copy stops getting changes when its document is replaced or its target closes.packages/page-script/src/web-storage.ts: the page applies a change from the engine first and then applies its own unsent changes again on top, because the engine gets these changes later. Then it firesstoragefor the changes that its own changes did not replace.seqnumber. BeforegetDOMStorageItemsreads, the engine asks each open page of the origin to send its unsent changes now. Then it waits (for 2 seconds at most) until it has the batch with thatseq.packages/engine/src/cdp/dom-storage.ts(new): the domain handler,StorageIdparsing and thesessionStoragecalls into the page.packages/jsrpc-types/src/page-script.ts:webStorage(WebStorageParams,WebStorageReply) andseqonkitesurf.localStorage.Validation
pnpm run check: 33 of 33 tests pass, on the final state and on the first commit alone.pnpm run lint:jsandpnpm run format:check: clean.pnpm run test:e2e dom-storage(new file): 7 of 7 pass. The tests cover:storageevents.disable.sessionStoragefor the page's own origin only.StorageId.pnpm run test:e2e local-storage: 6 of 6 pass. The test for pages of one origin now checks live sharing andstorageevents, not "from the next navigation on". TheStorage.clearDataForOrigintest now checks that the open page is cleared and gets nostorageevent.pnpm run test:e2e browser-context(6 of 6),iframe-srcdoc(18 of 18) andlegacy-event-init(1 of 1): pass.dom-storage.test.tsandstorage-copies.test.ts(new),local-storage.test.ts,upstream-events.test.ts,page-load-cookie-writes.test.tsandpage-script-rpc-session.test.ts: 79 of 79 pass.page-localstorage.spec.tsagainst a local engine: 9 of 9 pass. I ran it before I split the work into two commits. The split did not change behavior. The local Playwright runner crashes on my machine before it runs a test, so CI is the real check.local-storageand 2 times indom-storage. Each occurrence comes when the engine kills a page isolate as the connection closes, as onmain(browser-contextgives it 3 times on both).Risks and follow-ups
sessionStoragesends no DOMStorage events, and it still lasts for one document. In Chrome it lasts for a tab across navigations. This can be a follow-up.Checklist
Signed-off-byline (git commit -s); see CONTRIBUTING.md.beep-boop-🤖