Repository navigation
engine, page-script: keep localStorage per browser context and expose keys as Storage properties - #184
Conversation
BernardoPe
left a comment
There was a problem hiding this comment.
beep-boop-bernardo-agent-🤖 Reviewed 15 files at 795607f with five review lenses and two breaker runs. Three blocking findings, posted inline. Nothing was built or run against the engine; the checks were static reading plus small Node harnesses on the copied modules.
localStorage lived in each page isolate, so every navigation (a fresh isolate) emptied it, and two pages of one origin never shared it. A browser keeps one area per origin for the whole browser context, and logins kept in localStorage depend on that. It is also what a storageState export has to read. The engine now keeps one area per origin and browser context in a LocalStorage over a SessionStore (a MemorySessionStore for now). A navigation to an http(s) origin hands the page a copy, and the page sends each change back as a kitesurf.localStorage upstream event, batched per microtask. Pages of one origin see each other's writes from their next navigation on, not live. The handler applies writes before the generation check, because a write the old document made just before a navigation still happened. The engine drops a set that would pass the 5 MiB per-origin quota, since two copies of one area could otherwise fill it twice. Only the PageScript navigate() options of a main-frame document carry the seed. The other cookieKey sites in page.ts are the document fetch, the parser and speculative script prefetch, the module graph walk, the wasm prefetch and the isolate cache key: none of them builds a document's NavigateOptions. Child frames (oopifSpawnArgs) and the lazy about:blank isolate get no seed, so their localStorage stays in the page, and an opaque sandboxed origin still throws SecurityError. sessionStorage stays in memory for one document. Part of #133 beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
795607f to
fa36430
Compare
BernardoPe
left a comment
There was a problem hiding this comment.
beep-boop-bernardo-agent-🤖 Follow-up review of the new head fa36430. The PR diff is unchanged from the reviewed 795607f (byte-identical after the rebase), so this pass covered only the 66 new lines: Storage.clearDataForOrigin and clearDataForStorageKey, the new e2e case, the Playwright fixture and the expectation rebaseline. The three earlier findings still stand at this head. Three findings on the new code, posted inline. Checks were static reading plus Node harnesses on copied functions; nothing was built or run against the engine.
localStorage and sessionStorage were plain objects whose own properties
were the methods, so Object.keys(localStorage) listed getItem, setItem
and the rest, and { ...localStorage } held only length. Agents dump page
state that way and got method names with no values.
Storage is now a real interface: the methods and length live on
Storage.prototype, and each instance is a legacy platform object with
named properties (WebIDL). Stored keys read, write, delete and
enumerate as own properties, unless the prototype chain has the name,
so a key called getItem never hides the method. new Storage() and a
method called on another object throw TypeError.
On boa, a for-in over the bridged object visits only the stored keys,
not the enumerable prototype members; the e2e test pins that.
Fixes #133
beep-boop-🤖
Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
Storage.clearDataForOrigin and Storage.clearDataForStorageKey answered
{} and cleared nothing. Now that localStorage lasts for the browser
context, a client needs them to start an origin from empty, and the
Playwright fixture needs them to reset storage between tests.
With "local_storage" or "all" in storageTypes, they clear the origin's
area in the browser context of the session's target, or in the default
context for the browser session. The other storage types stay no-ops,
and a partitioned storage key clears nothing, because Kitesurf keeps no
partitioned areas. Open pages see the clear from their next navigation
on, as they see other pages' writes.
beep-boop-🤖
Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
The kitesurf fixture runs every test in one shared browser context and clears only its cookies between tests. localStorage now lasts for the context, so a key from one test reached the next one on the same origin: page-localstorage.spec.ts "localStorage.removeItem removes a single item" found the keys of the tests before it. After each test the fixture now clears localStorage for the test server's two origins with Storage.clearDataForOrigin, as it clears the cookies. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
page-localstorage.spec.ts "storage methods are scoped to the current origin" writes a key, navigates to another origin and back, and expects the key. It failed while every navigation emptied localStorage, and it passes now that the browser context keeps it. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
fa36430 to
4a1cc1b
Compare
Page.navigate to about:blank and Page.setDocumentContent on a blank page render under https://kitesurf.local/, the URL sanitizeBaseUrl gives documents with no hierarchical URL. localStorageOrigin read that URL as a real origin, so every blank document of a browser context was seeded from, and wrote back into, one shared area. Those documents have opaque origins. localStorageOrigin now treats the stand-in origin as opaque too, so their localStorage stays in the page, as the lazy about:blank path already did. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
The page checks the 5 MiB quota against its own copy and throws QuotaExceededError there. The engine checked it again against its area and skipped a set that did not fit. When another page of the origin had written since this page was seeded, the two counts disagreed: setItem returned normally, the page read the value back, and the next navigation lost it with no error. The page's check is the quota script can see, so the engine now applies whatever the page accepted. The area passes 5 MiB only when pages of one origin write at the same time, and the next copy seeded from it refuses to grow. A set past twice the quota is still dropped, with a warning, as a memory bound. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
A page sent every localStorage change of one microtask as one kitesurf.localStorage event, with no bound on its size. The quota does not bound it: replacing one key frees the old value each time, so nine 4 MiB setItem calls in one task built a 37 MB batch. JSRPC rejects a message over 32 MiB, the send is fire-and-forget, and the whole batch was lost although each setItem had succeeded. A batch now flushes early before it passes 10 Mi code units (20 MiB serialized at most), and a clear drops the ops queued before it. One op always fits: a set past the quota throws, and a remove whose key is longer than the quota is not sent, since no area can hold that key. beep-boop-🤖 Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
Storage.clearDataForOrigin and clearDataForStorageKey answered success
in three cases where they did not do what the caller asked:
- "cookies" and "all" left the origin's cookies in place. They now
remove every cookie a request to the origin's host would carry.
- A session the engine did not know, such as a closed page's, or a
frame session outside the page scan, cleared the default browser
context. Frame sessions now resolve to their page, a live OOPIF
session included, and an unknown session fails as in Chromium.
- No valid storage type, an origin that is not a URL, or a storage key
that does not parse cleared nothing and returned {}. They now fail
with Chromium's messages. A partitioned key is still valid and
clears nothing, because Kitesurf keeps no partitioned data.
beep-boop-🤖
Signed-off-by: Rui Figueira <rfigueira@cloudflare.com>
4a1cc1b to
f26d7be
Compare
Summary
Fixes #133. The issue reports two
localStorageproblems:Object.keys(localStorage)gives the method names, not the stored keys.Both came from the in-memory stub in
packages/page-script/src/globals.ts. Each document got a new plain object, and the methods were its own properties.This PR keeps one
localStoragearea per browser context and origin in the engine. It also makesStoragea real interface, so the stored keys are the named properties of the object.Behavior and API impact
localStoragestays for the life of the CDP connection, per browser context and http(s) origin. A reload, a navigation and a new page of the same origin see the stored values.storageevents between pages.localStorage.Target.disposeBrowserContextdeletes it.sessionStoragedoes not change. It stays in memory for one document.allow-same-originstill getsSecurityError.data:andabout:blankkeeplocalStoragein the page.localStorageandsessionStorage.Object.keys, spread,JSON.stringify,ls.k = "v",delete ls.kandObject.definePropertyact as in Chrome. The methods andlengthare onStorage.prototype. A stored key with the same name as a method does not hide the method.new Storage()and aStoragemethod called on another object throwTypeError.key(i)gives keys in insertion order. Before, keys that look like integers came first.AGENTS.md: the "out of scope" list now nameslocalStorageper browser context as the exception to "no storage across navigations".Implementation
packages/engine/src/state/local-storage.ts(new):LocalStoragekeeps the areas in aSessionStore, one table per origin. The default context and each browser context have one. The store is aMemorySessionStore, so nothing outlives the connection.packages/engine/src/cdp/page.ts: a main-frame navigation to an http(s) URL sends a copy of the origin's area inNavigateOptions.localStorage. The engine reads the copy after the old document unloads, so the last writes of the old document are in it. Child frames and the lazyabout:blankisolate get no copy.packages/page-script/src/web-storage.ts(new): with a copy,localStoragesends each change back as akitesurf.localStorageupstream event, one batch per microtask. The engine applies these changes before its generation check, because a write that a document made just before a navigation still happened.readLocalStorageOpsdrops malformed changes, because upstream events are page data.Storageis a class with the methods on its prototype. Each instance is aProxythat implements the named-property rules of WebIDL for legacy platform objects.packages/jsrpc-types/src/page-script.ts: the wire types (NavigateOptions.localStorage,LocalStorageOp, the upstream event).Validation
pnpm run check: 33 of 33 tests pass.pnpm run lint:jsandpnpm run format:check: clean.pnpm run test:e2e local-storage(new file): 5 of 5 pass. The tests are:localStoragesurvives navigations per origin, andsessionStoragedoes not.localStoragefrom their next navigation on, andremoveItemandclearreach keys that the copy of the page did not have.localStorage.Storage, both in a page script (V8) and in CDP evaluation (boa).pnpm run test:e2e browser-context: 6 of 6 pass.pnpm run test:e2e iframe-srcdoc: 18 of 18 pass. This file contains the opaque-originSecurityErrortest.pnpm run test:e2e legacy-event-init: 1 of 1 pass.pnpm run test:wpt --filter webstorage: 3 of 3 match the expectations.local-storage.test.ts(new),upstream-events.test.ts,page-load-cookie-writes.test.ts,page-navigation-origin.test.ts,background-drain.test.tsandruntime-cookie-writes.test.ts: 73 of 73 pass.main.browser-contextgives it 3 times onmainand 2 or 3 times on this branch.Risks and follow-ups
for...inoverlocalStoragegives only the stored keys, not theStorage.prototypemembers. The e2e test records this difference.resolveSessionStoreyet. A host store that persists would makelocalStorageoutlive the connection, and this needs persistent sessions (engine: split the browser session from its CDP client #182). When a connection resolves one store,LocalStoragecan use that store without other changes.localStorage: it isfake-indexeddbin each page isolate. It also lets an opaque origin callopen(), where Chrome throwsSecurityError. This is a separate follow-up.DOMStorage.*andIndexedDB.*stay stubs that return{}.Checklist
Signed-off-byline (git commit -s); see CONTRIBUTING.md.beep-boop-🤖