Using a shared Worker (instead of SharedWorker)

[rhashimoto]

TL;DR It's possible to share an OPFS database connection across multiple browser tabs. There's a toy demo page to show how this could be done, working around the lack of OPFS in SharedWorker. I think this will turn out to be the best way to use OPFS.

Earlier I outlined an access handle pooling approach to take advantage of the new more synchronous access handles of the Origin Private File System API (OPFS). The result is a completely synchronous VFS that doesn't need Asyncify or SharedArrayBuffer to bridge to asynchronous calls. A proof-of-concept implementation works quite well - I think this is as fast as OPFS can go, which is also faster than IndexedDB except for write transaction overhead and perhaps on concurrent reads. The main drawback of this VFS is it only supports one database connection, and while multiple browser tabs could detach and reattach their VFS cooperatively, that would be expensive, probably prohibitively expensive. What can we do if we want to support access from multiple contexts?

An alternative way to support multiple tabs accessing the same database without detach/attach on every transaction is to multiplex requests from tabs onto a single connection, i.e. handling arbitration at the application level instead of inside SQLite. The question is where would the database live? Ideally the database could go into a SharedWorker but unfortunately that won't work because OPFS FileSystemSyncAccessHandle isn't available in SharedWorker. In order to share access to an OPFS-based SQLite database, the database has to live in a regular Worker belonging to one of the tabs.

Sharing a Worker on one tab with other same-origin tabs seems like a real headache. How do you decide which tab is the one that exposes its Worker? What happens if the user closes that tab? Fortunately, however, getting this to work turns out to be easier than expected if you take advantage of some of the newer browser APIs.

There's a proof of concept for sharing a simple arithmetic service across multiple tabs without putting the service in SharedWorker at this link (source). Try it by opening the page on a few browser tabs. The first tab you open will initially be designated as the service provider. If you close or reload that tab, another one will automatically become the provider. Clicking the buttons will make calls to the service from the clicked tab to the provider tab (including when both are the same tab). Use the slow operation button to verify that overlapping calls works, and that an exception is thrown if the service moves during a call (a real app would retry on this exception). Going from this toy page to a shared database connection shouldn't be difficult (proof left as an exercise for the reader demos).

How does this work? It does actually use a small SharedWorker to pass MessagePorts between tabs, but it is not involved with calls to the service (an alternative implementation replacing SharedWorker with a service worker is linked later in this thread). Web Locks turn out to be a great way to watch the lifetime of a tab and initiate the migration of the service to a new tab. And BroadcastChannel tells all the tabs to execute that migration. Those mechanics are only a couple hundred lines of code, and that includes providing a (simplified) Comlink-style proxy so the application doesn't have to deal with raw messages.

The OPFS access handle pooling approach now works on all the major browsers. With this option to support multiple browser tabs, I think this is going to be the best way to implement an OPFS-backed database. IndexedDB will likely remain superior for applications prioritizing lower write transaction overhead, the option to trade durability for performance, concurrent reads, and browser support. Everything else can enjoy OPFS faster I/O, no Asyncify, no COOP/COEP headers, and never invalidating the read cache.

[tantaman]

Wow, this is great work.

The main drawback of that approach is it only supports one database connection

I think the biggest drawback here is related to transaction management. You'd need to do some locking to exclude one tab (or worker) from starting a transaction while another tab has an open transaction. If one tab has a long running transaction then that could starve other tabs or workers.

Each worker/tab having their own connection allows long-ish read transactions to at least time-slice their reads so each reader can jointly make progress. A transaction locking mechanism over a single connection would preclude this.

Maybe this drawback isn't that big of a deal though. Were you thinking of other downsides to only having a single DB connection?

I've been assuming that many connections doesn't really buy much in the browser since:

• Many connections to the same DB from the same thread can't improve concurrency given the WASM instance is constrained to running in a single thread.
• In some not very in-depth testing, spinning up many instances of the WASM module in many threads and connecting to the same DB file in each, reads appeared to be doing time-slicing rather than being truly concurrent.

[rhashimoto]

I think the biggest drawback here is related to transaction management. You'd need to do some locking to exclude one tab (or worker) from starting a transaction while another tab has an open transaction.

Yes, that's correct - the application has to arbitrate access. I don't think this is very difficult, though. I think you can approach it in one of two ways:

1. Each call to the database service consists of a complete transaction (or multiple complete transactions), and the service manages a request queue and processes them sequentially, or
2. By convention, each tab acquires a Web Lock to make calls to the database service, and each "session" begins with a command to rollback any incomplete write transaction in a previous session.

If one tab has a long running transaction then that could starve other tabs or workers.

That's true, but it's only a difference with read transactions. Write transactions will block other tabs even with multiple connections.

Each worker/tab having their own connection allows long-ish read transactions to at least time-slice their reads so each reader can jointly make progress. A transaction locking mechanism over a single connection would preclude this.

I think you would be able to do this time slicing with a variant on (2) above, where readers acquire the Web Lock in shared mode. This would put the granularity at a statement instead of a transaction.

Were you thinking of other downsides to only having a single DB connection?

Technically, the lack of read concurrency is a limitation of OPFS access handles - only one context at a time can have an access handle to a file. So even with multiple connections you would have to work around that, e.g. the original OPFS VFS I wrote uses the File interface for read transactions because of this. But I don't know where the breakeven point is between slower reads that allow concurrency and faster reads that don't. I suspect that more people will be happier with the latter.

There are a couple more issues (at least) that need to be considered with this approach:

• When you submit a write transaction and an exception occurs because the service migrated to another tab, you won't actually know if the transaction reached the database or not (e.g. a tab could die right before or after finishing a commit). So simply resubmitting the transaction may not produce the correct result. Care must be taken either to make each write transaction idempotent or to produce a side effect that can be checked for conditional resubmission.

• Only database state will be preserved on a migration; connection state will be lost. So something like CREATE TEMPORARY TABLE or ATTACHing an auxiliary database would need to be re-established on migration, either by the new database service provider or by the interested tabs.

In some not very in-depth testing, spinning up many instances of the WASM module in many threads and connecting to the same DB file in each, reads appeared to be doing time-slicing rather than being truly concurrent.

This will depend on the VFS you use. Are you using the stock IDBBatchAtomicVFS class? Currently all the wa-sqlite example VFS classes that use locks do only exclusive locking, i.e. no concurrency, but switching to allow shared locks is a one-line change. If you're already doing this, then it could be a bug - maybe I'm using an IndexedDB "readwrite" transaction somewhere that should be "readonly".

[tantaman]

This will depend on the VFS you use. Are you using the stock IDBBatchAtomicVFS class? Currently all the wa-sqlite example VFS classes that use locks do only exclusive locking

Ah, thanks for pointing that out. I am using the stock IDBBatchAtomicVFS.

[rhashimoto]

Care must be taken either to make each write transaction idempotent or to produce a side effect that can be checked for conditional resubmission.

If application transactions aren't idempotent and there is nothing already in the database to check whether the last transaction attempt committed, it's possible to add that with an extra table:

-- Setup table to record a transaction id for a tab
-- and a trigger that doesn't allow repeating the previous id.
CREATE TABLE IF NOT EXISTS LastTx (tabId PRIMARY KEY, txId);
CREATE TRIGGER IF NOT EXISTS RollbackDuplicateTx
  BEFORE UPDATE ON LastTx
  WHEN NEW.txId=OLD.txId BEGIN SELECT RAISE(ROLLBACK,'dup'); END;

-- Then write transactions could look like this:
BEGIN IMMEDIATE;
INSERT INTO LastTx VALUES('my-random-tab-id', 'some-transaction-id')
  ON CONFLICT(tabId) DO UPDATE SET txId='some-transaction-id';

-- Actual application SQL goes here...
COMMIT;

A verbatim retry on a migration exception should then succeed only if the prior attempt failed and vice versa. I have no experience with creating triggers so this could probably be improved, but that's the basic idea.

[gabrielgrant]

@rhashimoto am i understanding correctly that this last-transaction-id check may break if there are two (or more) transaction write requests in-flight simultaneously? (if both have been written, and then are retried in the same order, they will be applied again, right?)

[rhashimoto]

I'm not sure whether you're talking about two write transactions from one tab or two write transactions from separate tabs.

I wasn't considering an design where write transactions could be streamed so one tab could have multiple transactions in flight - I was assuming a tab would wait for each transaction to complete before submitting the next. But if you did stream transactions, you could use ordered transaction ids (e.g. a counter).

For handling write transactions from multiple tabs, note that the approach above saves a separate transaction id for each tab - one doesn't overwrite the other. So I think this works. Retries across multiple tabs might not happen in the same order as the original tries (and so could produce different database contents), but typically the original order was arbitrary anyway.

I haven't tried this at all, let alone under stress, so I could be wrong.

[rhashimoto]

One of the advantages of multiplexing requests from different contexts onto a single database connection is that you can set PRAGMA locking_mode=exclusive on that connection. This has a couple benefits for performance:

• SQLite doesn't make any per-transaction locking calls to the VFS.
• SQLite knows its read cache remains valid across transactions.

This means that sharing one connection could be faster even for a VFS that would allow multiple connections, depending on your workload. For example, I measured how fast IDBBatchAtomicVFS is on repeating a small write transaction (one update and one insert) as fast as possible on my aging Mac mini:

locking mode	transactions per second
normal	53.6
exclusive	65.8

This is a decent increase in performance. At least part of the reason is that SQLite doesn't need to make a round trip to the file system to make sure that nothing has changed in its cache.

IDBBatchAtomicVFS has a "relaxed" durability mode that lets you trade durability for performance. As a refresher, durability guarantees that once a write transaction completes it is stored on persistent media. Relaxing durability means that in some cases when service is interrupted (e.g. crash, power failure, etc.) one or more of the most recent transactions may be missing on reconnection. Durability is critically important for some applications and not very important for others. How does performance compare if we relax durability on IDBBatchAtomicVFS?

locking mode	transactions per second
normal	133.5
exclusive	1722.2

Wow! What's happening here with exclusive mode? What's happening is we're not waiting for IndexedDB at all. We don't wait for writes because we're allowing relaxed durability, and we don't wait for reads because the data is in the cache. No round trips to storage. Effectively we're streaming data to IndexedDB.

This is a pretty interesting option if your application can tolerate reduced durability. Also note that IDBBatchAtomicVFS should work in a SharedWorker so it wouldn't need to be a migrating service (but it could be in case it doesn't work in SharedWorker for some reason).

[tantaman]

Any thoughts on supporting this setup for Android where SharedWorkers are not yet an option?

One idea I had was to:

1. Use dedicated workers
2. To coordinate between workers, each dedicated worker requests a weblock where the resource name is the db name. Only the tab with the lock gets to proceed. If the tab is closed, the browser ensures another tab immediately takes over.
3. Use a signaling broadcast channel, with some static name, to indicate when a new tab has joined
4. This signaling channel is used to create names for dedicated broadcast channels between the active worker and different tabs. Other workers would record this information as well since they could be promoted in the future.
5. Something around heartbeating and reaping dead broadcast channels that were created in (4) but become irrelevant as tabs close

I worry, however, that a broadcast channel isn't as performant as a MessagePort.

[rhashimoto]

I didn't know SharedWorker doesn't work on Android. That's disappointing.

I think your idea will work. It will send all the database requests and responses to all the participating contexts, so that's potentially a lot of structured cloning. You can implement #​5 by having each tab acquire an exclusive WebLock with its name before it sends the name over the BroadcastChannel to announce itself. Then other tabs can request a shared lock on the tab name to clean up any resources associated with that name. No heartbeat necessary.

An alternative to your idea is to coordinate communication with a service worker. This is the fallback I originally had in mind, though I have not tried it. I think this will allow establishing a MessageChannel between two tabs by forwarding a MessagePort over MessageChannels between each tab and the service worker. It would be more trouble to set up than using SharedWorker but should behave equivalently once the connections are established. I could understand being reluctant to introduce a service worker, though.

[rhashimoto]

An alternative to your idea is to coordinate communication with a service worker. This is the fallback I originally had in mind, though I have not tried it.

@tantaman I tried it. It turns out to be surprisingly easy to use a service worker to establish a MessageChannel between two tabs, as long as one knows the other's clientId (which it can get via BroadcastChannel). It doesn't require adding any state to the service worker, so the ephemeral service worker lifetime isn't a factor at all.

You can play with an alternative implementation of SharedService that replaces SharedWorker with a service worker here. Both SharedService implementations have the same application interface, except for starting the service worker (which is not inside the class because the application might have its own service worker).

[tantaman]

working around the lack of OPFS in SharedWorker

Was this just a case in the past?

navigator.storage.getDirectory().then((opfsRoot) => {
  console.log(opfsRoot);
});

Returns an access handle in a shared worker context in both Firefox and Chrome. Can't seem to find the SharedWorker logs in Safari.

[rhashimoto]

You might be able to get asynchronous handles, but I'm pretty sure synchronous handles (which are the performant ones we're interested in) won't work. The reasoning is that a shared resource shouldn't block.

The spec shows that FileSystemFileHandle is defined on both Window and Worker (which I think includes SharedWorker) but its createSyncAccessHandle() method is only defined on DedicatedWorker.

[tantaman]

ah, right. Makes sense.

[AntonOfTheWoods]

I have this successfully working for the demo worker but would like to use the same system to set up a second, distinct worker ( const worker = new Worker('./second-worker.js', { type: 'module' });). My initial assumption was that if I did something like the following:

  // Connect Worker and SharedService.
  const worker = new Worker('./ahp-worker.js', { type: 'module' });
  const sharedService = new SharedService(SHARED_SERVICE_NAME, async () => {
    const providerPort = await new Promise(resolve => {
      worker.addEventListener('message', event => {
        resolve(event.ports[0]);
      }, { once: true });
      worker.postMessage(null);
    });
    return providerPort;
  });
  sharedService.activate();

  const two = new Worker('./separate-non-sqlite-worker2.js', { type: 'module' });
  const sharedService2 = new SharedService(SHARED_SERVICE_NAME + "2", async () => {
    const providerPort = await new Promise(resolve => {
      two.addEventListener('message', event => {
        resolve(event.ports[0]);
      }, { once: true });
      two.postMessage(null);
    });
    return providerPort;
  });
  sharedService2.activate();

That I would be able to use sharedService2 sharedService2.proxy.aservice("Hello there!");. However, it seems that only the first service activated can actually be used. I thought it might be the BroadcastChannel name so put the initialisation in the constructor this.#clientChannel = new BroadcastChannel('SharedService_' + this.#serviceName); (and also in the lock request) but this doesn't seem to make a difference. Whichever is declared first takes control.

Is this probably something to do with the SharedWorker/ServiceWorker clientId management? Any pointers on how to get multiple workers working together? Thanks!

[AntonOfTheWoods]

Thanks, I spent quite a while on it and there is definitely an issue regarding having the clientId in there more than once for different services. I didn't manage to find a solution though, sorry!

[rhashimoto]

The problem (or at least the first problem) is that every instance of SharedService is acquiring a Web Lock on the context clientId, so all instances after the first will block. It should just take the lock once per context no matter how many instances there are. I'll make that fix.

The lock is used for the service provider to detect the death of a service consumer so it can clean up any connection resources. It would probably be better design to use separate locks, e.g. by sending a lock name in the service request, to allow connections to different services to end independently. I'm not going to make that change though because (1) most service connections will last for the life of the context, (2) this is just a proof of concept, and (3) I'm really lazy.

[AntonOfTheWoods]

I took a few stabs in the dark by appending the service name to the client ID but it broke things so thought I'd ping back here to get an opinion. I'll confess I don't see how to make it possible to just make one service provider able to manage multiple independent services without them being able to start/end independently but either of those does everything I need so if you have time that would be awesome!

[rhashimoto]

Fix inbound landed. There is a simple two-service demo (without SQLite) here (source).

[AntonOfTheWoods]

You are a gentleman and a scholar, fine sir! Thank you! I think this code definitely deserves to migrate to a "neat little typescript lib" and if I have the time at some point will try and make that happen. This project is awesome!

[ispb88]

I've built a basic library that does this (for the purposes of using sqlite wasm or wa-sqlite), basically a backend agnostic comlink. Generifying it now and I'll probably just throw it on github open source. One layer does the processing, the other layer negotiates communication channels, so you can plug in a simple tab->shared/dedicated worker messaging channel, a dumb non-optimized broadcast channel or a more advanced channel that manages a migrating worker and creates/closes new channels as needed.

Also supports observables so you can subscribe to database updates. I'm doing this with sqlite and it works great, broadcasts out new query results when something happens to the db, and since the comparison logic is in the worker it doesn't bog down the main thread.

[AntonOfTheWoods]

That sounds awesome. I have been suggesting the electric-sql folks move on from the IDB-only setup they have now, and having a robust system with observables might just well convince them. Please let us know when you have something on gh and I will loudly ping them!

[ispb88]

Almost done. It's working great for me, but there's a lot of messy code, channels being left open and things like that which I'm cleaning up now. I can expose a class with methods returning promises and rxjs observables, and call those methods from another tab as if they were local to that thread and it's all seamless.

Here is some architecture, I'm open to feedback if anyone has any.

• At the core of it is a sender/receiver which translates command data into function calls and responses into observables and promises.
• On top of that you have a channel abstraction. A channel is just an observable and and observer for command data to go in and out of. A channel can be backed by anything, a worker onmessage/postmessage, a broadcast channel, a migrating worker, theoretically could even work fine with a local callback. The first couple of implementations end there.

So like I said, comlink, but backend agnostic. You call expose(object) on the worker and wrap(new Worker) on the frontend, literally 2 lines of code.

Migrating workers work like this. I did steal most of this from @rhashimoto (would be great to hear your thoughts by the way) but I left out the service worker part.

1. There's a named "lookup" channel - all other channel names are autogenerated. You can have multiple named lookup channels for multiple migrating services.
2. When a new worker registers (restricted to only one at a time by a web lock), it broadcasts its availability, and the ID of its dedicated registration channel.
3. When a client (ie a tab) starts, it broadcasts a request for worker availability.
4. When a worker and a client become aware of each other, they do a registration process over a the registration channel. The client creates two dedicated broadcast channels for itself, one for commands and one for responses. I did this because I figured it would keep load off of the other tabs if I kept these channels separate. The client also acquires an arbitrary web lock. The IDs of the channels and the web lock are passed to the migrating worker.
5. Once a worker and client are connected they can exchange commands issued by the client, responses to those commands, and the worker can push responses on demand (ie observables). Unsubscription logic and everything is automatic.
6. If a client dies, its web lock dies, and the worker sees that and drops all of its subscriptions.
7. If a worker dies, a new worker broadcasts its availability, all of the clients drop their old connections we start over from number 4 with all the clients. They also resubcribe to their observables on the new worker automatically but they do NOT reissue pending commands where the return types are promises.

I'm relying on the type system to govern the runtime behavior of observables vs promises, I pass a merged observable and promise back from all calls that are delegated to the worker, behavior is determined lazily by how the return object is treated (meaning do you call "then" or "subscribe" on it). It is all typed appropriately using a mapped type over the exposed type so if you follow the type system there shouldn't be any mixups.

Sqlite shared access pool plugs right into this and works fine observables and all. This doesn't solve the sqlite transaction issue automatically but that's pretty basic to do once this is in place, I wanted to leave this portion of it generic. I'll share my sqlite code too but it's really basic compared to this.

There are also a couple of things like automatic debouncing and batching of messages which has helped when I need to call a worker function a few hundred (or thousand) times in a loop, but those are optional.

I'm a little concerned about the performance of broadcast channels, but for me, it seems okay so far, I haven't benchmarked it but it doesnt feel very different than the regular worker I was using before. And this includes me sending 20mb uint8arrays back and forth pretty regularly in what feels like a few milliseconds.

In any case it is MUCH faster than using the other sqlite opfs methods. No comparison. I can open dozens of tabs, all with active observables watching the database, whereas before it would start to choke after a handful.

Oh and re performance - one thing I forgot to mention. It's very easy to automatically drop and resubcribe tabs using the browser's visibility events if you don't want to leave broadcast channels open for every tab. At some point I'll benchmark this and see if that's worth adding as a flag of some sort.

[ispb88]

I put up some code at https://github.com/sampbarrow/observable-worker

It's not at all done, but it's working pretty smoothly, ironed out some bugs today. There's some kind of issue where rxjs treats the promise+observables as promises in some cases (happens on my deployment but not my dev instance) which gives a cryptic error and crashes, so I might have to change the structure of the proxy object entirely to be safe. For modularity if I do that I'll add a dummy wrapper so you can swap in a local object if needed.

Chrome seems to be give a small grace period to let a worker finish previous requests before terminating it, so for very short queries, closing the current active worker tab even during an active query in another tab doesn't necessarily fail, which is nice.

I'd like to add the ability to pass worker related params (timeout, autoRetry) on a per-call basis so I'll be implementing that soon.

Usage is simple. For a migrating worker, in your main thread(s), do this:

const service = wrapMigrating<{YourServiceClass}>()

Then you can call service.remote.methodOfYourServiceClassThatReturnsAnObservable().subscribe(value => ...)

In the worker, do this.

exposeMigrating({ target: new YourServiceClass() })

You can pass a { context: string } to both of these calls to specifically name the lookup channel. this way you can have multiple migrating services.

[Matt-TOTW]

Hi rhashimoto, thanks for this demo, it's excellent, but I have a quick question. Could this be done with just broadcast channels? If communication between the main and worker threads was done via broadcast channels, wouldn't that be simpler than setting up the shared worker and passing message ports around? We'd use web locks to establish the Providing tab and only that one would respond to any message requests. I haven't actually implemented my suggestion - I have successfully implemented your solution with some modest modifications and it works great, thanks! 😊 - but I'm just wondering if there is a simpler way to do it. It would also be good as shared workers aren't universally supported.
Am I missing something? Perhaps there's something I haven't thought of that would mean it won't work.
Thanks for reading!

[ispb88]

FWIW @nickchomey Given recent advances I have abandoned this approach and moved to OPFSAdaptiveVFS. Slight performance hit, but was worth it to not have to maintain all of that code, also gets rid of the unsolvable problem of my "main" tab closing mid-query. Filesystem transparency is nice for future proofing as well.

Re your comments, I am heavily dependent on rxjs anyway, my sqlite worker monitors queries and streams updates back to the tab, that's why I used it. For the level of modularity and flexibility I wanted it was less complicated than rolling my own. I do not remember why I did two channels, I don't think it was strictly necessary. All of the fancy stuff is in deprecated.ts in my repo if you want to look but it's buggy and I'm not planning to fix it since I'm no longer using this approach.

[nickchomey]

Thanks for the update! That makes lots of sense.

As it turns out, my interest in this is primarily for something other than sqlite, so all the more reason for me to focus on the other, simpler, mechanism. I'll still use yours for inspiration though.

[gabrielgrant]

@ipb26 Can you explain more/share a link to what you mean by OPFSAdaptiveVFS/how it works? A quick search doesn't seem to be turning it up...

[ispb88]

https://github.com/rhashimoto/wa-sqlite/tree/master/src/examples#vfs-comparison

[Matt-TOTW]

Hi @nickchomey, you're very welcome to add comments/questions etc. My repo is just an example so I can't promise to dig deep into issues, but I can say that I am using a variant of it in the wild.

[nickchomey]

FYI, android chrome is finally planning to add support for sharedworkers. https://chromestatus.com/feature/6265472244514816

Hopefully it'll be fully available in something like 8 months!

I assume that the OPFS FileSystemSyncAccessHandle stuff would still be an issue here, but it's a massive step in the right direction

[AlphabetsAlphabets]

Is there an update on this? Is this feature available already? If so, is there a guide or a starting point of how I can use it?

[abhay-agarwal]

While this repo may not be the right / best place to implement this strategy, I would like to note that Notion (notion.so) has publicly announced that they actually use this strategy to enable faster SQL operations while allowing for a single "shared" worker. Presumably, this single worker operates the connection to the server and things like push notifications, etc., so having multiple independent workers that connect to the same local db isn't viable.

Wherever it ends up being, therefore, I think an open implementation of this system that gets battle tested and has a public issue list would go a long way to creating a robust synchronous "shared" worker.

Either that or we wait 5-inf. years for all the browsers to implement sync access handles on shared workers.

[rhashimoto]

Note that at long last Chromium is finally adding SharedWorker support on Android, so in a few months SharedWorker may be available on all major browsers on all platforms.

The standard says that a SharedWorker context can construct a (dedicated) Worker. I can't find a caniuse or similar page that says whether this works on all browsers, but if it does then a SharedWorker can manage lifetime and access to a Worker that makes synchronous OPFS calls. You can bypass all the leader election stuff, and just ask the SharedWorker for a MessagePort to the Worker.

[abhay-agarwal]

Whatever the spec says, this is not how SharedWorkers are allowed to work in practice. All Dedicated Workers must be parented to a tab, and their lifecycle matches that tab. It's possible that there's a hidden "LRU" inside each shared worker to know what it's current default parent is, but I haven't encountered that anywhere. Instead, on all browsers I'm aware of, Shared Workers are not allowed to spawn dedicated workers. On Chrome, I believe shared workers are allowed to spawn other shared workers, though, oddly enough. That is not something that exists in conservative browsers like Safari though (which incidentally doesn't even have a way of retrieving logs on shared workers unless you postMessage them over somewhere).

[rhashimoto]

Whatever the spec says, this is not how SharedWorkers are allowed to work in practice.

Well, that's disappointing. Looks like a SharedWorker can start a Worker in Firefox, but not in Chrome or Safari. I guess we know who put it in the standard.

[simolus3]

There's an open Chromium issue for enabling dedicated workers in shared workers, it might be worth adding a +1 to that.

[didimitrie]

Thank you for this writeup and generous sharing, this approach helped us resolve a similar issue (multi-tab access, opfs contention) with duckdb-wasm. Were it not for your share, we'd have been stuck cursing at browser apis.

🙇

[arnold-graf]

Hi all!

First off, thanks — this discussion has been super helpful for my own projects that use OPFS.

I just wanted to also let you know that I took the content of this thread as the starting point for a library that tries to solve the general case. I’m personally using it with wa-sqlite, my own WebGPU/OPFS vector DB and some other projects.

Broadly speaking, it implements the initial idea described here, but maintains the standard WebWorker API as its narrow target. I’ve tried to make it a little more robust — e.g. by buffering outbound messages in case a leader isn’t ready, among some other small improvements.

Hopefully somebody finds this useful. I’d love to get some feedback on it!
https://github.com/arnold-graf/cross-tab-worker/

[paralin]

I have also implemented a cross-tab synchronization and OPFS paging system that may be useful to others - check it out!

It's highly optimized for efficient cross-tab storage of any data structure.

https://github.com/s4wave/spacewave/tree/master/db/volume/js/opfs

[Waldenesque]

FYI, looks like SharedWorker on Android has finally shipped in Chrome 148.

[nickchomey]

Chrome also shipped extended lifetime shared workers, which persist for a period even after the final tab has closed so that any background actions can complete.

More importantly, as far as I'm concerned, when you only have one tab open and do a full page nav (click link), it won't reload the sharedworker in the brief moment in which the page doesn't exist. This allows heavy things like wasm to remain instantiated.

https://developer.chrome.com/blog/extended-lifetime-shared-workers-origin-trial

Firefox intends to implement and Safari is being safari.

[csquillen]

Saw this in WebKit's WWDC26 post and figured it was worth flagging here — Safari 27 beta adds support for creating dedicated workers inside shared workers, "per the HTML Standard."

I tried SharedWorker → dedicated worker → createSyncAccessHandle again today — works in Firefox, but Chrome throws ReferenceError: Worker is not defined in SharedWorkerGlobalScope. Have not yet tested Safari 27 because it requires macOS 27.

Has anyone confirmed SAH works in a nested worker there?

[AntonOfTheWoods]

Saw this in WebKit's WWDC26 post and figured it was worth flagging here — Safari 27 beta adds support for creating dedicated workers inside shared workers, "per the HTML Standard."

I tried SharedWorker → dedicated worker → createSyncAccessHandle again today — works in Firefox, but Chrome throws ReferenceError: Worker is not defined in SharedWorkerGlobalScope. Have not yet tested Safari 27 because it requires macOS 27.

Has anyone confirmed SAH works in a nested worker there?

You have opfs directly in sharedworker on recent mac safari, and at least since the 18.4 on iOS. Chrome doesn't seem to want nested workers or direct access though

[csquillen]

This is an old discussion — July 18, 2018 — but Chrome did intend to support a dedicated worker inside a shared worker. In the Intent to Implement and Ship:

allowing the creation of dedicated workers from a shared worker would lead to dependencies on deprecated code that would be difficult to untangle, so for the initial commit we are only going to support nesting dedicated->dedicated. I hope to add shared->dedicated nesting soon.

https://groups.google.com/a/chromium.org/g/blink-dev/c/KZx0i3e5nZM

[nickchomey]

here's the relevant chromium issue if you want to voice support

https://issues.chromium.org/issues/40695450

[simolus3]

Has anyone confirmed SAH works in a nested worker there?

I have just tested that in Safari Technology Preview Release 249 on macOS 26, it works! 🎉

[rhashimoto]

Next year's Interop proposal window is coming up soon (last year was September 4-24):

The goal of the Interop Project is to improve the web by making it easier to make websites and web apps that work in every browser or browser engine at the same time.

The relevant WPT test is here.

[nickchomey]

Somewhat related to this topic, I created a proposal for Interop 2027 for universal support for extendedlifetime SharedWorkers. Perhaps folks from here would like to lend their support to it? It would make sharedworkers (which are now universally supported, after chromium 148 added support to android) that much more powerful as they'd be truly long-lived (rather than unloading when you navigate with 1 tab open).

web-platform-tests/interop#1382

[Waldenesque]

@sgbeal

A potential for abuse already exists today using service workers (among other possibilities).

How specifically would extended lifetime shared workers increase, expand or enhance such potential for abuse?

[sgbeal]

How specifically would extended lifetime shared workers increase, expand or enhance such potential for abuse?

Chrome, for example, sometimes takes ages to actually let go of OPFS handles when a tab is closed, frequently causing locking errors for me when re-opening a tab after, say, 2-5 seconds. When a worker is held open arbitrarily long after closing its tab, that problem will be exacerbated. "Shut down the browser, wait 30 seconds, and try again" is the ugliest possible workaround, but that's what we're going to be told to do, and it's going to be a huge pain in the buttox for web developers who have to reload the app 130 times a day during development.

When an app does not close when told to by the user, then the app has Gone Rogue, period, and the people who told it to do that are acting maliciously, period. They do not own your hardware. They have no say-so in how you utilize it. This gives them some leeway they should never have been offered. Anything the app does after ctrl-w amounts to a violation of trust and theft of system resources. Again: "anyone who says otherwise is selling something."

There's a reason processes are not allowed to ignore SIGKILL: an explicit kill/close request is inviolate, an expression of the user's Ultimate Right to decide what is and is not running on their system. This proposal throws out 50 years of well-established, well-justified dogma to hand the keys over to some nameless marketing person halfway across the world.

Per Nick, WHATWG has already green-lit this, so it's a done deal and my objections are moot.

I was just sharing it here given that people are evidently interested in robust rogue sharedworker capabilities

Fixed that for you.

[nickchomey]

Its hard, at this point, to consider you to be communicating in good faith. This proposal simply asks to universally implement functionality that is already defined in the web Spec, which doesn't really add any further risk/burden beyond what is already possible via things like Service Workers. I shared it here in case others are interested in it - I did not come to be maligned, harangued and gaslit.

[sgbeal]

I did not come to be maligned and gaslit.

And it's not my intent to malign or gaslight anyone, but i will unrelentingly malign any group which standardizes any browser behavior which takes control away from the user (e.g. "the user closed it but it refuses to die because because someone else decided that the user doesn't really know what they want and will be back soon"). It is certainly possible that i've entirely misunderstood, in which case you have my profuse apologies and the satisfaction of knowing that i've made a complete jackass out of myself in an undying public record, but i'm not seeing such a misunderstanding in a re-read.

In any case, i'm dropping out and apologize for any disruption i've caused.

[nickchomey]

Given that you care so much about these things, perhaps it would be worth keeping a closer eye on standards committees etc, to share your viewpoints where they can have more impact than places like this. I'd very much welcome your user-centered intolerance of nonsense when, for example, Apple is trying to hold back the web in favour of its App store extortion racket. Recent (and still-active) examples:

• Question: Review manifest-first Web Install API w3ctag/design-reviews#1245
• BeforeInstallPromptEvent WebKit/standards-positions#619