bitswap: runOnLimitedConnections has no effect — two call sites drop the flag

[NiKrause]

Summary

@helia/bitswap documents runOnLimitedConnections, but setting it to true does not make bitswap work over a limited (circuit-relay) connection. Two independent places drop the flag, and each one alone is enough to break it — patching either has no effect, patching both makes it work.

Versions: @helia/bitswap 4.0.11, helia 7.1.7, libp2p 3.3.8, @libp2p/circuit-relay-v2 4.2.11. Two browsers (Chromium), meeting through a circuit relay.

Where the flag is lost

1. The registrar topology never notifies about limited connections.

src/network.js, start() registers a topology that does not set notifyOnLimitedConnection:

const topology = {
  onConnect: (peerId) => { this.safeDispatchEvent('peer:connected', { detail: peerId }) },
  ...
}

libp2p/src/registrar.js skips exactly that case:

if (connection.limits != null && topology.notifyOnLimitedConnection !== true) { /* skipped */ }

So bitswap never learns the peer exists and never asks it for anything.

2. sendMessage() dials without the flag.

src/network.js:

const stream = await this.libp2p.dialProtocol(peerId, BITSWAP_120, options)

options here is the queue job's options; this.runOnLimitedConnections is not merged in. handle() and findProviders()'s isDialable() both honour it — the call that actually asks a peer for a block is the one that does not.

Measured

Two browser nodes, no direct path possible (no DCUtR service and no /webrtc listen address), so the circuit is the only path they can ever have. The connection reports limits != null. One node adds a block; the other reads it by CID.

patch applied	runOnLimitedConnections	result
none	true	times out
topology only	true	times out
sendMessage only	true	times out
both	true	block arrives
both	false (default)	times out

The last row matters: with both fixed, the default behaviour is unchanged. Nothing starts crossing circuits unless the option asks for it.

Worth noting separately — the bitswap protocol stream itself opens fine over the circuit. Dialling /ipfs/bitswap/1.2.0 by hand with runOnLimitedConnection: true succeeds while a read is still timing out, which is what pointed at these two sites rather than at libp2p refusing the protocol.

Suggested patch

         const topology = {
+            notifyOnLimitedConnection: this.runOnLimitedConnections,
             onConnect: (peerId) => {

-            const stream = await this.libp2p.dialProtocol(peerId, BITSWAP_120, options);
+            const stream = await this.libp2p.dialProtocol(peerId, BITSWAP_120, {
+                ...options,
+                runOnLimitedConnection: this.runOnLimitedConnections
+            });

Why this matters for browsers

Two browsers behind carrier NAT cannot hole-punch reliably, and the circuit is often the only path they have. Today that means content cannot move between them at all through Helia, while everything else on the same connection — identify, ping, gossipsub, an application protocol declaring runOnLimitedConnection: true — works. The option exists to allow that trade; it just does not reach the code that would honour it.

Happy to open a PR with the two-line change and a test if that is useful.

Possibly related, though not the same thing: #253.

[NiKrause]

An update on urgency, since it would be unfair to leave this looking like it blocks somebody.

It no longer blocks us. We routed around it: the application's own protocol declares runOnLimitedConnection on both the handler and the dial, and carries the bytes itself. Measured on one connection at one moment — the app protocol carried 256 KiB over a circuit while bitswap carried nothing, and with framing in place the same circuit carried 16 MiB in 585 ms. So the transport was never the constraint, and we did not need this fix to move data over a relay.

The bug is still a bug, though. runOnLimitedConnections is documented and does nothing, and the failure is silent — a caller who sets it gets no error and no blocks. That is worth fixing regardless of whether anyone is currently blocked, and the two-line patch and the measurement matrix in the original report still stand.

One thing worth adding for anyone reading this later: enabling the option is node-wide, so it would let any peer reachable over a relay fetch any block by CID. For us that trade was not obviously good — it would have meant encrypting the blockstore before turning it on. Not an argument against fixing it, but perhaps against reaching for it as a default.

[NiKrause]

One more measurement, because it changes how much the fix in this issue would actually buy — and it is worth knowing before anyone implements it.

A relay's default budget is 128 KiB and two minutes. @libp2p/circuit-relay-v2 applies DEFAULT_DATA_LIMIT = 1 << 17 and DEFAULT_DURATION_LIMIT = 2 minutes to every reservation unless the relay is configured otherwise, and that budget covers everything on the circuit.

So with the two-line fix applied and runOnLimitedConnections: true, bitswap over a relay would still stop at 128 KiB against a stock relay. Measured on our side with our own protocol, which does set the flag: 256 KiB crosses a circuit whose relay grants 1 GiB, and does not cross the same circuit when the relay is left at library defaults.

That is not an argument against fixing this — the option is documented and silently does nothing, and that remains true. It is an argument about expectations: the flag decides whether a protocol may use a circuit; the relay's reservation limits decide how much it can move. Both have to be right, and only one of them is in this repository.

Two things follow that might be worth doing here regardless of the patch:

• the option's documentation could say that a relay's limits still apply, since "enable this and bitswap works over relays" is the natural reading and is not true against a default relay
• applyDefaultLimit: false on the relay side produces an unlimited circuit, and then bitswap works over it today with no changes at all. We measured a file crossing that way. Whether that is a configuration anyone should be encouraged toward is a separate question — an unlimited circuit means the relay carries unbounded traffic for whoever reserves — but it does mean the current behaviour is reachable without this fix, for anyone who controls their own relay.

[NiKrause]

Still present in 4.0.14, and never absent in any release.

Two additions to the report above, measured today.

1. The newest published version has the same two sites unchanged. @helia/bitswap 4.0.14, network.js:246 still reads dialProtocol(peerId, BITSWAP_120, options) with runOnLimitedConnections not merged in, and the topology registered in start() still lacks notifyOnLimitedConnection.

2. The option has never worked in any published version. Checked the first and last release of every major — 0.0.0, 1.0.0 / 1.1.4, 2.0.0 / 2.2.0, 3.0.0 / 3.2.3, 4.0.0 / 4.0.14: the option exists in all of them and is wired to neither site in any. So this is not a regression to bisect; the option was published without ever being connected to the two calls it governs.

Re-confirmed with the two-line patch applied, same two-browser circuit-only setup as above: with the option on, a block crosses in eleven seconds; with the option off, nothing crosses. The default is unchanged by the fix. We now carry that patch via patch-package and would be glad to see it land here so we can drop it.