Gossipsub rejects go-ssv IHAVE above 16 KiB, cutting SSV operators off from Anchor operators in a mixed committee #1335

Summary

Anchor keeps rust-libp2p gossipsub's default max_control_message_size of 16 KiB. SSV nodes (go-libp2p-pubsub) gossip IHAVEs of up to 1,500 message IDs per topic, and SSV message IDs are 12 bytes, so a full IHAVE is about 21 KB. Anchor rejects the whole RPC (rpc control size exceeds max control message size) and closes that peer's inbound substream.

The SSV side then treats Anchor as dead but still connected: go-libp2p-pubsub drops Anchor from every topic and from its mesh, and reopens its own stream. It never re-learns Anchor's subscriptions, because Anchor doesn't re-announce them on an existing connection. From then on, that SSV node can't graft, gossip to, or flood-publish to the Anchor node. Its messages reach that Anchor node only while Anchor keeps it in Anchor's own mesh.

Lighthouse sets .max_control_message_size(128 << 10) on the same gossipsub fork.

Impact

In a mixed committee, a 3-of-4 quorum needs both clients. On a local 2 SSV + 2 Anchor devnet (ssv-mini local_testnet_gloas, 3,000 validators on the one cluster), the committee lost most of its duties in recurring episodes of 2–3 epochs, starting about 1.6 hours after the Gloas fork. In one epoch none of its 2,991 attestations landed; in another, none of its 11 proposals did. It ran healthy for 28 epochs between two episodes.

Each episode starts when Anchor's slow-peer scoring prunes the SSV peers from the committee topic's mesh, and keeps them out of it for up to about 40 minutes (#1337). With Anchor's subscriptions already forgotten, the SSV nodes have no way back until Anchor re-grafts them.

In that devnet the SSV nodes peered only with the two Anchor nodes, which made the cut total. That came from ssv-mini, not from where discovery bootstraps: the SSV nodes advertised an address nothing in the enclave could dial (ssvlabs/ssv-mini#63 fixes it). With that fix the SSV nodes peer with each other, and a rerun still lost its committee's duties right after the Gloas fork, through this issue together with #1336 and #1337. With more peers an SSV node has other paths, but the link to the Anchor operator is still lost.

Evidence

From the Anchor node's libp2p.log (libp2p_gossipsub at DEBUG):

1. Slow-peer penalties at one slot boundary: [Penalty] Expired message penalty 105 times for one SSV node, 78 for the other and 125 for the exporter (all go peers; none for the other Anchor node).
2. One heartbeat later: HEARTBEAT: Prune peer with negative score for both SSV nodes on the committee topic (−18.5 and −13.1).
3. Twelve seconds later: Failed to read from inbound stream: Failed to encode/decode message: rpc control size exceeds max control message size.

At the next per-minute sample, that SSV node's count of peers subscribed to the committee topic (go-libp2p-pubsub ListPeers) went from 1 to 0 and stayed there, while Anchor still counted the SSV node as subscribed (gossipsub_topic_peers_counts 4).

Restarting the Anchor node gives the SSV nodes a fresh connection, and they list it as subscribed again within a minute.

Suggested fix

Raise max_control_message_size well above what go-ssv sends, e.g. 128 KiB as Lighthouse does (go-libp2p-pubsub's own default is 512 KiB).

Related

• gossipsub: default max_control_message_size (16 KiB) rejects the IHAVE/IWANT the sender builds with default max_control_messages_sent (5000) libp2p/rust-libp2p#6667: the 16 KiB default rejects the IHAVE/IWANT a default-configured sender builds.
• gossipsub: after the receiver rejects one RPC, the sender can't reach that peer again until reconnect libp2p/rust-libp2p#6668: after one rejected RPC, a rust sender can't reach that peer again until reconnect.
• A still-connected peer declared dead loses its topic subscriptions until reconnect libp2p/go-libp2p-pubsub#734: on the go side, a still-connected peer declared dead loses its subscriptions until reconnect (still present in v0.18.0).

Environment

Anchor epbs as of 2026-10-01 (sigp/rust-libp2p fork as pinned there); unstable keeps the same default. SSV stage (go-libp2p-pubsub v0.15.0). ssv-mini local_testnet_gloas with Lighthouse v8.3.0-rc.0 and Geth v1.17.7.