Skip to content

feat!: fast join through fast_join and FastJoin (3RTT T23) - #11

Merged
tbarbugli merged 6 commits into
factory/3rttfrom
3rtt/t23-fast-join
Oct 1, 2026
Merged

tbarbugli merged 6 commits into
factory/3rttfrom
3rtt/t23-fast-join

Conversation

@tbarbugli

Copy link
Copy Markdown
Member

3RTT task T23, SDK side. Call.Join now joins through the coordinator's fast_join (chat #17828, T21) and the SFU's FastJoin (video-sfu #1627, T22). Both are open and not merged; the local stack runs them. On the local stack with 100 ms injected, warm time to media drops from 12.3–12.7 RTT to 9.7–11.1 RTT, and cold from 13.8–14.7 to 10.8–11.2.

API

  • JoinFlowFast is the default. WithJoinFlow(JoinFlowLegacy) asks for the legacy flow, which stays for benchmarks until T50. Call.JoinFlow() reports the flow the first join took, so a bench or a test can tell a fallback from a fast join.
  • WithTrack(info, track) publishes from the start. Its offer is built while fast_join is in flight and is answered by the FastJoin itself. AddTrack after Join still works, but it costs a renegotiation.
  • The fast flow is a default, not a separate method, because a deployment without fast join falls back by itself (see below). Callers need no change, and rule 13 allows the break.

The flow

  1. fast_join (location auto, no connection_id) runs in parallel with pcs.create, which builds both PCs, adds the join's tracks, creates the publisher offer, and builds the recvonly subscriber_sdp.
  2. FastJoin goes to candidate 1 with that candidate's token (the Authorization: Bearer header), grant and ICE servers. The SFU websocket is dialled at the same time.
  3. On success the SDK applies the publisher answer and the subscriber offer, and answers the offer. SendAnswer (with subscriber_negotiation_id) is not awaited.
  4. The attach JoinRequest (attach_fast_join) goes out once the FastJoin has succeeded. The dial takes as long as the FastJoin or longer, so this costs no round trip, and a candidate that refused gets no attach.
  5. There is no local trickle and no debounce. The SFU is ICE-lite, and the local stack confirms it connects without client candidates, which T22 had left open.

Errors and fallback

SFU or coordinator answer What the SDK does
SFU_FULL, SFU_SHUTTING_DOWN, UNAUTHENTICATED, transport error tries the next candidate; if none takes the client, calls fast_join once more
CALL_PARTICIPANT_LIMIT_REACHED the join fails
fast_join 404 that is not an unknown user; every SFU without FastJoinServer (Twirp bad_route); "join through the coordinator" legacy join, starting from a call that was never joined

Reconnects and migrations keep the legacy path. Pre-set credentials (GetCred, and UseSFU from #4 once it merges) also go the legacy way, because fast_join returns candidates, not credentials.

Trace (T01)

New spans:

  • coord.fastjoin, with Server-Timing;
  • sfu.fastjoin, with the SFU's server_timings as .server;
  • sfu.ws.dial and sfu.ws (the attach);
  • pub.sfu.candidates and sub.sfu.candidates;
  • sub.answer, and sub.sendanswer (not awaited).

Each candidate records into a scratch recorder, which is merged only for the candidate that took the client.

Measured (joinbench, local stack, 100 ms injected, 10 runs; medians)

legacy fast
cold pubsub, publish / subscribe 14.0 / 14.3 RTT 10.9 / 11.2 RTT
warm pubsub, publish / subscribe 12.4 / 12.7 RTT 9.7 / 11.1 RTT
warm one-to-one, publish / subscribe 12.3 / 12.7 RTT 9.8 / 10.4 RTT

Warm fast critical path: coord.fastjoin (1) > sfu.ws.dial (2) > sfu.ws (1) > *.sfu.candidates > ICE (4–6) > DTLS (2) > RTP. The websocket is still on the media path because the SFU trickles its candidates on it; T31 removes that. Staging is not measured: the shared edge still answers 404 on fast_join.

Tests

  • fastjoin_test.go, against FakeSFU:
    • the whole sequence with media both ways: answer applied, subscriber offer answered, SendAnswer held by the SFU while Join has already returned, no IceTrickle or SetPublisher, attach after FastJoin;
    • falling back to candidate 2 on 600, 700, UNAUTHENTICATED and an unreachable SFU;
    • a second fast_join round, then giving up;
    • a full call;
    • the legacy fallback in its three cases (no fast_join, no FastJoin, no bus credential);
    • a late attach;
    • an unknown user;
    • a run under WithNetworkDelay that pins every signalling step's RTTs.
  • FakeSFU serves FastJoinServer and answers an attach like the SFU does. The fake coordinator serves fast_join.
  • TestFastJoinWithinTheInterimBudget (JOINBENCH_LIVE=local, skipped otherwise) runs warm fast joins against the local stack and asserts the median time to media ≤ 11.5 RTT + 30 ms. It passed at 988 / 1038 ms.

Notes

  • go generate ./coordinator/... fails under Go 1.27 (moq). The mock is extended by hand in moq's format.
  • The PCs are built before the SFU is known, so its TURN servers are set afterwards with SetConfiguration, and gathering only uses them after an ICE restart. audio_receive_slots is not sent (T33).

make ci passes locally. GitHub Actions can't run on this repo right now because of org billing.

FastJoinCall POSTs to /api/v2/video/call/{type}/{id}/fast_join with join's
body and query. The response has the call state and up to five candidate
SFUs, each with its own token, ICE servers and setup grant. Error keeps the
HTTP status, and IsNotFound tells a coordinator (or edge) without the route.

The moq mock is extended by hand in moq's format: go generate fails under
Go 1.27 (moq v0.5.3 and v0.7.1: "package bytes without types").
…nRequest

Bumps github.com/GetStream/protocol to v1.50.0-3rtt.1 for the FastJoinServer
service and JoinRequest.attach_fast_join. FastJoin shares the SignalServer
client's base URL, HTTP client and auth interceptor. Dial and Join split
Connect, so a fast join can dial the websocket while its FastJoin is in
flight and send the attach once the FastJoin has created the participant.
Scratch and Merge let a fast join record each candidate SFU apart and keep
only the spans of the one that took the client. Remove drops a step that is
redone. ServerDuration puts a server's own timing (FastJoin server_timings)
on a step as ServerTiming does for the Server-Timing header.
FakeSFU serves the FastJoinServer next to the SignalServer (WithoutFastJoin
for an SFU from before it). Like the SFU, it answers an attach with no
FastJoin before it with participant-not-found, and sends a fast-joined
client nothing before its websocket has attached.
Join now takes JoinFlowFast: fast_join returns candidate SFUs while the peer
connections and the publisher offer are built (WithTrack passes the tracks at
join). One FastJoin to the first candidate that takes the client answers the
publisher offer and returns the subscriber offer; the answer to it goes out
without waiting. The SFU websocket is dialled alongside and attaches
(attach_fast_join) once the FastJoin has succeeded. No local trickle and no
negotiation debounce on the fast path: the SFU is ICE-lite.

Candidates are tried in order: SFU_FULL, SFU_SHUTTING_DOWN, UNAUTHENTICATED
and transport errors move to the next; if none takes the client, fast_join is
asked once more. CALL_PARTICIPANT_LIMIT_REACHED fails the join. A coordinator
without fast_join (404 that is not an unknown user), SFUs without FastJoin, or
an SFU that cannot create the call ("join through the coordinator") fall back
to the legacy join from a call that was never joined. Call.JoinFlow says which
flow ran; WithJoinFlow(JoinFlowLegacy) asks for the legacy one. Reconnects and
migrations keep the legacy path.

The join trace gets coord.fastjoin, sfu.fastjoin (with the SFU's
server_timings), sfu.ws.dial, sfu.ws, pub/sub.sfu.candidates, sub.answer and
sub.sendanswer (not awaited). Each candidate records into its own scratch
recorder, merged only for the one that took the client.

BREAKING CHANGE: Join calls fast_join, and needs a coordinator and SFUs with
fast join to take the fast flow; elsewhere it falls back to the legacy flow.
The fast flow joins with WithJoinFlow(JoinFlowFast) and passes the publisher's
audio with WithTrack; the legacy flow publishes with AddTrack after Join as
before. A run whose join took another flow than asked fails.

TestFastJoinWithinTheInterimBudget (JOINBENCH_LIVE=local) runs warm fast joins
against the local stack and checks the median time to media both ways within
11.5 RTT + 30 ms, today's interim bound.
@tbarbugli
tbarbugli merged commit 9753e5f into factory/3rtt Oct 1, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant