fix(live-voice): let Realtime use a plain-HTTP user endpoint - #1422
Merged
Merged
Conversation
The OpenAI Realtime adapter required an HTTPS base URL and the Live WebSocket transport accepted only wss, so a Realtime-compatible server on the user's own machine (http://127.0.0.1:8010/v1) could not be used. ADR 0304 already lets endpoints the user typed reach loopback and LAN over plain http under the default relaxed network mode; the Realtime path was the one user-supplied endpoint still hard-coded to TLS. Map an http base URL to ws and let the existing public-network guard decide, from networkPolicy.mode, whether that plaintext hop is allowed. Third-party Live endpoints (Gemini) keep wss only, base URLs with credentials, a query or a fragment stay refused, and a plain ws endpoint on a proxied route fails closed rather than being tunneled in the clear. The scheme rules move to a small Electron-free module so they can be tested directly. Refs vastsa#1318
Exercise the production transport against a loopback WebSocket and keep the proxied plaintext refusal covered. Add a plain-HTTP fixture mode so the full isolated call journey can validate the user-endpoint policy.
Owner
|
Thanks for the focused fix. I added coverage for the production loopback WebSocket transport, the proxied-plaintext refusal, and a plain-HTTP mode for the isolated Live Voice fixture. The PR is merged. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1318.
Problem
Live Voice's OpenAI Realtime adapter only accepts an
httpsProvider base URL, and the Live WebSocket transport only dialswss. A Realtime-compatible server on the user's own machine (http://127.0.0.1:8010/v1) is therefore refused before any connection is attempted.ADR 0304 already allows this for every other endpoint the user types in: under the default
relaxednetwork mode, a user-supplied endpoint may reach loopback/LAN over plainhttp, andstrictkeeps the public-HTTPS-only rule. The Realtime base URL is user-supplied (openLiveWebSocket(..., endpointOrigin: "user")), but it was the one path still hard-coded to TLS.Change
live-voice/websocket-endpoint.ts:realtimeSocketUrl: moved out of the adapter.https→wssas before, andhttp→ws. Credentials, query and fragment are still refused, and so are other schemes.liveSocketGuardUrl:wssis accepted for any origin.wsis accepted only forendpointOrigin: "user"; a third-partywsURL (Gemini) still fails withLIVE_NETWORK_POLICY_UNSUPPORTED.openLiveWebSockethands the mapped URL to the existingendpointGuard.assertPublicUrl, so whether plainwsis allowed is decided bynetworkPolicy.modein one place:relaxedallows it and raises the one-time plaintext notice,strictrefuses it.wsendpoint on a proxied route fails closed (LIVE_NETWORK_POLICY_UNSUPPORTED).ProxyTunnelAgentspeaks TLS to the destination, and a cleartext call should not be tunneled through a proxy; loopback/LAN are already in the default bypass list (ADR 0304 item 6).Not in scope
A plain transcription server (
POST /v1/audio/transcriptions→{"text": ...}, e.g. stock faster-whisper) still cannot drive Live Voice: Live Voice is a full-duplex Realtime session (/realtime,session.update, audio output, function calls). After this change, a Realtime-compatible local server can be used. Plain HTTP STT already exists as the host speech capability (openai_audio, spec20-speech.md), but ADR 0291 deliberately gives it no app surface, so wiring it into Dictation is a separate product decision.Docs
docs/spec/03-runtime/live-voice.md: scheme rules per adapter and the proxy behavior.docs/spec/06-delivery/04-e2e-test-plan.md: new scenarioE2E-LIVE-VOICE-realtime-plaintext-user-endpoint.Validation (macOS arm64, Node 23.11, pnpm 12.8.1)
apps/desktop/test/live-voice-websocket-endpoint.test.mjs: 3 new tests covering the scheme mapping, the user/third-party split and the refused base URL shapes.node --test test/live-voice-*.test.mjsplus the endpoint/public-network suites: 100/100 pass.pnpm --filter @pi-desktop/desktop typecheck,pnpm lint,pnpm build:js,cargo build -p host-core --locked: pass.apps/desktopsuite: 3541/3551 pass. 3 fail and 7 are cancelled, all inplugin-websocket,chat-error-message,imported-package-skills-runtimeandsettings-inline-import-user-path. None of those files load the changed modules. The failures look environmental (Node 23 error wording; loopback socket tests hang in my sandbox), but I have not re-run them on unmodifiedmain.Even when it runs, the existing Live Voice E2E uses a local WSS fixture, so it would only show that the TLS path did not regress. The new
wspath is covered by the unit tests above, not by an end-to-end call.