Skip to content

fix(live-voice): let Realtime use a plain-HTTP user endpoint - #1422

Merged
vastsa merged 4 commits into
vastsa:mainfrom
hxj04121-lab:fix/realtime-loopback-http
Oct 5, 2026
Merged

vastsa merged 4 commits into
vastsa:mainfrom
hxj04121-lab:fix/realtime-loopback-http

Conversation

@hxj04121-lab

Copy link
Copy Markdown
Contributor

Refs #1318.

Problem

Live Voice's OpenAI Realtime adapter only accepts an https Provider base URL, and the Live WebSocket transport only dials wss. A Realtime-compatible server on the user's own machine (http://127.0.0.1:8010/v1) is therefore refused before any connection is attempted.

ADR 0304 already allows this for every other endpoint the user types in: under the default relaxed network mode, a user-supplied endpoint may reach loopback/LAN over plain http, and strict keeps the public-HTTPS-only rule. The Realtime base URL is user-supplied (openLiveWebSocket(..., endpointOrigin: "user")), but it was the one path still hard-coded to TLS.

Change

  • New Electron-free module live-voice/websocket-endpoint.ts:
    • realtimeSocketUrl: moved out of the adapter. https → wss as before, and http → ws. Credentials, query and fragment are still refused, and so are other schemes.
    • liveSocketGuardUrl: wss is accepted for any origin. ws is accepted only for endpointOrigin: "user"; a third-party ws URL (Gemini) still fails with LIVE_NETWORK_POLICY_UNSUPPORTED.
  • openLiveWebSocket hands the mapped URL to the existing endpointGuard.assertPublicUrl, so whether plain ws is allowed is decided by networkPolicy.mode in one place: relaxed allows it and raises the one-time plaintext notice, strict refuses it.
  • A plain ws endpoint on a proxied route fails closed (LIVE_NETWORK_POLICY_UNSUPPORTED). ProxyTunnelAgent speaks TLS to the destination, and a cleartext call should not be tunneled through a proxy; loopback/LAN are already in the default bypass list (ADR 0304 item 6).

Not in scope

A plain transcription server (POST /v1/audio/transcriptions → {"text": ...}, e.g. stock faster-whisper) still cannot drive Live Voice: Live Voice is a full-duplex Realtime session (/realtime, session.update, audio output, function calls). After this change, a Realtime-compatible local server can be used. Plain HTTP STT already exists as the host speech capability (openai_audio, spec 20-speech.md), but ADR 0291 deliberately gives it no app surface, so wiring it into Dictation is a separate product decision.

Docs

  • docs/spec/03-runtime/live-voice.md: scheme rules per adapter and the proxy behavior.
  • docs/spec/06-delivery/04-e2e-test-plan.md: new scenario E2E-LIVE-VOICE-realtime-plaintext-user-endpoint.

Validation (macOS arm64, Node 23.11, pnpm 12.8.1)

  • apps/desktop/test/live-voice-websocket-endpoint.test.mjs: 3 new tests covering the scheme mapping, the user/third-party split and the refused base URL shapes.
  • node --test test/live-voice-*.test.mjs plus the endpoint/public-network suites: 100/100 pass.
  • pnpm --filter @pi-desktop/desktop typecheck, pnpm lint, pnpm build:js, cargo build -p host-core --locked: pass.
  • Full apps/desktop suite: 3541/3551 pass. 3 fail and 7 are cancelled, all in plugin-websocket, chat-error-message, imported-package-skills-runtime and settings-inline-import-user-path. None of those files load the changed modules. The failures look environmental (Node 23 error wording; loopback socket tests hang in my sandbox), but I have not re-run them on unmodified main.
Task candidate: fix/realtime-loopback-http @ dabc1100c
Base main:      upstream/main @ 1f5302335
E2E suites:     pnpm test:e2e:live-voice
Result:         NOT RUN — no Electron 43.6.0 binary on this host
Environment:    macOS arm64, Node 23.11.0

Even when it runs, the existing Live Voice E2E uses a local WSS fixture, so it would only show that the TLS path did not regress. The new ws path is covered by the unit tests above, not by an end-to-end call.

hxj04121-lab and others added 4 commits October 5, 2026 14:29
The OpenAI Realtime adapter required an HTTPS base URL and the Live
WebSocket transport accepted only wss, so a Realtime-compatible server
on the user's own machine (http://127.0.0.1:8010/v1) could not be used.
ADR 0304 already lets endpoints the user typed reach loopback and LAN
over plain http under the default relaxed network mode; the Realtime
path was the one user-supplied endpoint still hard-coded to TLS.

Map an http base URL to ws and let the existing public-network guard
decide, from networkPolicy.mode, whether that plaintext hop is allowed.
Third-party Live endpoints (Gemini) keep wss only, base URLs with
credentials, a query or a fragment stay refused, and a plain ws
endpoint on a proxied route fails closed rather than being tunneled in
the clear. The scheme rules move to a small Electron-free module so
they can be tested directly.

Refs vastsa#1318
Exercise the production transport against a loopback WebSocket and keep the proxied plaintext refusal covered. Add a plain-HTTP fixture mode so the full isolated call journey can validate the user-endpoint policy.
@vastsa
vastsa merged commit 424e82b into vastsa:main Oct 5, 2026
4 checks passed
@vastsa

vastsa commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Thanks for the focused fix. I added coverage for the production loopback WebSocket transport, the proxied-plaintext refusal, and a plain-HTTP mode for the isolated Live Voice fixture. The PR is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants