[Enhancement]: support private caller-credential use by native execution plugins

[bbednarski9]

Affected area

Plugins; Rust core runtime; CLI gateway; observability or exporters.

Problem or opportunity

Native execution plugins that make their own provider calls need a supported way to use a caller's provider credential without exposing it in observable request headers.

This limitation follows from the public source:

• Relay's gateway request construction passes observable_headers(headers) to LlmRequest.
• Its header filter intentionally excludes Authorization and provider API keys.
• Switchyard's native execution intercept handles configured routes itself rather than calling Relay's normal forwarding continuation.
• Switchyard's forwarded authentication requires those caller headers.

Consequently, a Switchyard route using forward_auth = true cannot obtain the original caller credential from the gateway-provided request. This is a source-level finding; no new binary or live-provider reproduction is claimed here.

Proposed enhancement

Add an explicit, request-scoped provider-call capability for native execution plugins. It should apply the caller's provider credential privately to an authorized upstream call, without adding the secret to general plugin/event headers.

A host-owned dispatch capability or opaque credential handle could provide this. Define which destinations and provider families may use it, including routing-model calls, answer calls, retries, and fallbacks.

Runtime contract and binding impact

Start with Rust/native plugins and document capability discovery and minimum host/SDK versions. Preserve the same ownership rules if other bindings expose the capability later.

Keep gateway invocation credentials separate from provider credentials. Plugins must not automatically gain permission to forward credentials to arbitrary destinations. Existing observable-header filtering must remain intact.

Alternatives considered

• Deployment-owned api_key_env credentials support shared deployment authentication, but cannot preserve each caller's own provider account.
• Standalone Switchyard supports caller forwarding, but does not provide the native Relay integration.
• Integrations can reject unsupported forwarding configurations until a capability is available.
• Restoring credentials to observable headers would weaken the existing confidentiality boundary.

Acceptance criteria

• A real native-plugin test uses synthetic credentials and a local upstream for Chat Completions and Responses, buffered and streaming.
• Authorized calls receive the correct caller credential and required companion headers; concurrent requests cannot mix credentials.
• Credentials remain absent from observable headers, events, logs, metrics, and exported traces.
• Unauthorized destinations, redirects, and incompatible provider families cannot receive source credentials.
• Routing-model calls, retries, and fallback candidates obey the same explicit policy.
• Gateway invocation tokens never become provider credentials.
• Missing credentials or unsupported capability returns a clear error.
• Existing deployment-owned authentication, keyless targets, ordinary forwarding, and cancellation behavior remain supported.

Related integration change

Switchyard's startup-rejection mitigation is proposed in NVIDIA-NeMo/Switchyard#759. That change rejects unsupported configurations; it does not implement this capability.

[willkill07]

Today, Relay already does something close to this safely:

1. The gateway receives the caller’s request and API key.
2. Relay hides the API key before giving the request to plugins.
3. Relay keeps the real headers, including the API key, in its own private forwarding code.
4. If the plugin calls next, Relay can forward the request to the original provider using those private headers.

So, if the plugin only needs to continue to the same provider the caller originally chose, using next is simpler. It does not need a new provider API, and the plugin never sees the caller’s secret. This still permits changing the model in use without creating or using another client.

The biggest concern I have for this issue/PR is the "trusted" upstream call could be arbitrary and go anywhere. This is a security violation outright and we should not cross this boundary.

The problem starts when the plugin wants to send the request somewhere else—for example, to a routing model, a fallback provider, or a retry endpoint. Relay currently removes the caller’s API key when a plugin selects a new URL. This is important: otherwise, a plugin could send a caller’s secret to any website.

To me, keeping normal next calls as the path for forwarding to the original provider is the right approach. This already preserves the private original headers.