Skip to content

Commit 323dee1

Browse files
committed
docs(providers): tighten token grant failure documentation
Signed-off-by: Shiju <shiju@nvidia.com>
1 parent c47fa3a commit 323dee1

3 files changed

Lines changed: 32 additions & 35 deletions

File tree

‎docs/how-it-works/providers/profiles.mdx‎

Lines changed: 16 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -660,16 +660,15 @@ The gateway only brokers an intermediate token for a sandbox principal, and only
660660

661661
### Token Grant Failures
662662

663-
When a token grant fails, OpenShell refuses the protected request before sending any of its bytes upstream. The workload receives `502 Bad Gateway` with an `x-openshell-request-id` header and the body described in [Proxy Error Responses](/observability/logging#proxy-error-responses). The response, the supervisor's OCSF events, and `policy.local` denials never say why the grant failed, because the sandboxed agent can read them. Users authorized to read the sandbox can retrieve the cause from `Sandbox.status.last_token_grant_failure` and `openshell logs`.
663+
When a token grant fails, OpenShell refuses the request before sending any of its bytes upstream. The workload receives `502 Bad Gateway` with an `x-openshell-request-id` header and the body described in [Proxy Error Responses](/observability/logging#proxy-error-responses). The agent can read that response, the OCSF events, and `policy.local` denials, so none of them state the cause. Users who can read the sandbox get it from `Sandbox.status.last_token_grant_failure` and `openshell logs`.
664664

665-
The request ID is a random UUID that the supervisor mints for each HTTP exchange. It names the same exchange everywhere:
665+
The supervisor mints a random UUID request ID for each HTTP exchange and carries it through the grant:
666666

667-
- The supervisor sends it to the token service in the `x-openshell-request-id` request header.
668-
- For `token_exchange` grants, the supervisor sends it to the gateway as gRPC `x-request-id` metadata, and the gateway forwards it to the intermediate token endpoint.
669-
- The supervisor's `Token grant failed` OCSF event carries it as `request_id`. In `openshell logs`, the supervisor's log line with the same message adds the failure code, stage, and upstream HTTP status.
670-
- When the gateway hop fails, the gateway's `dynamic token grant failed` log line carries it with the failure code, stage, and upstream HTTP status.
667+
- To the token service as `x-openshell-request-id`.
668+
- For `token_exchange` grants, to the gateway as gRPC `x-request-id` metadata, which the gateway forwards to the intermediate token endpoint.
669+
- In the supervisor's `Token grant failed` OCSF event as `request_id`, and in the gateway's `dynamic token grant failed` log line. The log lines add the code, stage, and upstream HTTP status.
671670

672-
OpenShell classifies each failure into one of these codes. The recovery action uses the same values as provider refresh status:
671+
Each failure gets one code. Recovery actions match provider refresh status:
673672

674673
| Code | Meaning | Recovery action |
675674
|---|---|---|
@@ -680,16 +679,16 @@ OpenShell classifies each failure into one of these codes. The recovery action u
680679
| `unavailable` | A required service or connection was unavailable. | `retry` |
681680
| `unclassified` | The available evidence does not establish a more specific cause. | `investigate` |
682681

683-
The stage names the step that observed the failure: `configuration`, `workload_identity`, `gateway_rpc`, `intermediate_exchange`, `token_endpoint`, or `injection`. OpenShell maps token-service responses as follows:
682+
The stage is the step that observed the failure: `configuration`, `workload_identity`, `gateway_rpc`, `intermediate_exchange`, `token_endpoint`, or `injection`. OpenShell maps token-service responses as follows:
684683

685-
- A response body over 64 KiB is `unclassified`, whatever its HTTP status.
686-
- HTTP 429 and 5xx responses are `unavailable` unless they carry a recognized extension code, described below. Failures to connect, including TLS certificate or hostname verification failures, are also `unavailable`, but a connection that fails after it is established is `unclassified`. A timeout, including a slow response body, is `timeout`.
687-
- Without a recognized extension code, any other failed HTTP status is `unclassified`, including 401, 403, and 409. OpenShell never infers `provider_authorization_required` from a 401.
688-
- For `token_exchange` grants, the gateway reports `provider_authorization_required` when the user's stored subject credential is missing or expired, and `workload_identity_rejected` when it rejects the supervisor's JWT-SVID. The supervisor accepts these causes only over its verified TLS connection to the gateway.
684+
- A response body over 64 KiB is `unclassified`.
685+
- HTTP 429, 5xx, and failures to connect, including TLS certificate or hostname verification, are `unavailable`. A timeout, including a slow body, is `timeout`.
686+
- Everything else is `unclassified`: other failed statuses including 401, 403, and 409, a connection lost after it is established, and a success without a nonempty, header-compatible `access_token`. OpenShell never infers `provider_authorization_required` from a 401.
687+
- For `token_exchange` grants, the gateway reports `provider_authorization_required` when the user's stored subject credential is missing or expired, and `workload_identity_rejected` when it rejects the supervisor's JWT-SVID. The supervisor accepts these only over its verified TLS connection to the gateway.
689688

690-
Sandbox status and the workload response keep only the code, stage, and HTTP status. The log of the component that called the token service, the gateway or the supervisor, records the same code with the HTTP status and request ID. OpenShell never logs the token service's `error` or `error_description` text, because a token service can echo a submitted credential there. A successful response without a nonempty header-compatible `access_token` is `unclassified`.
689+
Sandbox status and the workload response keep only the code, stage, and HTTP status. Logs add the request ID. OpenShell never logs the token service's `error` or `error_description`, which can echo a submitted credential.
691690

692-
A token service can also state a cause by adopting the `openshell.token-grant.v1` error extension. For example, a service that requires the user to authorize a provider returns HTTP 409 with:
691+
A token service can state a cause with the `openshell.token-grant.v1` error extension. For example, a service that needs the user to authorize a provider returns HTTP 409 with:
693692

694693
```json
695694
{
@@ -698,15 +697,15 @@ A token service can also state a cause by adopting the `openshell.token-grant.v1
698697
}
699698
```
700699

701-
OpenShell accepts this cause only from the configured endpoint over verified HTTPS. It verifies the server certificate and hostname and does not follow redirects. Loopback and Kubernetes-service HTTP endpoints cannot authenticate these signals. The extension recognizes these status and code pairs:
700+
OpenShell accepts the extension only from the configured endpoint over verified HTTPS, without following redirects. Plain HTTP, including loopback and Kubernetes-service endpoints, cannot authenticate it. These status and code pairs are recognized, each with the exact `openshell_error_contract` marker:
702701

703702
- HTTP 409 with `provider_authorization_required`.
704703
- HTTP 401 or 403 with `workload_identity_rejected`.
705704
- HTTP 400 with `configuration_error`.
706705

707-
Each response also requires the exact `openshell_error_contract` marker. OpenShell classifies a response with an unknown contract version or an unrecognized code by its HTTP status alone. A recognized code with a different HTTP status, or one received over plain HTTP, is `unclassified`. OpenShell does not infer identity rejection from generic OAuth errors such as `invalid_client` or `invalid_grant`, which can have more than one cause.
706+
An unknown contract version or code falls back to HTTP status. A recognized code with another status, or over plain HTTP, is `unclassified`. Generic OAuth errors such as `invalid_client` and `invalid_grant` are never read as identity rejection, because they have more than one cause.
708707

709-
Refer to [Read Token Grant Failures](/how-it-works/sandboxes/overview#read-token-grant-failures) for how to read `last_token_grant_failure` and when it clears. MCP-over-HTTP endpoint status also carries the cause as `last_failure`. The no-bytes guarantee excludes connection setup: an inspected CONNECT path can already have completed an upstream TLS handshake before a grant fails.
708+
Refer to [Read Token Grant Failures](/how-it-works/sandboxes/overview#read-token-grant-failures) to read `last_token_grant_failure` and see when it clears. MCP-over-HTTP endpoint status also carries the cause as `last_failure`. The no-bytes guarantee does not cover connection setup: on an inspected CONNECT path, the upstream TLS handshake can finish before a grant fails.
710709

711710
## Provider Instances
712711

‎docs/how-it-works/sandboxes/overview.mdx‎

Lines changed: 15 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -726,9 +726,9 @@ openshell sandbox get my-sandbox --policy-only
726726
727727
### Diagnose Failed Calls to Tool Servers
728728
729-
A sandbox can be `Ready` while an agent's call to an external tool server fails with `fetch failed`. Run `openshell sandbox get my-sandbox` and look under `Tool server connections` for the server's address, `Last result`, and `Reported at`. This information is available for configured endpoints that use MCP over HTTP. It helps you find where the call failed without changing sandbox readiness.
729+
A sandbox can be `Ready` while an agent's call to an external tool server fails with `fetch failed`. Run `openshell sandbox get my-sandbox` and look under `Tool server connections` for the server's address, `Last result`, and `Reported at`. This is available for configured MCP-over-HTTP endpoints. It shows where the call failed and does not change sandbox readiness.
730730
731-
JSON and YAML output include an `endpoint_statuses` list with `last_reported_at`. The protobuf API exposes the corresponding timestamp as `Sandbox.status.endpoint_statuses[].last_reported_time`; SDKs render it using their native or curated time representation. Select the endpoint by its address to read the result directly. Treat `endpoint_id` as opaque.
731+
JSON and YAML output include an `endpoint_statuses` list with `last_reported_at`. The protobuf API exposes the same timestamp as `Sandbox.status.endpoint_statuses[].last_reported_time`, which SDKs render in their own time type. Select an endpoint by its address, not by `endpoint_id`, which is opaque.
732732
733733
For example, use `jq` to select the tool server at `tools.example.com`, path `/mcp`, and port `443`:
734734
@@ -742,16 +742,16 @@ openshell sandbox get my-sandbox --output json | jq -e \
742742
743743
Read `last_result` to choose the next check:
744744
745-
- `NoObservedExchange`: no result has been reported for the current configuration and supervisor session. Try the operation and inspect its logs if no result appears.
746-
- `HttpResponseReceived`: the server returned a final HTTP status below 400, including a protocol upgrade. Informational responses alone do not establish success. Check the client's response for protocol or tool errors; an HTTP 200 response can still contain an error.
745+
- `NoObservedExchange`: nothing has been reported for the current configuration and supervisor session. Try the operation, then check the logs if no result appears.
746+
- `HttpResponseReceived`: the server returned a final HTTP status below 400, including a protocol upgrade. Informational responses alone do not count. Check the client's response for protocol or tool errors, because an HTTP 200 can still carry one.
747747
- `PolicyDenied`: OpenShell denied the request, including MCP protocol-version, request-header, request-body, and HTTP-method checks. Check the sandbox policy and denial logs.
748-
- `CredentialUnavailable`: required credentials were unavailable. Check the endpoint's attached provider and `last_failure`.
748+
- `CredentialUnavailable`: a required credential was unavailable. Check the endpoint's attached provider and `last_failure`.
749749
- `TlsFailed`: TLS setup or the handshake failed. Check certificates and TLS configuration.
750750
- `TransportFailed`: the network exchange failed. Check name resolution, connectivity, and the server process.
751-
- `UpstreamRejected`: the server returned an HTTP status of 400 or higher. Check the server's response and logs.
752-
- `Unspecified`: the result is unrecognized or absent. Do not infer success from it.
751+
- `UpstreamRejected`: the server returned HTTP 400 or higher. Check the server's response and logs.
752+
- `Unspecified`: the result is unrecognized or absent. Do not infer success.
753753
754-
For a provider-credentialed MCP-over-HTTP endpoint, a failed [dynamic token grant](/how-it-works/providers/profiles#token-grant-failures) also carries `last_failure`, which explains the `CredentialUnavailable` result. It describes why OpenShell could not complete the exchange; the tool server can still be healthy. `openshell sandbox get` prints it under the result:
754+
For a provider-credentialed MCP-over-HTTP endpoint, a failed [dynamic token grant](/how-it-works/providers/profiles#token-grant-failures) sets `last_failure`, which explains the `CredentialUnavailable` result. The tool server can still be healthy. `openshell sandbox get` prints it under the result:
755755
756756
```text
757757
Last result: Required OpenShell-managed credential was unavailable.
@@ -761,34 +761,32 @@ Recovery: Reauthorize the provider credential.
761761
Request ID: 7e6d5c4b-3a29-4180-9f6e-5d4c3b2a1908
762762
```
763763
764-
In JSON and YAML output, `last_failure` has `domain`, `code`, `recovery_action`, `provider`, `credential_key`, `stage`, `upstream_http_status`, and `request_id`; it is `null` when no cause was reported. `recovery_action` says who acts next:
764+
`last_failure` has `domain`, `code`, `recovery_action`, `provider`, `credential_key`, `stage`, `upstream_http_status`, and `request_id` in JSON and YAML, and is `null` when no cause was reported. `recovery_action` says who acts next:
765765
766766
- `reauthorize`: the user authorizes the provider again.
767767
- `fix_configuration`: an operator repairs the provider, grant, or workload identity configuration.
768768
- `retry`: the failure can be temporary; retry later.
769769
- `investigate`: an operator inspects the supervisor and gateway logs for the request ID.
770770
771-
`request_id` matches the `x-openshell-request-id` header of the refused response. `provider` and `credential_key` are names, never credential values; they are empty when the grant failed before a credential was selected. Like `last_result`, `last_failure` describes only the last observation: a later exchange replaces it, and a change to the attached provider, such as storing a new credential, clears it.
771+
`request_id` matches the `x-openshell-request-id` header of the refused response. `provider` and `credential_key` are names, never values, and are empty if the grant failed before credential selection. Like `last_result`, `last_failure` is only the last observation: a later exchange replaces it, and a change to the attached provider clears it.
772772
773773
For example, a stopped server can produce `TransportFailed` while the sandbox stays `Ready`. After the server recovers, an HTTP response below 400 updates the result to `HttpResponseReceived`.
774774
775-
Results come from actual traffic and do not expire. A result can remain unchanged after the server stops until another observed exchange or configuration/session reset. Reports combine recent observations and can drop them when full, so this list is not a request history. The address remains available when a result resets to `NoObservedExchange`. To verify current tool availability, run the actual operation.
775+
Results come from actual traffic and do not expire. A result can stay unchanged after the server stops, until another exchange is observed or the configuration or session resets. Reports combine recent observations and can drop them when full, so this is not a request history. The address remains when a result resets to `NoObservedExchange`. To verify current availability, run the operation.
776776
777-
`last_reported_at` is the RFC 3339 rendering of the protobuf `last_reported_time` when the gateway accepted the result. It is empty until a result is reported. New accepted observations advance it, including repeated results; identical report retries do not. If a configuration reset supersedes a report whose acknowledgement was lost, the gateway can accept still-valid pending evidence again and advance this timestamp without another exchange.
777+
`last_reported_at` is the RFC 3339 form of `last_reported_time`, the time the gateway accepted the result. It is empty until a result is reported. Each newly accepted observation advances it, including a repeated result; a retry of an identical report does not. If a configuration reset supersedes a report whose acknowledgement was lost, the gateway can accept the still-valid pending evidence again and advance the timestamp without another exchange.
778778
779-
Addresses use lowercase hosts, canonical paths, and sorted distinct effective ports. Equivalent addresses share one record that combines observations from all callers and ports; it does not establish that every caller or port works. A failure before the HTTP path is known, such as a TLS failure during CONNECT, updates status only when the host and port identify one distinct endpoint. If several configured paths share that host and port, inspect the logs for the failure.
779+
Addresses use lowercase hosts, canonical paths, and sorted distinct effective ports. Equivalent addresses share one record that combines all callers and ports, so it does not show that every caller or port works. A failure before the HTTP path is known, such as a TLS failure during CONNECT, updates status only when the host and port identify one endpoint. Otherwise, check the logs.
780780
781781
### Read Token Grant Failures
782782
783-
`openshell sandbox get` shows `last_token_grant_failure` for the latest classified grant refusal, including REST requests and failures before an HTTP path is known. The protobuf field is `Sandbox.status.last_token_grant_failure`. It carries `destination_host`, `destination_port`, and a `failure` with the bounded fields described above. It describes the attempted call; the remote server can still be healthy.
784-
785-
Read it in JSON or YAML, or select it directly:
783+
`openshell sandbox get` shows `last_token_grant_failure` for the latest classified grant refusal, including REST requests and failures before an HTTP path is known. The protobuf field is `Sandbox.status.last_token_grant_failure`. It has `destination_host`, `destination_port`, and a `failure` with the fields above. It describes the attempted call, not the server's health.
786784
787785
```shell
788786
openshell sandbox get my-sandbox --output json | jq '.last_token_grant_failure'
789787
```
790788
791-
Match `failure.request_id` with the `x-openshell-request-id` from the refused response before starting recovery. A newer grant failure can replace this record. A successful grant for the same destination, provider, and credential clears it; when a failure happened before credential selection, the next successful grant to the same destination clears it. Policy, provider, and supervisor-session resets also clear it. JSON/YAML show `null` when no cause is retained. This is one latest observation, not a request history.
789+
Match `failure.request_id` with the refused response's `x-openshell-request-id` before recovering, because a newer failure replaces this record. A successful grant for the same destination, provider, and credential clears it, or for the same destination if the failure came before credential selection. A policy, provider, or supervisor-session reset also clears it. The value is `null` when nothing is retained.
792790
793791
### Inspect Logs and Activity
794792

0 commit comments

Comments
 (0)