Skip to content

Commit 400e2d7

Browse files
committed
feat(sandbox): add delegated identity for token exchange
Signed-off-by: Gordon Sim <gsim@redhat.com>
1 parent 5b9daab commit 400e2d7

56 files changed

Lines changed: 13893 additions & 3574 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎architecture/compute-runtimes.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -586,7 +586,8 @@ would allow them to claim an existing sandbox identity.
586586
The Kubernetes Secrets credential driver (`openshell-driver-kubernetes-secrets`)
587587
stores secrets in workspace-specific namespaces when `workspace_mode` is managed
588588
or operator. In shared mode, all secrets render into the single configured
589-
namespace.
589+
namespace. Gateway-global secret material has no workspace and always renders
590+
into the configured credential namespace in every workspace mode.
590591

591592
When runtime infrastructure changes, validate the relevant sandbox e2e path and
592593
update the matching driver README if a maintainer-facing constraint changes.

‎architecture/gateway.md‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -455,6 +455,18 @@ state is stored as a separate object scoped to the provider instance through
455455
`objects.scope`. Its non-secret configuration remains inline, while refresh
456456
tokens, client secrets, private keys, and other secret source material are
457457
stored through the active credential driver and represented by opaque handles.
458+
Delegated identity credentials use the same boundary: access and refresh token
459+
bytes exist only in the selected credential backend, while the generic
460+
delegated credential object retains issuer, subject, expiry, revocation state,
461+
and opaque handles. The sandbox record contains only its authorization window
462+
and a reference to that generic credential object. The CLI obtains delegated
463+
material through a browser Authorization Code flow with PKCE separate from its
464+
normal login. It transfers that bundle without writing it to the local login
465+
store and continues to authenticate gateway RPCs with the normal login. The
466+
gateway is therefore the sole owner of the delegated refresh-token chain. A
467+
healthy gateway-owned grant is shared by that user’s delegated sandboxes;
468+
create and extend carry only a relative duration, and the gateway derives the
469+
absolute authorization window from its own clock.
458470
The provider record keeps only the current injectable credential handles and
459471
optional per-credential expiry timestamps. A refresh normally mints one
460472
credential, but a strategy may co-mint several (AWS STS mints the access key,
@@ -468,6 +480,36 @@ credential storage for defense in depth. Multi-replica deployments can use that
468480
default with a shared database and shared key-encryption key, or opt into an
469481
external backend such as Vault or Kubernetes Secrets.
470482

483+
Credential-driver writes and generic object CAS writes form a compensating
484+
transaction. New delegated material is staged before its handles are committed.
485+
A failed CAS deletes the staged handles. If deletion fails, recording the staged
486+
handles as pending deletion retries CAS conflicts and merges into the latest
487+
credential without overwriting concurrent changes. A successful replacement
488+
records old handles as pending deletion before cleanup. Revocation and deletion
489+
first clear active handles and make the generic credential unusable, then delete backend
490+
material. Pending opaque handles remain durable across cleanup failures and a
491+
gateway worker retries them. Final deletion keeps a disabled tombstone until
492+
all recorded backend material is gone. This compensation is not atomic across
493+
stores: a missing parent credential or failed cleanup-record persistence can
494+
still leave untracked material; the gateway logs a cleanup warning.
495+
496+
Delegated credential refresh also uses a short-lived lease in the generic
497+
credential record. Gateway replicas acquire the lease with object-store CAS
498+
before resolving or submitting the refresh token. Non-owners reload the record
499+
until the owner commits the rotated handles. The owner renews its lease with
500+
owner-checked CAS throughout exchange, validation, staging, and commit, and
501+
renews immediately before the bounded HTTP exchange. Renewal cannot revive an
502+
expired lease or overwrite a takeover. Safe reads, validation, renewal, and
503+
commit retries have bounded waits; credential writes finish before compensation
504+
rather than being cancelled mid-write. Renewal failure prevents further exchange
505+
or commit and triggers staged-handle cleanup. Renewal stops at commit or when
506+
the owning call ends; an expired lease permits recovery from a failed replica.
507+
The final CAS verifies that the credential is still active and that the writer
508+
owns an unexpired lease, so revocation cannot race with refresh and restore
509+
usable token handles. This does not make the OAuth exchange atomic with local
510+
persistence: a crash or partition after upstream rotation can still require
511+
reauthorization.
512+
471513
Sandbox workload templates are workspace-scoped gateway resources. Workspace
472514
admins create and delete them; workspace users can read and list them. A
473515
template owns reusable workload intent: image, environment, CPU and memory
@@ -490,7 +532,9 @@ short-lived credentials still fail closed at their recorded expiry.
490532
Credential handles remain bound to the driver that created them. Before the
491533
0.1.0 compatibility boundary, gateways do not migrate inline refresh material
492534
or move handles between credential drivers; operators reconfigure affected
493-
grants when upgrading or changing backends.
535+
grants when upgrading or changing backends. Delegated identity has not shipped
536+
with inline token storage, so development deployments created from an earlier
537+
feature branch must delete and recreate those branch-only records.
494538

495539
### Optimistic Concurrency (CAS)
496540

‎architecture/sandbox-limits.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,8 @@ can contain a valid sequence gap.
128128
| Corporate proxy CONNECT response headers | 8 KiB | Fail the tunnel. |
129129
| Corporate proxy CONNECT handshake | 30 s total | Fail the tunnel; validated-address attempts share the aggregate budget. |
130130
| Token-grant HTTP request | 30 s request and connect | Fail credential resolution. |
131-
| Response-derived token cache TTL | 5 min default; 1 h response cap; 30 s expiry margin | A positive profile `cache_ttl_seconds` override replaces the response-derived calculation. |
131+
| Client-credentials token cache TTL | 5 min default; 1 h response cap; 30 s expiry margin | A positive profile `cache_ttl_seconds` override replaces the response-derived calculation without changing existing client-credentials behavior. |
132+
| Token-exchange token cache TTL | 5 min maximum per entry from insertion | The cap applies to positive profile overrides. Response-derived TTLs apply the 30 s expiry margin after the cap, for a 4 min 30 s maximum. Already-authorized exchanges may finish and populate caches after delegation withdrawal or credential revocation, so this is not a withdrawal-to-containment deadline. |
132133

133134
## Sandbox-Local Surfaces
134135

‎architecture/sandbox.md‎

Lines changed: 60 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,9 @@ identifies the calling binary, checks trust-on-first-use binary identity, reject
6060
unsafe internal destinations, and evaluates the active policy. On Linux, it
6161
maps an accepted proxy connection back to the workload socket by matching the
6262
complete local-to-remote TCP tuple before resolving every process that owns the
63-
socket inode.
63+
socket inode. If fork or descriptor passing creates multiple owners, they must
64+
have compatible policy identities; the proxy denies ambiguous shared ownership
65+
before middleware evaluation or upstream connection.
6466

6567
CONNECT and absolute-form forward HTTP are explicit-proxy adapters over the same
6668
egress pipeline. Each adapter normalizes its request into an egress intent, and
@@ -374,20 +376,69 @@ Provider profiles can also declare dynamic token grants. For matching HTTP
374376
endpoints, the supervisor obtains or exchanges OAuth2 access tokens, caches
375377
them, and injects them before forwarding the request. `client_credentials`
376378
grants use the supervisor SPIFFE JWT-SVID directly as the client assertion.
377-
`token_exchange` grants ask the gateway to broker an intermediate token using a
378-
stored provider subject credential and the gateway's own SPIFFE JWT-SVID; the
379-
supervisor then exchanges that intermediate token for the final upstream token
380-
using its own JWT-SVID. The gateway validates that its own JWT-SVID has the
379+
`token_exchange` grants ask the gateway to broker an intermediate token using
380+
either a stored provider subject credential or the sandbox creator's delegated
381+
OIDC identity, plus the gateway's own SPIFFE JWT-SVID; the supervisor then
382+
exchanges that intermediate token for the final upstream token using its own
383+
JWT-SVID. Delegated identity is opt-in at sandbox creation. The CLI first checks
384+
for a usable gateway-owned credential for the authenticated user. When one is
385+
missing, revoked, or no longer refreshable, the CLI runs a separate device
386+
authorization and transfers its token bundle to the gateway without replacing
387+
the normal CLI login. Sandbox create and extend then send only a per-sandbox
388+
duration; the gateway derives the absolute authorization window from its own
389+
clock. Delegated access and refresh tokens live behind opaque
390+
handles in the gateway's active credential driver; the generic credential and
391+
sandbox records never contain their bytes. Rotation stages replacement handles
392+
before updating metadata and uses a shared-store CAS lease so gateway replicas
393+
cannot submit the same refresh token concurrently. Revocation disables
394+
resolution before cleanup, and deletion retains a disabled tombstone until
395+
cleanup completes. Within one gateway process, sandbox creation and delegation
396+
extension share the gateway-wide state guard with credential deletion so a
397+
credential cannot be deleted between its active-reference check and a new
398+
delegation write. Like the gateway's other cross-object state guards, this
399+
coordination is process-local pending DB-backed multi-writer synchronization.
400+
Only the delegating user can
401+
extend or withdraw that window. Only the delegator can start or interact with
402+
the sandbox. Workspace and platform admins can stop it for containment or
403+
delete it to recover resources, but cannot start it. Other workspace users
404+
cannot start, stop, delete, or interact with it. Ordinary sandboxes retain the
405+
workspace role model. These ownership restrictions govern direct sandbox access,
406+
not confidentiality against configuration administrators. Exchanges load the
407+
current provider profile; delegation does not pin its exchange configuration.
408+
Workspace admins can change workspace-scoped custom profiles, including the
409+
token endpoint, and redirect a subsequent exchange. The gateway sends the raw
410+
delegated access token as the subject token, not the delegated refresh token or
411+
normal CLI login credentials. Both provider-credential and delegated-identity
412+
subject-token paths therefore trust administrators who control exchange
413+
configuration with their subject tokens.
414+
The gateway rejects requests whose authorization checks
415+
observe delegation expiry, withdrawal, missing credential state, or credential
416+
revocation. Withdrawal is not instantaneous containment. Already-authorized
417+
refreshes or exchanges may finish and populate token caches. Provider profile
418+
lowering trims surrounding whitespace from subject-token source and credential
419+
identifiers before the gateway resolves them with exact matching.
420+
The gateway validates that its own JWT-SVID has the
381421
requested audience, a SPIFFE subject, and a non-expired `exp` claim when
382-
present. It also validates that the stored subject credential is declared by the
383-
provider profile, and that the supervisor JWT-SVID is a well-formed
422+
present. For provider-credential subject tokens, it also validates that the
423+
stored subject credential is declared by the provider profile. For delegated
424+
identity subject tokens, it validates that the sandbox was created with active
425+
delegation for the stored credential principal. The gateway also verifies that
426+
the supervisor JWT-SVID is a well-formed
384427
three-segment JWT with a SPIFFE subject in the same trust domain as the gateway
385428
SVID. The gateway verifies the supervisor JWT-SVID signature with JWT bundles
386429
fetched from its SPIFFE Workload API. Token grant endpoints are HTTPS-only
387430
except for loopback and Kubernetes service DNS hosts, and returned access tokens
388-
must be bearer-compatible before they are cached or injected. Token response
431+
must be bearer-compatible before they are cached or injected. Delegated OIDC
432+
discovery and refresh use a dedicated 30-second HTTP client that ignores ambient
433+
proxy settings and rejects redirects. Token response
389434
lifetimes are capped and cached with an expiry margin unless a profile supplies
390-
an explicit cache TTL override. Cache entries are scoped by the sandbox provider
435+
an explicit cache TTL override. Every `token_exchange` cache entry, including
436+
an override, is capped at five minutes from insertion. This limits reuse of each
437+
entry, not elapsed time from delegation withdrawal or credential revocation;
438+
in-flight exchanges can populate caches afterward. Response-derived entries
439+
apply the 30-second margin after that cap and therefore live for at most four
440+
minutes and 30 seconds. This bound does not invalidate an access token that an upstream
441+
service has already received. Cache entries are scoped by the sandbox provider
391442
environment revision so provider credential updates miss the old token cache
392443
without changing endpoint matching semantics. Gateway-brokered intermediate
393444
tokens are cached separately by provider resource version, supervisor SPIFFE

‎crates/openshell-cli/Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,3 +97,4 @@ serde_json = { workspace = true }
9797
temp-env = "0.3"
9898
tokio-stream = { workspace = true }
9999
url = { workspace = true }
100+
wiremock = "0.6"

0 commit comments

Comments
 (0)