Skip to content

RFC: Deployer backend tunnel, with the rust-core implementation - #781

Open
filvecchiato wants to merge 27 commits into
mainfrom
rfc/product-secrets-rebase
Open

filvecchiato wants to merge 27 commits into
mainfrom
rfc/product-secrets-rebase

Conversation

@filvecchiato

@filvecchiato filvecchiato commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Tracking issue: #490. The RFC document is @BigTava's, carried over with authorship intact.

A product cannot hold a server-side API key, so the deployer runs a backend that holds it. Backend::request lets a product name that backend and a request against it; the host resolves the identifier to a base URL and its own credential, performs the call, and returns the status, an allowlisted set of headers and the body.

Two credentials, two owners. The third-party key lives in the deployer's backend, the only place it can. The backend's own credential lives in the host, which already ships secrets. The core holds neither and never learns the origin it reached.

Rendered

No SSRF surface, rather than SSRF rules

Moving outbound HTTP into the protocol is what a proxy does, and the objection to a proxy is that it needs SSRF rules. This needs none, because a product cannot express an origin: it supplies an opaque identifier and a relative path, and scheme, host, port and userinfo have no field to travel in.

That holds only while the path and query cannot reconstitute one, which is what host_logic::backend enforces — absolute path, no // prefix, no empty or dot segments, no ?, #, \, space, control or non-ASCII byte, and no %. The percent ban is the load-bearing one: %2e%2e%2f passes a check for the literal characters and becomes traversal once a URL parser normalizes it. Variable data belongs in the query, which the host encodes. Rules reject rather than normalize, so no normalizer has to stay in step across the boundary.

Review notes

  • No permission gate, by design. Backends are deployer-owned and first-party, so the host's registry is the trust boundary. RemotePermission is untouched and the stored-key concern that shaped the earlier enum change does not arise. The posture this leaves — any product can call any registered backend — is stated in the RFC, along with the requirement it puts on deployers: a backend on a host running more than one product must authorize on the forwarded X-Polkadot-Product.
  • The body carries its content type (Json / Form) instead of being bare bytes. Left to guess, fetch labels a string body text/plain and the native clients label nothing, so a backend expecting JSON refuses two hosts out of three — the same per-platform divergence that counted against a proxy in the first place.
  • Response headers are an allowlist, not pass-through and not nothing. content-type so a product knows whether to parse, retry-after and the x-ratelimit-* trio so it can back off. Deliberately absent: set-cookie, which would be ambient authority in the product's realm, and location, which would name the origin the tunnel keeps to itself. The host filters and the core re-screens, so a host that forgets cannot reach a product through this call.
  • Three obligations sit with the host because the core cannot see them: set the path on the parsed base rather than concatenating (a base carrying a query would otherwise swallow the path), do not follow redirects, cap the response. They are stated on the trait and implemented in the CLI host.
  • The capability is optional on purpose. A browser-based host cannot keep a credential from its own users — which is this RFC's opening argument one layer up — so dotli and the web host register nothing and answer Unsupported.
  • The CLI host is a real implementation, not a stub: it starts a loopback echo backend when nothing is configured, which is what makes the generated ts example runnable. Backend/request passes the battery on both the signing and pairing host roles.

BigTava and others added 5 commits September 15, 2026 15:15
Appends RemotePermission::Credential { domain, path, method }, granting
outbound access to one endpoint rather than a whole domain, and carries the
RFC document describing it.

Appended last because the enum is SCALE-encoded into the persisted permission
key: amending an existing variant would re-key every stored decision. The
canonicalization arm normalizes the domain and upper-cases the method so a
case variant cannot open a second storage slot; the path is keyed verbatim.
host_logic::credential owns the grant triple, the request digest a covered
request is signed over, and the sr25519 key the signature is made under.

The key tree hangs off the session's pre-hashed root entropy source, which a
signing host and a paired host both hold, so either derives the same key
locally without an SSO round trip. Its domain separator keys the product-id
layer rather than the caller-supplied layer, which is what keeps it out of
reach of host_derive_entropy: separating at the caller layer would leave a
product able to derive its own credential keys and sign covered requests with
no grant at all.

Digest and grant vectors are pinned literally, since a consuming backend has
to reproduce those bytes to verify.
credential_request_headers gives a host the four X-Polkadot-* values for one
outbound request: it resolves the endpoint to a grant, requires that grant to
be authorized, requires a session, and signs a digest of the request under the
endpoint's key. Exported on NativeProductExecution for the native hosts and as
credentialRequestHeaders on WasmProductRuntime for the browser hosts.

It never prompts. A grant is the standing consent and an HTTP request cannot
raise a dialog, so an endpoint with no grant is refused and the product asks
through request_remote_permission instead. That ask is in turn refused without
a prompt when there is no session to derive an identity from, or the triple
names something a grant cannot cover.

The timestamp and nonce are minted in the core rather than at each boundary,
so every host binds a signature the same way; credential_request_headers_at
takes both as inputs so a signature stays reproducible from a vector.
every_remote_permission gains the variant, which puts it under the existing
trusted-product cases: a blessed product holds a credential grant without a
prompt, exactly as it holds every other remote permission.

The added cases pin what makes this variant different from a domain grant: it
occupies one slot and prompts once, a grant on one endpoint leaves a different
path, method or domain undecided, and a broad domain grant never becomes
authority to sign requests to that domain.

The wire schema hash moves because RemotePermission gained a variant. That is
the fingerprint working: a host pairing an old core with a new client would
otherwise mis-decode the frames silently.
The host attaches an sr25519 identity derived from the user's root entropy and
scoped to one wallet, product and endpoint, rather than a ring VRF proof of
people-set membership. A backend verifies one signature against the key the
request carries, so it needs no chain connection, no ring cache, no verifier
library, and it serves callers who are not people-set members.

The cost is Sybil resistance: the identity meters one wallet rather than one
person. The proof keeps its place under Alternatives, with what it buys and
what it costs, because an endpoint that genuinely needs one-per-person has to
reach for it.

Records the derivation, the signed digest, the backend contract, and why the
domain separator sits on the product-id layer rather than the caller-supplied
one. Host interception coverage is uneven enough that Android and dotli cannot
implement this as written; that is an unresolved question, not a silent gap.
@filvecchiato
filvecchiato requested a review from a team September 15, 2026 13:40
@github-actions github-actions Bot added documentation Improvements or additions to documentation rfc labels Sep 15, 2026
@filvecchiato
filvecchiato marked this pull request as draft September 15, 2026 13:42
@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown

CI Status: 19 required jobs green, 17 passed and 2 skipped by path filter.

All job results
job result
android-bindings success
changes success
changeset-guard success
cli-package success
codegen success
e2e skipped
explorer success
ios-bindings success
ios-swift success
licenses success
playground success
provider-android-bindings success
release-guard success
rust success
ts-client success
ts-debugger success
ts-host success
wasm-provider success
workflow-lint skipped

Commit f5ade387 · run log

@filvecchiato filvecchiato linked an issue Sep 15, 2026 that may be closed by this pull request
8 tasks
Three findings from review, in descending order of how much they mattered.

The test guarding the derivation was vacuous. It compared product-reachable
entropy against the first 32 bytes of the expanded sr25519 secret, which is a
hashed scalar rather than the seed, so the assertion held even for a seed the
product could reach. It now compares seeds, via a credential_seed the keypair
expands, and a second test pins that the comparison is one that can fail.

A grant is keyed by domain, and from_request dropped the port and userinfo, so
one grant stretched across origins: a user granting example.com/session also
authorized signing to a different service on example.com:8443. Both are now
refused rather than silently widened.

The two boundaries disagreed on encoding. The hex helper was reachable only
from wasm, leaving native hosts to invent their own, which would have made the
same wallet verify on one platform and fail on another. Both now return
finished CredentialHeader name/value pairs from one place, and the rule for
which headers a host must strip is exported alongside them rather than living
where no host could call it.
A grant and the request it covers are spelled by different callers: the
product names a triple, the host parses a live URL. Those two paths disagreed.

A granted path was kept verbatim while a request path came back
percent-encoded and dot-resolved, so a product granting "/user profile" held a
grant that "/user%20profile" never matched. And a grant naming a port or
userinfo was accepted even though a request carrying either is refused, which
prompted the user for access no request could exercise.

Both constructors now resolve https://<domain><path> through the same parse,
so a grant and its request land on one value or neither is reachable. The RFC
records the canonical form, the default-origin restriction, and the header
encoding hosts must emit.
The prompt was built from the product's raw triple while the storage key and
enforcement used the canonical one, so the two could name different endpoints.
A product asking for /session/../../admin showed the user something that reads
as living under /session and received a grant over /admin. That defeats the
one thing the variant exists for: letting the user tell what they approved.

Canonicalization now happens before the prompt rather than only on the way to
storage, so the endpoint named in the dialog, the endpoint keyed in storage,
and the endpoint a request is matched against are one value.

The header-stripping helper goes: hosts reach the rule through the exported
is_reserved_header, and nothing but its own test called the Vec wrapper.
@filvecchiato filvecchiato added R2 blocker Must be done for R2 Funding Funding modality workstream Product permissions Cross-product proof, manifest, blessed products and removed documentation Improvements or additions to documentation labels Sep 15, 2026
@filvecchiato filvecchiato self-assigned this Sep 15, 2026
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Sep 15, 2026
@filvecchiato filvecchiato changed the title RFC 0025: Credential-endpoint remote permission, with the rust-core implementation RFC: Credential-endpoint remote permission, with the rust-core implementation Sep 15, 2026
The document carried a metadata table, a sequence diagram, the variant's Rust
definition, the derivation pseudocode, the digest preimage and a section of
implementation notes. docs/rfcs/0001-template.md asks for Summary, Motivation,
Approach, Trade-offs and Open questions, and says the design section stops
where the implementer takes over.

What remains is what a reviewer needs to agree or object: the shape of the
grant, why it is appended rather than folded into Remote, that the prompt names
the canonical endpoint, and the two properties the design rests on — that the
derivation is out of the product's reach and that the host, never the product,
attaches the identity. The byte-level digest and derivation live in
host_logic::credential, pinned by literal vectors.

Alternatives and drawbacks fold into Trade-offs, each saying what it costs
rather than restating what it is.
The shortened document lost the authors table along with the rest of the
metadata block, and the design it now describes is not the one Tiago drafted:
the identity is entropy-derived rather than a ring VRF proof. Frontmatter
carries both names, with owner unchanged.

The index row points at the PR that carries the document, and spells the handle
GitHub actually resolves. `@filippovecchiato`, used on row 0021, is a 404.
@filvecchiato

Copy link
Copy Markdown
Collaborator Author

@lore-app-bot review

One conflict: both sides moved the wire schema hash in the golden. Regenerated;
the merged schema is 8fd68b26b780b249, neither side's value.
Appending RemotePermission::Credential left the bridge's switch non-exhaustive,
which is what broke build, test and the simulator preview.

iOS has no prompt for it yet, and this permission is the prompt: it attaches a
host-derived caller identity to an outbound request. So toDomainRequest answers
nil and the caller refuses. Returning an empty permission list instead would
have granted it silently, since a batched request with nothing to prompt for
answers yes.
@github-actions github-actions Bot added the host-work Needs implementation in one or more host repos label Sep 16, 2026
@filvecchiato
filvecchiato marked this pull request as ready for review September 16, 2026 09:53
The wire schema hash conflicted because both sides moved it. Neither value
describes the merged contract, so the golden is regenerated from it:
19d063ef98e38863.
@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown

iOS simulator preview

Built from f5ade3875, stamped with it in TrUAPICommit.

gh run download 35352955495 --name simulator-preview-f5ade3875
unzip polkadot-app-*.app.zip
xcrun simctl install booted polkadot-app.app
xcrun simctl launch booted io.parity.polkadotapp.develop

Or download it in a browser, which arrives as a zip wrapping
the .app.zip, so it needs unzipping twice.

An arm64 simulator slice, so it needs an Apple Silicon Mac and does not
install on a device. Kept for 14 days, after which the link stops
resolving and a new push rebuilds it.

@TarikGul TarikGul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the parts that carry the weight rather than the prose: discriminants are pinned with Credential last, the separator sits at the product-id layer so host_derive_entropy cannot reach the tree, and the digest binds method, domain, path, query, timestamp, nonce and body under its own label. I also dropped the length prefix from push_field and two tests caught it, so the field boundaries are properly pinned.

One comment inline, on the spec rather than the code.

- **The derivation is out of the product's reach.** It hangs off a namespace that `host_derive_entropy` ([RFC-0007](0007-derive-entropy.md)) cannot address. If a product could derive its own credential key it could sign covered requests with no grant at all, and the permission would mean nothing.
- **The host attaches the identity, never the product.** Hosts strip caller-supplied `X-Polkadot-*` headers before attaching their own, and emit one encoding across platforms so a backend sees the same caller whichever host the request came from.

A backend verifies the signature against the key in the header and meters on that key, never on any other field. It needs an sr25519 implementation and nothing else.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says a backend needs an sr25519 implementation and nothing else, but that leaves replay open. The timestamp and nonce are in the digest so they cannot be swapped onto another request, but nothing tells the verifier to actually check them, so one captured request replays forever.

Worth stating that a backend must reject a stale timestamp and remember nonces inside its window.

@TarikGul TarikGul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a review and had my agents review. One small comment. Overall looks solid - approving so I am not a blocker.

RFC 0025 answered "a product cannot hold a server key" with a caller
identity: a per-endpoint grant plus an entropy-derived sr25519 signature
the host attached to the product's own outbound request. That answers
which user is calling, when a deployer's backend is asking which host.
It also needed outbound interception, which Android cannot do for a
request body and dotli cannot do at all.

The core now carries the request instead. A product names a backend and
a request against it; the host resolves the identifier to a base URL and
its own credential, performs the call, and returns the status, an
allowlisted set of headers and the body. Two credentials, two owners:
the third-party key stays in the deployer's backend, the backend's
credential stays in the host, and the core holds neither.

A product cannot express an origin, so there is no SSRF surface to
screen for; what the core does screen is that the path and query cannot
reconstitute one. Backends are deployer-owned and first-party, so the
host's registry is the trust boundary and no permission variant is
involved: RemotePermission is untouched.

Serves through the optional BackendHost capability, so a host that
registers none answers Unsupported. The CLI host implements it against a
loopback echo backend, which is what makes the generated example runnable
in the battery.
@github-actions github-actions Bot added the javascript Pull requests that update javascript code label Sep 18, 2026
@filvecchiato filvecchiato changed the title RFC: Credential-endpoint remote permission, with the rust-core implementation RFC: Deployer backend tunnel, with the rust-core implementation Sep 18, 2026
The path rule refused `/` and any trailing slash, because the leading
slash's empty segment and a trailing one were treated as interior. Both
are ordinary paths; an empty segment between them still is not.

The CLI host built its client with `unwrap_or_default`, and the default
reqwest client follows up to ten redirects — a silent fallback out of an
obligation the design leans on. It now fails loudly, carries a 30s
timeout, and enforces its response cap against the declared length and
again while streaming, rather than after buffering the whole body.

A native host installs a tunnel for every execution, so "registers no
backends" surfaces there as UnknownBackend, not Unsupported. The trait,
the product-facing docs and the RFC now say which is which.
A product had no way to learn which backend identifiers a host accepts.
It hardcoded one from a vendor's documentation and found out from a
failed call, which on a browser host is indistinguishable from any other
failure. That made "adding a backend needs a host release" a cost only
the host could see.

`list` answers with the identifiers the calling product may reach. It is
product-scoped, so a registry pinning its entries to product ids answers
each product with its own set, and an empty list means this host serves
this product nothing rather than that it has no backends. Identifiers
only: where a backend lives stays host-side, which is why they are ids.

A host with no tunnel still answers Unsupported, so the two questions a
product actually has — can this host do backends, and will it give me
one — have different answers.
UniFFI cannot lower a record defined in another crate out of an async
callback return: the generated Kotlin reached for truapi's RustBuffer
where truapi_server's was expected, and the Android shell would not
compile. Every existing async callback returns a primitive or a
truapi-server type, so nothing had hit it before.

The response leg now carries NativeBackendResponse, mirroring
v01::HostBackendResponse the way NativeDevicePermissionStatus already
mirrors a boundary value, and the adapter converts. The request leg is
unaffected — lowering a foreign record into a call works.

The Kotlin shell also gains the imports it was missing for the types it
names.
The pipeline was drawn as far as the deployer's backend and stopped
there, so the hop that actually uses the third-party key was left
implicit. It is the reason there are two credentials at all: the host
authenticates itself to the backend, and the backend authenticates
itself to the provider.

Nothing said a registered base must be a service the deployer runs.
Pointing an entry straight at the provider works mechanically and puts
that key in the host, where it ships in an app binary or sits in
devtools — the objection this RFC opens with, one layer up. The core
cannot tell the two apart, so it is stated as an obligation.
@filvecchiato
filvecchiato force-pushed the rfc/product-secrets-rebase branch from 9a2048a to fce4d81 Compare September 18, 2026 13:51
@pgherveou

pgherveou commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

the design still leaks secret to anyone spying the traffic getting out of the host
if the creds are sensitive we do need a backend so we don't leak them at all

 PRODUCT                CORE                     HOST (CLI)                  DEPLOYER BACKEND           MELD
 onramp.example         truapi-server            CliBackendHost              onramp.acme.dev            api.meld.io
 holds no secret        holds no secret          holds Bearer token          holds Meld key             third party
      │                      │                        │                           │                        │
      │ backend.request      │                        │                           │                        │
      │  backend "meld"      │                        │                           │                        │
      │  Get /quotes         │                        │                           │                        │
      │  ?source=USD         │                        │                           │                        │
      │  &dest=DOT           │                        │                           │                        │
      │  &amount=100         │                        │                           │                        │
      ├─────────────────────>│                        │                           │                        │
      │   wire 19/0          │                        │                           │                        │
      │                      │ screen_request         │                           │                        │
      │                      │  slug ok               │                           │                        │
      │                      │  path ok: no % ? # \   │                           │                        │
      │                      │  query 3 items, 34B    │                           │                        │
      │                      │  no body on Get        │                           │                        │
      │                      │                        │                           │                        │
      │                      │ product_id from the    │                           │                        │
      │                      │ connection, not the    │                           │                        │
      │                      │ payload                │                           │                        │
      │                      ├───────────────────────>│                           │                        │
      │                      │  backend_request(      │                           │                        │
      │                      │   "onramp.example",    │                           │                        │
      │                      │   request)             │                           │                        │
      │                      │                        │ registry["meld"]          │                        │
      │                      │                        │  base  = https://         │                        │
      │                      │                        │    onramp.acme.dev/api    │                        │
      │                      │                        │  token = hst_7f3c9a2b     │                        │
      │                      │                        │                           │                        │
      │                      │                        │ set_path on parsed base   │                        │
      │                      │                        │  /api + /quotes           │                        │
      │                      │                        │  append_pair each query   │                        │
      │                      │                        │                           │                        │
      │                      │                        │ ### INJECT #1 ###         │                        │
      │                      │                        │ Authorization:            │                        │
      │                      │                        │   Bearer hst_7f3c9a2b     │                        │
      │                      │                        │ X-Polkadot-Product:       │                        │
      │                      │                        │   onramp.example          │                        │
      │                      │                        │ redirects off, 30s cap    │                        │
      │                      │                        ├──────────────────────────>│                        │
      │                      │                        │  GET https://onramp       │                        │
      │                      │                        │  .acme.dev/api/quotes     │                        │
      │                      │                        │  ?source=USD&dest=DOT     │                        │
      │                      │                        │  &amount=100              │                        │
      │                      │                        │                           │ check Bearer           │
      │                      │                        │                           │ authorize + meter on   │
      │                      │                        │                           │ X-Polkadot-Product     │
      │                      │                        │                           │                        │
      │                      │                        │                           │ ### INJECT #2 ###      │
      │                      │                        │                           │ Authorization:         │
      │                      │                        │                           │   BASIC <meld_key>     │
      │                      │                        │                           ├───────────────────────>│
      │                      │                        │                           │  GET /payments/crypto  │
      │                      │                        │                           │      /quote?...        │
      │                      │                        │                           │<───────────────────────┤
      │                      │                        │                           │  200 {"quote": ...}    │
      │                      │                        │<──────────────────────────┤                        │
      │                      │                        │  200                      │                        │
      │                      │                        │  content-type: app/json   │                        │
      │                      │                        │  x-ratelimit-remaining:49 │                        │
      │                      │                        │  set-cookie: sid=...      │                        │
      │                      │                        │  location: https://...    │                        │
      │                      │                        │                           │                        │
      │                      │                        │ header allowlist:         │                        │
      │                      │                        │  keep content-type,       │                        │
      │                      │                        │       x-ratelimit-*       │                        │
      │                      │                        │  drop set-cookie,         │                        │
      │                      │                        │       location            │                        │
      │                      │                        │ stream body, cap 1 MiB    │                        │
      │                      │<───────────────────────┤                           │                        │
      │                      │ screen_response        │                           │                        │
      │                      │  re-apply allowlist    │                           │                        │
      │                      │  <=8 headers, <=512B   │                           │                        │
      │                      │  body <= 1 MiB         │                           │                        │
      │<─────────────────────┤                        │                           │                        │
      │  status 200          │                        │                           │                        │
      │  headers [           │                        │                           │                        │
      │   content-type,      │                        │                           │                        │
      │   x-ratelimit-       │                        │                           │                        │
      │     remaining ]      │                        │                           │                        │
      │  body 0x7b2271...    │                        │                           │                        │
      │                      │                        │                           │                        │

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation Funding Funding modality workstream host-work Needs implementation in one or more host repos javascript Pull requests that update javascript code Product permissions Cross-product proof, manifest, blessed products R2 blocker Must be done for R2 rfc

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFC: Credential-endpoint remote permission

4 participants