RFC: Deployer backend tunnel, with the rust-core implementation - #781
filvecchiato wants to merge 27 commits into
Conversation
Appends RemotePermission::Credential { domain, path, method }, granting
outbound access to one endpoint rather than a whole domain, and carries the
RFC document describing it.
Appended last because the enum is SCALE-encoded into the persisted permission
key: amending an existing variant would re-key every stored decision. The
canonicalization arm normalizes the domain and upper-cases the method so a
case variant cannot open a second storage slot; the path is keyed verbatim.
host_logic::credential owns the grant triple, the request digest a covered request is signed over, and the sr25519 key the signature is made under. The key tree hangs off the session's pre-hashed root entropy source, which a signing host and a paired host both hold, so either derives the same key locally without an SSO round trip. Its domain separator keys the product-id layer rather than the caller-supplied layer, which is what keeps it out of reach of host_derive_entropy: separating at the caller layer would leave a product able to derive its own credential keys and sign covered requests with no grant at all. Digest and grant vectors are pinned literally, since a consuming backend has to reproduce those bytes to verify.
credential_request_headers gives a host the four X-Polkadot-* values for one outbound request: it resolves the endpoint to a grant, requires that grant to be authorized, requires a session, and signs a digest of the request under the endpoint's key. Exported on NativeProductExecution for the native hosts and as credentialRequestHeaders on WasmProductRuntime for the browser hosts. It never prompts. A grant is the standing consent and an HTTP request cannot raise a dialog, so an endpoint with no grant is refused and the product asks through request_remote_permission instead. That ask is in turn refused without a prompt when there is no session to derive an identity from, or the triple names something a grant cannot cover. The timestamp and nonce are minted in the core rather than at each boundary, so every host binds a signature the same way; credential_request_headers_at takes both as inputs so a signature stays reproducible from a vector.
every_remote_permission gains the variant, which puts it under the existing trusted-product cases: a blessed product holds a credential grant without a prompt, exactly as it holds every other remote permission. The added cases pin what makes this variant different from a domain grant: it occupies one slot and prompts once, a grant on one endpoint leaves a different path, method or domain undecided, and a broad domain grant never becomes authority to sign requests to that domain. The wire schema hash moves because RemotePermission gained a variant. That is the fingerprint working: a host pairing an old core with a new client would otherwise mis-decode the frames silently.
The host attaches an sr25519 identity derived from the user's root entropy and scoped to one wallet, product and endpoint, rather than a ring VRF proof of people-set membership. A backend verifies one signature against the key the request carries, so it needs no chain connection, no ring cache, no verifier library, and it serves callers who are not people-set members. The cost is Sybil resistance: the identity meters one wallet rather than one person. The proof keeps its place under Alternatives, with what it buys and what it costs, because an endpoint that genuinely needs one-per-person has to reach for it. Records the derivation, the signed digest, the backend contract, and why the domain separator sits on the product-id layer rather than the caller-supplied one. Host interception coverage is uneven enough that Android and dotli cannot implement this as written; that is an unresolved question, not a silent gap.
|
CI Status: 19 required jobs green, 17 passed and 2 skipped by path filter. All job results
Commit |
Three findings from review, in descending order of how much they mattered. The test guarding the derivation was vacuous. It compared product-reachable entropy against the first 32 bytes of the expanded sr25519 secret, which is a hashed scalar rather than the seed, so the assertion held even for a seed the product could reach. It now compares seeds, via a credential_seed the keypair expands, and a second test pins that the comparison is one that can fail. A grant is keyed by domain, and from_request dropped the port and userinfo, so one grant stretched across origins: a user granting example.com/session also authorized signing to a different service on example.com:8443. Both are now refused rather than silently widened. The two boundaries disagreed on encoding. The hex helper was reachable only from wasm, leaving native hosts to invent their own, which would have made the same wallet verify on one platform and fail on another. Both now return finished CredentialHeader name/value pairs from one place, and the rule for which headers a host must strip is exported alongside them rather than living where no host could call it.
A grant and the request it covers are spelled by different callers: the product names a triple, the host parses a live URL. Those two paths disagreed. A granted path was kept verbatim while a request path came back percent-encoded and dot-resolved, so a product granting "/user profile" held a grant that "/user%20profile" never matched. And a grant naming a port or userinfo was accepted even though a request carrying either is refused, which prompted the user for access no request could exercise. Both constructors now resolve https://<domain><path> through the same parse, so a grant and its request land on one value or neither is reachable. The RFC records the canonical form, the default-origin restriction, and the header encoding hosts must emit.
The prompt was built from the product's raw triple while the storage key and enforcement used the canonical one, so the two could name different endpoints. A product asking for /session/../../admin showed the user something that reads as living under /session and received a grant over /admin. That defeats the one thing the variant exists for: letting the user tell what they approved. Canonicalization now happens before the prompt rather than only on the way to storage, so the endpoint named in the dialog, the endpoint keyed in storage, and the endpoint a request is matched against are one value. The header-stripping helper goes: hosts reach the rule through the exported is_reserved_header, and nothing but its own test called the Vec wrapper.
The document carried a metadata table, a sequence diagram, the variant's Rust definition, the derivation pseudocode, the digest preimage and a section of implementation notes. docs/rfcs/0001-template.md asks for Summary, Motivation, Approach, Trade-offs and Open questions, and says the design section stops where the implementer takes over. What remains is what a reviewer needs to agree or object: the shape of the grant, why it is appended rather than folded into Remote, that the prompt names the canonical endpoint, and the two properties the design rests on — that the derivation is out of the product's reach and that the host, never the product, attaches the identity. The byte-level digest and derivation live in host_logic::credential, pinned by literal vectors. Alternatives and drawbacks fold into Trade-offs, each saying what it costs rather than restating what it is.
The shortened document lost the authors table along with the rest of the metadata block, and the design it now describes is not the one Tiago drafted: the identity is entropy-derived rather than a ring VRF proof. Frontmatter carries both names, with owner unchanged. The index row points at the PR that carries the document, and spells the handle GitHub actually resolves. `@filippovecchiato`, used on row 0021, is a 404.
|
@lore-app-bot review |
One conflict: both sides moved the wire schema hash in the golden. Regenerated; the merged schema is 8fd68b26b780b249, neither side's value.
Appending RemotePermission::Credential left the bridge's switch non-exhaustive, which is what broke build, test and the simulator preview. iOS has no prompt for it yet, and this permission is the prompt: it attaches a host-derived caller identity to an outbound request. So toDomainRequest answers nil and the caller refuses. Returning an empty permission list instead would have granted it silently, since a batched request with nothing to prompt for answers yes.
The wire schema hash conflicted because both sides moved it. Neither value describes the merged contract, so the golden is regenerated from it: 19d063ef98e38863.
iOS simulator previewBuilt from gh run download 35352955495 --name simulator-preview-f5ade3875
unzip polkadot-app-*.app.zip
xcrun simctl install booted polkadot-app.app
xcrun simctl launch booted io.parity.polkadotapp.developOr download it in a browser, which arrives as a zip wrapping An arm64 simulator slice, so it needs an Apple Silicon Mac and does not |
TarikGul
left a comment
There was a problem hiding this comment.
Checked the parts that carry the weight rather than the prose: discriminants are pinned with Credential last, the separator sits at the product-id layer so host_derive_entropy cannot reach the tree, and the digest binds method, domain, path, query, timestamp, nonce and body under its own label. I also dropped the length prefix from push_field and two tests caught it, so the field boundaries are properly pinned.
One comment inline, on the spec rather than the code.
| - **The derivation is out of the product's reach.** It hangs off a namespace that `host_derive_entropy` ([RFC-0007](0007-derive-entropy.md)) cannot address. If a product could derive its own credential key it could sign covered requests with no grant at all, and the permission would mean nothing. | ||
| - **The host attaches the identity, never the product.** Hosts strip caller-supplied `X-Polkadot-*` headers before attaching their own, and emit one encoding across platforms so a backend sees the same caller whichever host the request came from. | ||
|
|
||
| A backend verifies the signature against the key in the header and meters on that key, never on any other field. It needs an sr25519 implementation and nothing else. |
There was a problem hiding this comment.
This says a backend needs an sr25519 implementation and nothing else, but that leaves replay open. The timestamp and nonce are in the digest so they cannot be swapped onto another request, but nothing tells the verifier to actually check them, so one captured request replays forever.
Worth stating that a backend must reject a stale timestamp and remember nonces inside its window.
TarikGul
left a comment
There was a problem hiding this comment.
Did a review and had my agents review. One small comment. Overall looks solid - approving so I am not a blocker.
RFC 0025 answered "a product cannot hold a server key" with a caller identity: a per-endpoint grant plus an entropy-derived sr25519 signature the host attached to the product's own outbound request. That answers which user is calling, when a deployer's backend is asking which host. It also needed outbound interception, which Android cannot do for a request body and dotli cannot do at all. The core now carries the request instead. A product names a backend and a request against it; the host resolves the identifier to a base URL and its own credential, performs the call, and returns the status, an allowlisted set of headers and the body. Two credentials, two owners: the third-party key stays in the deployer's backend, the backend's credential stays in the host, and the core holds neither. A product cannot express an origin, so there is no SSRF surface to screen for; what the core does screen is that the path and query cannot reconstitute one. Backends are deployer-owned and first-party, so the host's registry is the trust boundary and no permission variant is involved: RemotePermission is untouched. Serves through the optional BackendHost capability, so a host that registers none answers Unsupported. The CLI host implements it against a loopback echo backend, which is what makes the generated example runnable in the battery.
The path rule refused `/` and any trailing slash, because the leading slash's empty segment and a trailing one were treated as interior. Both are ordinary paths; an empty segment between them still is not. The CLI host built its client with `unwrap_or_default`, and the default reqwest client follows up to ten redirects — a silent fallback out of an obligation the design leans on. It now fails loudly, carries a 30s timeout, and enforces its response cap against the declared length and again while streaming, rather than after buffering the whole body. A native host installs a tunnel for every execution, so "registers no backends" surfaces there as UnknownBackend, not Unsupported. The trait, the product-facing docs and the RFC now say which is which.
A product had no way to learn which backend identifiers a host accepts. It hardcoded one from a vendor's documentation and found out from a failed call, which on a browser host is indistinguishable from any other failure. That made "adding a backend needs a host release" a cost only the host could see. `list` answers with the identifiers the calling product may reach. It is product-scoped, so a registry pinning its entries to product ids answers each product with its own set, and an empty list means this host serves this product nothing rather than that it has no backends. Identifiers only: where a backend lives stays host-side, which is why they are ids. A host with no tunnel still answers Unsupported, so the two questions a product actually has — can this host do backends, and will it give me one — have different answers.
UniFFI cannot lower a record defined in another crate out of an async callback return: the generated Kotlin reached for truapi's RustBuffer where truapi_server's was expected, and the Android shell would not compile. Every existing async callback returns a primitive or a truapi-server type, so nothing had hit it before. The response leg now carries NativeBackendResponse, mirroring v01::HostBackendResponse the way NativeDevicePermissionStatus already mirrors a boundary value, and the adapter converts. The request leg is unaffected — lowering a foreign record into a call works. The Kotlin shell also gains the imports it was missing for the types it names.
The pipeline was drawn as far as the deployer's backend and stopped there, so the hop that actually uses the third-party key was left implicit. It is the reason there are two credentials at all: the host authenticates itself to the backend, and the backend authenticates itself to the provider. Nothing said a registered base must be a service the deployer runs. Pointing an entry straight at the provider works mechanically and puts that key in the host, where it ships in an app binary or sits in devtools — the objection this RFC opens with, one layer up. The core cannot tell the two apart, so it is stated as an obligation.
9a2048a to
fce4d81
Compare
|
the design still leaks secret to anyone spying the traffic getting out of the host |
Tracking issue: #490. The RFC document is @BigTava's, carried over with authorship intact.
A product cannot hold a server-side API key, so the deployer runs a backend that holds it.
Backend::requestlets a product name that backend and a request against it; the host resolves the identifier to a base URL and its own credential, performs the call, and returns the status, an allowlisted set of headers and the body.Two credentials, two owners. The third-party key lives in the deployer's backend, the only place it can. The backend's own credential lives in the host, which already ships secrets. The core holds neither and never learns the origin it reached.
Rendered
No SSRF surface, rather than SSRF rules
Moving outbound HTTP into the protocol is what a proxy does, and the objection to a proxy is that it needs SSRF rules. This needs none, because a product cannot express an origin: it supplies an opaque identifier and a relative path, and scheme, host, port and userinfo have no field to travel in.
That holds only while the path and query cannot reconstitute one, which is what
host_logic::backendenforces — absolute path, no//prefix, no empty or dot segments, no?,#,\, space, control or non-ASCII byte, and no%. The percent ban is the load-bearing one:%2e%2e%2fpasses a check for the literal characters and becomes traversal once a URL parser normalizes it. Variable data belongs in the query, which the host encodes. Rules reject rather than normalize, so no normalizer has to stay in step across the boundary.Review notes
RemotePermissionis untouched and the stored-key concern that shaped the earlier enum change does not arise. The posture this leaves — any product can call any registered backend — is stated in the RFC, along with the requirement it puts on deployers: a backend on a host running more than one product must authorize on the forwardedX-Polkadot-Product.Json/Form) instead of being bare bytes. Left to guess,fetchlabels a string bodytext/plainand the native clients label nothing, so a backend expecting JSON refuses two hosts out of three — the same per-platform divergence that counted against a proxy in the first place.content-typeso a product knows whether to parse,retry-afterand thex-ratelimit-*trio so it can back off. Deliberately absent:set-cookie, which would be ambient authority in the product's realm, andlocation, which would name the origin the tunnel keeps to itself. The host filters and the core re-screens, so a host that forgets cannot reach a product through this call.Unsupported.tsexample runnable.Backend/requestpasses the battery on both the signing and pairing host roles.