feat: the Gate C vertical — the product installs, runs, and proves its own claims - #4
Merged
Merged
Conversation
The review found both, and both were the same failure: a claim nobody
checked.
1. LICENSING.md was linked from the README and did not exist. GitHub
reported license: null on a repository whose README asserts BUSL-1.1
and "commercial production use requires separate written terms".
That is unenforceable. LICENSE, LICENSING.md, COMMERCIAL_LICENSE.md
and NOTICE now exist, adapted for AAE rather than copied from REMORA:
they name Assured Agent Execution as the Licensed Work and state
explicitly that the pinned REMORA core is a SEPARATE Licensed Work,
so an AAE license grants nothing in REMORA and vice versa. A
deployment runs both and needs to be permitted under both.
2. product/core-release-manifest.json had been stale for two pin bumps
— commit f3e58db, 28 SDK symbols, old digests — while the lock had
moved to 4c85937 and 36. verify_core_pin.py downloaded five contract
assets and the wheel, and never looked at the manifest at all.
Fixed in three places, because one would have been decoration:
- the file is now the manifest the pinned release actually published;
- its digest is pinned in the lock and verified (7 artifacts, not 6),
including the copy checked into this tree — verifying only the
download would leave the in-tree file free to rot again;
- test_pin_manifest_agreement.py compares the hand-written lock to
the release-generated manifest field by field: commit, version,
every artifact digest, the symbol count three ways, and the
consumer contract. That last one matters most — if core widened
its internal namespace set and the lock kept the old list, the
import-boundary test would keep passing while no longer covering
the boundary.
These read JSON only, so they fail before anything is downloaded or
built.
58 compatibility tests pass (was 46).
… Order ToolPack ## The pin Moves to the core built from master 4d7b62a, which closes the three things that kept this product to three of its four decisions: deployment-declared tool classification, the deterministic GROUNDED_READ_ACCEPT path, and per-surface production prerequisites. Every frozen contract artifact is byte-identical to the previous candidate — public_api_v1.json, openapi.json, and all three schemas. Only the wheel moved. That is worth stating precisely, and it is only checkable because the contracts are pinned separately from the code: the SDK surface AAE depends on did not change, so nothing in tests/compatibility needed to. ## The ToolPack Five tools over a Postgres work-order system of record, chosen so each of the four decisions is reachable by a real action rather than a contrived one — read (ACCEPT when grounded), priority and create (VERIFY), close (VERIFY, then effect verification), purge (ESCALATE). Three separations that are the actual product content: - Classification is DATA, not code. tool_metadata.json declares the risk tiers; registry.py holds only callables. Code that could classify itself would be a tool granting itself a risk tier. The file's resolved content is hashed into REMORA's policy identity, so editing a tier invalidates every outstanding lease. - Authority is resolved SERVER-SIDE. work_orders.json says what each work-order id authorizes; the agent names an id and can never assert what it means. That asymmetry is what an ACCEPT rests on. - Credentials are split at the database. aae_worker reads and writes; aae_reader is SELECT only and is what the postcondition reader will hold. A verifier that could write would be reporting on itself, and store.py refuses to fall back from one DSN to the other — that substitution would delete the property silently. The worker holds no DELETE grant anywhere. purge_work_order_history is declared and deliberately unimplemented, so the ESCALATE scenario runs against a tool the deployment genuinely declares rather than an unknown name, which would test the fail-closed floor instead. ## The dependency I am not hiding bundle.py imports nine types from remora.toolcall.* to declare tool semantics. That namespace is not remora.sdk, is not in public_api_v1.json, and is not forbidden either — it is undeclared territory, and no equivalent exists in the SDK, so a deployment cannot author its own semantics without it. test_toolpack_authoring_surface.py pins the exact symbols AND the constructor keywords, so an upstream rename fails in CI on a pin bump rather than in a container as "every proposal abstains". Its last test asserts remora.sdk does NOT yet offer this surface — when that fails, it is the signal to move the imports and delete the file. ## Also docker/control-plane.Dockerfile installs the hash-verified wheel and re-verifies the digest inside the image against the lock it copies in. Twice is not redundant: the first check gates the download, the second makes the image self-describing, so an image built from an unverified wheel fails its own build. 71 compatibility tests pass (was 58).
make up && aae scenarios
6/6 scenarios behaved as documented
One command brings up a control plane built from the hash-verified core
wheel, two Postgres instances, and a migrated system of record. The four
decisions are then exercised against it for real:
ACCEPT a grounded read under signed WO-1201 executes with no human;
the token is refused on replay (ReplayRefusedError)
VERIFY close WO-1202 is held; the agent token is refused approval;
the DOMAIN_EXPERT identity — the role that decision asked for
— approves; the operator executes; the effect is read back on
a SELECT-only credential and attested in the chain as
EFFECT_VERIFIED
ABSTAIN the same read under an authority that does not resolve stops,
and offers nothing to execute
ESCALATE the destructive tool is refused and routed to senior_authority
plus payload binding (binding_refused: "tool-call hash differs from the
approved payload") and role separation.
110 tests pass: 71 compatibility, 39 end-to-end.
## What running it actually found
Six defects, none of which any unit test would have reached:
1. The approver token was `admin`. Both this compose file and the OT
pilot's claimed "the approver token cannot execute" while configuring
a role holding every capability including execute — so the approver
approved its own proposal and then executed it. The e2e test caught it
by trying. Now `reviewer`/`domain_expert`/`senior_authority`, none of
which can act.
2. One approver identity is not enough. REMORA's escalation contract
decides per decision which role may release it: a priority change
takes a reviewer, a production close takes a domain expert, a purge
takes senior authority. The product now reads `required_role` off the
decision and presents the matching identity, and refuses loudly when
it holds none — approving with a weaker identity than the decision
asked for is exactly what the escalation existed to prevent.
3. A refusal arrives as an OUTCOME, not an exception. The binding
scenario wrapped execute() in try/except, saw nothing raised, and
reported "a payload the approver never saw was executed". It had not
been — the system of record was untouched, and `outcome` said
`binding_refused`. A consumer that only catches exceptions treats a
refusal as success. Both the scenario and the tests now read the field.
4. `psql -v` cannot take a dotted variable name, and psql does not
substitute variables inside dollar-quoted strings — so the first
migration's DO blocks reading current_setting('aae.worker_password')
could never have worked. Rewritten with \gexec.
5. A CRLF entrypoint.sh fails in a container as "no such file or
directory", naming the interpreter rather than the script. .gitattributes
forces LF.
6. The Windows console cannot encode the report's arrows, so `aae
scenarios` crashed with UnicodeEncodeError AFTER running every
scenario — governance decisions recorded, operator shown a traceback.
The CLI now forces UTF-8 with errors="replace".
## Two findings left standing, deliberately
- A fully grounded MEDIUM-risk mutation falls through every rule to
default_safe_abstain: no review item, no ResolutionPlan, no path an
approver could take, for an action a signed work order authorizes. High
risk hits evidence_insufficient and ungrounded arguments hit their own
rule; medium-and-grounded matches neither, so declaring a risk tier
leaves a tool worse off than leaving it unknown. The role-separation
scenario is retargeted with that reason recorded in the code rather
than papered over.
- REMORA's evidence bundle is not byte-stable across exports. Exactly one
field moves — manifest.exported_at — which is correct, and means an
outer file digest identifies THAT export rather than the evidence. The
test now asserts nothing ELSE moves, and names any field that starts to.
## Structure
toolpacks/work_order/ registry (callables + credentials), bundle
(semantics + server-side authority resolution),
tool_metadata.json (risk classification as DATA,
hashed into REMORA's policy identity),
work_orders.json (signed authorities)
db/workorders/ the product's own schema, applied by a one-shot
migration under an admin role; the worker and
reader roles it creates hold no DDL
src/aae/ config (identity per decision), postcondition
(the read-only verifier), scenarios, evidence,
cli
docker/ control plane from the verified wheel, with a
preflight that refuses on an unresolvable
toolpack rather than letting every dispatch
report unknown_tool
outsider verifies it with Closes the four remaining items. 126 tests pass, 6/6 scenarios. ## Signed ToolSpec bundle `make sign` computes each callable_digest from the SOURCE of the function actually registered, signs with the deployment's key, and pins the digest into .env. It refuses to sign in both directions: a declared tool with no callable, or a registered callable with no spec. A registered tool without a spec runs unattested; a spec without a tool is an authority nobody can exercise, which rots quietly until someone adds a function to match it. tests/e2e/test_toolspec.py takes the bundle this deployment actually runs and attacks it six ways — widening allowed_targets, rewriting the description an agent reads, escalating credential_scope, reclassifying a destructive tool as routine, removing the postcondition reader, loosening the argument schema — plus a revoked signer, an untrusted signer, the wrong key, and the subtle one: a correctly-signed OLDER bundle. Every signature check passes on that last one, because it really was signed by this deployment. Only the pinned digest refuses it. Three things this surfaced: - The runtime compares REMORA_TOOLSPEC_PINNED_DIGEST against a BARE hex digest. Everything else in the codebase uses the "sha256:" form, so the first version emitted one and the control plane refused the bundle as toolspec_bundle_stale — which reads exactly like a real staleness attack rather than a formatting mistake. - A refused bundle surfaces as a 500 on every request rather than a startup refusal. Strict mode with an unusable bundle should fail closed where an operator sees it. - callable_digest is recorded but never checked: verify_callable exists upstream and nothing calls it at dispatch. Computing the true digest is still worth it — it is signed, and it becomes enforced the moment core wires the check — and a test asserts it matches the deployed source, so the field is not decoration in the meantime. ## Operator console A separate image that imports no REMORA module and holds no policy. It shells out to the product CLI for the scenarios rather than keeping its own copy: a console that could show green while the CLI showed red would leave nobody able to say which one described the product. The posture panel is the reason it exists. A console that only showed activity would let a deployment look healthy while running unpinned, unenforced, or with a chain that no longer verifies. Running it found a real config bug: Config.from_env demanded AAE_READER_PASSWORD even when the full reader DSN was supplied, so the console's scenario run failed on a credential it does not need. ## The CLI's execute was broken, and the docs caught it cmd_execute passed a dict where the SDK wants a ToolCall — the same mistake the scenarios made, left behind in the CLI. Writing the onboarding walkthrough and then RUNNING it is what found it. It now takes the review item id, the tool name, the arguments, the intent and the environment, because the approval is bound to a hash over all of them. Restating the whole call is what lets the server detect that you are executing something other than what was approved. It also exits 1 on a refusal. binding_refused arrives as an OUTCOME, not an exception, so a script that only checked for a raised error would treat it as success — exactly the mistake this product's own scenario made. ## Documentation README rewritten for a product that runs, with a Known gaps section naming five, two of them upstream and open. docs/ARCHITECTURE.md — what runs where and why each boundary is where it is, with the test that checks each claim named at the end of its section. docs/ONBOARDING.md — 30 minutes, no research repo required, and half of it is trying to get past the controls rather than watching them work. Every command in it was executed against the running stack before it was written down; the CLI bug above is what that caught. docs/OPERATIONS.md — signing, rotation (and what rotation does to evidence already signed), evidence export including how to verify an archive WITHOUT this product, and what each non-terminal effect status means for an incident.
…achine
Wiped .venv, dist, .env and every volume, then followed the onboarding
guide as written. It failed three times, and all three were real.
1. `make` is not installed on a default Windows machine. The install
instruction for a product whose pitch is "one command" cannot be
"first install make". Every target now delegates to run.py, so there
is one implementation and the Makefile cannot drift from the docs.
2. The runner was named aae.py, which SHADOWS the aae package in src/ —
`from aae.config import ...` inside a script resolved to the runner
instead. Renamed to run.py. A latent trap for anything else importing
aae from the repository root, and only a from-scratch run finds it.
3. bootstrap_env.py hardcoded AAE_API_PORT=8080, which was already taken
on this machine, so `up` died on "port is already allocated". It now
probes for a free port and prints the URLs it chose. A fixed default
is wrong for everyone already running something on 8080, and that is
a poor first experience.
Verified from a genuinely clean slate afterwards:
python run.py up → one command, ends with the two URLs
python run.py scenarios → 6/6
python run.py verify → 71 compatibility + 55 end-to-end
python run.py doctor → pinned core, execution surface only
console /api/posture → toolspec signed + pinned, chain valid
Docs now use the commands that exist, and say the venv note the `aae`
console script needs. Every command in ONBOARDING.md was executed against
the running stack before it was written down — which is how the CLI's
broken `execute` was found in the previous commit, and these three here.
Two jobs. The contract job needs no Docker and fails in seconds if the pin, the manifest agreement or the import boundary is wrong. The vertical job runs `python run.py up` — the same command the README gives a new developer — then the four decisions, the end-to-end suite and the ToolSpec verification. If that step breaks, the install instructions are broken, which is exactly what we want CI to notice. Running it is what found everything interesting this product got wrong: an approver token that could also execute, a refusal that arrives as an outcome rather than an exception, a CRLF entrypoint failing as "no such file or directory", a hardcoded port, and a runner module shadowing the package it was meant to run. None of those were reachable by unit tests.
darklordVirtual
added a commit
that referenced
this pull request
Aug 6, 2026
PR #4 was squash-merged while this branch kept moving, so main holds the early work as one commit and this branch holds it as its originals plus everything since. Every file main has that this branch lacks is a deliberate removal here: Makefile run.py became the single implementation docs/ARCHITECTURE.md replaced by docs/architecture.md docs/ONBOARDING.md folded into the README and the attack tutorial docs/OPERATIONS.md replaced by docs/operations.md tool_specs.signed.json generated per install, now gitignored Checked with `git diff --diff-filter=A feat/gate-c-vertical origin/main` before merging: that list is exhaustive, so `-s ours` drops nothing unintentional. The doc renames are lowercase-only changes that a case-insensitive filesystem hides — verified against `git ls-tree`, not the working directory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
126 tests pass: 71 pinned-core contract, 55 end-to-end against a running stack.
What now runs
The VERIFY trace is the whole product in six lines: a production write was held; the agent that proposed it was refused the right to approve it; the identity the decision named released it; the operator executed; and a separate process holding a
SELECT-only credential went and looked at the database.The two P0 findings from the review
Licensing.
LICENSING.mdwas linked from the README and did not exist; GitHub reportedlicense: nullon a repository asserting BUSL-1.1. Now present and adapted for AAE, stating explicitly that the pinned REMORA core is a separate Licensed Work — an AAE license grants nothing in REMORA and vice versa.Manifest drift.
core-release-manifest.jsonhad been stale for two pin bumps — commitf3e58db, 28 SDK symbols — while the lock had moved to4c85937and 36.verify_core_pin.pynever looked at it. Fixed in three places: the file is now the one the release published, its digest is pinned and verified including the in-tree copy, andtest_pin_manifest_agreement.pycompares the hand-written lock to the release-generated manifest field by field.Upstream
Three core candidates, every one with the SDK contract byte-identical — only the wheel moved each time.
.2— deployment-declared tool classification,GROUNDED_READ_ACCEPT, per-surface production prerequisites. Without the first, ACCEPT was structurally unreachable for any tool a product names itself. Without the second, classification alone changed nothing: the policy-only kernel has no oracle, so the probabilistic ACCEPT could not fire there at all..3— the risk profile resolved against the working directory, so a wheel install refused to start in production mode while the file sat insite-packages. Found by installing the wheel in a container, which is what any deployment does.Also fixed upstream along the way: a pinned
target_environmentcould lower risk; enabling an opt-in ACCEPT path moved no policy hash; andconftest.pysilently skipped two tests asserting that an unknown risk tier in a production environment can never ACCEPT.What running it found
Every one of these was invisible to unit tests.
admin. This compose file and REMORA's OT pilot both claimed "the approver token cannot execute" while configuring a role holding every capability includingexecute. The e2e test caught it by trying. Nowreviewer/domain_expert/senior_authority, and the product readsrequired_roleoff each decision rather than holding one credential that can approve anything.execute()in try/except and reported "a payload the approver never saw was executed". It had not been — the system of record was untouched andoutcomesaidbinding_refused. Anything integrating with the SDK must read the field.executepassed a dict where the SDK wants aToolCall— found by writing the onboarding walkthrough and then running it.entrypoint.shfails in a container as "no such file or directory". The Windows console crashedscenariosafter every decision was recorded.psql -vcannot take a dotted variable name. The runner namedaae.pyshadowed theaaepackage.AAE_API_PORT=8080was already taken.Deliberately left standing
default_safe_abstain— no review item, noResolutionPlan, no path an approver could take, for an action a signed work order authorizes. Declaring a risk tier leaves a tool worse off than leaving it unknown. Recorded in the code where the scenario had to be retargeted, and in README "Known gaps".callable_digestis signed and true but not enforced:verify_callableexists upstream and nothing calls it at dispatch.Docs
docs/ARCHITECTURE.mdnames the test that checks each claim at the end of its section.docs/ONBOARDING.mdis 30 minutes and half of it is trying to get past the controls.docs/OPERATIONS.mdcovers signing, what rotation does to evidence already signed, and how to verify an evidence archive without this product.