feat: audit follow-ups — console hardening, backup/restore, SBOM and the docs rebuild - #5
Merged
Merged
Conversation
The review found both, and both were the same failure: a claim nobody
checked.
1. LICENSING.md was linked from the README and did not exist. GitHub
reported license: null on a repository whose README asserts BUSL-1.1
and "commercial production use requires separate written terms".
That is unenforceable. LICENSE, LICENSING.md, COMMERCIAL_LICENSE.md
and NOTICE now exist, adapted for AAE rather than copied from REMORA:
they name Assured Agent Execution as the Licensed Work and state
explicitly that the pinned REMORA core is a SEPARATE Licensed Work,
so an AAE license grants nothing in REMORA and vice versa. A
deployment runs both and needs to be permitted under both.
2. product/core-release-manifest.json had been stale for two pin bumps
— commit f3e58db, 28 SDK symbols, old digests — while the lock had
moved to 4c85937 and 36. verify_core_pin.py downloaded five contract
assets and the wheel, and never looked at the manifest at all.
Fixed in three places, because one would have been decoration:
- the file is now the manifest the pinned release actually published;
- its digest is pinned in the lock and verified (7 artifacts, not 6),
including the copy checked into this tree — verifying only the
download would leave the in-tree file free to rot again;
- test_pin_manifest_agreement.py compares the hand-written lock to
the release-generated manifest field by field: commit, version,
every artifact digest, the symbol count three ways, and the
consumer contract. That last one matters most — if core widened
its internal namespace set and the lock kept the old list, the
import-boundary test would keep passing while no longer covering
the boundary.
These read JSON only, so they fail before anything is downloaded or
built.
58 compatibility tests pass (was 46).
… Order ToolPack ## The pin Moves to the core built from master 4d7b62a, which closes the three things that kept this product to three of its four decisions: deployment-declared tool classification, the deterministic GROUNDED_READ_ACCEPT path, and per-surface production prerequisites. Every frozen contract artifact is byte-identical to the previous candidate — public_api_v1.json, openapi.json, and all three schemas. Only the wheel moved. That is worth stating precisely, and it is only checkable because the contracts are pinned separately from the code: the SDK surface AAE depends on did not change, so nothing in tests/compatibility needed to. ## The ToolPack Five tools over a Postgres work-order system of record, chosen so each of the four decisions is reachable by a real action rather than a contrived one — read (ACCEPT when grounded), priority and create (VERIFY), close (VERIFY, then effect verification), purge (ESCALATE). Three separations that are the actual product content: - Classification is DATA, not code. tool_metadata.json declares the risk tiers; registry.py holds only callables. Code that could classify itself would be a tool granting itself a risk tier. The file's resolved content is hashed into REMORA's policy identity, so editing a tier invalidates every outstanding lease. - Authority is resolved SERVER-SIDE. work_orders.json says what each work-order id authorizes; the agent names an id and can never assert what it means. That asymmetry is what an ACCEPT rests on. - Credentials are split at the database. aae_worker reads and writes; aae_reader is SELECT only and is what the postcondition reader will hold. A verifier that could write would be reporting on itself, and store.py refuses to fall back from one DSN to the other — that substitution would delete the property silently. The worker holds no DELETE grant anywhere. purge_work_order_history is declared and deliberately unimplemented, so the ESCALATE scenario runs against a tool the deployment genuinely declares rather than an unknown name, which would test the fail-closed floor instead. ## The dependency I am not hiding bundle.py imports nine types from remora.toolcall.* to declare tool semantics. That namespace is not remora.sdk, is not in public_api_v1.json, and is not forbidden either — it is undeclared territory, and no equivalent exists in the SDK, so a deployment cannot author its own semantics without it. test_toolpack_authoring_surface.py pins the exact symbols AND the constructor keywords, so an upstream rename fails in CI on a pin bump rather than in a container as "every proposal abstains". Its last test asserts remora.sdk does NOT yet offer this surface — when that fails, it is the signal to move the imports and delete the file. ## Also docker/control-plane.Dockerfile installs the hash-verified wheel and re-verifies the digest inside the image against the lock it copies in. Twice is not redundant: the first check gates the download, the second makes the image self-describing, so an image built from an unverified wheel fails its own build. 71 compatibility tests pass (was 58).
make up && aae scenarios
6/6 scenarios behaved as documented
One command brings up a control plane built from the hash-verified core
wheel, two Postgres instances, and a migrated system of record. The four
decisions are then exercised against it for real:
ACCEPT a grounded read under signed WO-1201 executes with no human;
the token is refused on replay (ReplayRefusedError)
VERIFY close WO-1202 is held; the agent token is refused approval;
the DOMAIN_EXPERT identity — the role that decision asked for
— approves; the operator executes; the effect is read back on
a SELECT-only credential and attested in the chain as
EFFECT_VERIFIED
ABSTAIN the same read under an authority that does not resolve stops,
and offers nothing to execute
ESCALATE the destructive tool is refused and routed to senior_authority
plus payload binding (binding_refused: "tool-call hash differs from the
approved payload") and role separation.
110 tests pass: 71 compatibility, 39 end-to-end.
## What running it actually found
Six defects, none of which any unit test would have reached:
1. The approver token was `admin`. Both this compose file and the OT
pilot's claimed "the approver token cannot execute" while configuring
a role holding every capability including execute — so the approver
approved its own proposal and then executed it. The e2e test caught it
by trying. Now `reviewer`/`domain_expert`/`senior_authority`, none of
which can act.
2. One approver identity is not enough. REMORA's escalation contract
decides per decision which role may release it: a priority change
takes a reviewer, a production close takes a domain expert, a purge
takes senior authority. The product now reads `required_role` off the
decision and presents the matching identity, and refuses loudly when
it holds none — approving with a weaker identity than the decision
asked for is exactly what the escalation existed to prevent.
3. A refusal arrives as an OUTCOME, not an exception. The binding
scenario wrapped execute() in try/except, saw nothing raised, and
reported "a payload the approver never saw was executed". It had not
been — the system of record was untouched, and `outcome` said
`binding_refused`. A consumer that only catches exceptions treats a
refusal as success. Both the scenario and the tests now read the field.
4. `psql -v` cannot take a dotted variable name, and psql does not
substitute variables inside dollar-quoted strings — so the first
migration's DO blocks reading current_setting('aae.worker_password')
could never have worked. Rewritten with \gexec.
5. A CRLF entrypoint.sh fails in a container as "no such file or
directory", naming the interpreter rather than the script. .gitattributes
forces LF.
6. The Windows console cannot encode the report's arrows, so `aae
scenarios` crashed with UnicodeEncodeError AFTER running every
scenario — governance decisions recorded, operator shown a traceback.
The CLI now forces UTF-8 with errors="replace".
## Two findings left standing, deliberately
- A fully grounded MEDIUM-risk mutation falls through every rule to
default_safe_abstain: no review item, no ResolutionPlan, no path an
approver could take, for an action a signed work order authorizes. High
risk hits evidence_insufficient and ungrounded arguments hit their own
rule; medium-and-grounded matches neither, so declaring a risk tier
leaves a tool worse off than leaving it unknown. The role-separation
scenario is retargeted with that reason recorded in the code rather
than papered over.
- REMORA's evidence bundle is not byte-stable across exports. Exactly one
field moves — manifest.exported_at — which is correct, and means an
outer file digest identifies THAT export rather than the evidence. The
test now asserts nothing ELSE moves, and names any field that starts to.
## Structure
toolpacks/work_order/ registry (callables + credentials), bundle
(semantics + server-side authority resolution),
tool_metadata.json (risk classification as DATA,
hashed into REMORA's policy identity),
work_orders.json (signed authorities)
db/workorders/ the product's own schema, applied by a one-shot
migration under an admin role; the worker and
reader roles it creates hold no DDL
src/aae/ config (identity per decision), postcondition
(the read-only verifier), scenarios, evidence,
cli
docker/ control plane from the verified wheel, with a
preflight that refuses on an unresolvable
toolpack rather than letting every dispatch
report unknown_tool
outsider verifies it with Closes the four remaining items. 126 tests pass, 6/6 scenarios. ## Signed ToolSpec bundle `make sign` computes each callable_digest from the SOURCE of the function actually registered, signs with the deployment's key, and pins the digest into .env. It refuses to sign in both directions: a declared tool with no callable, or a registered callable with no spec. A registered tool without a spec runs unattested; a spec without a tool is an authority nobody can exercise, which rots quietly until someone adds a function to match it. tests/e2e/test_toolspec.py takes the bundle this deployment actually runs and attacks it six ways — widening allowed_targets, rewriting the description an agent reads, escalating credential_scope, reclassifying a destructive tool as routine, removing the postcondition reader, loosening the argument schema — plus a revoked signer, an untrusted signer, the wrong key, and the subtle one: a correctly-signed OLDER bundle. Every signature check passes on that last one, because it really was signed by this deployment. Only the pinned digest refuses it. Three things this surfaced: - The runtime compares REMORA_TOOLSPEC_PINNED_DIGEST against a BARE hex digest. Everything else in the codebase uses the "sha256:" form, so the first version emitted one and the control plane refused the bundle as toolspec_bundle_stale — which reads exactly like a real staleness attack rather than a formatting mistake. - A refused bundle surfaces as a 500 on every request rather than a startup refusal. Strict mode with an unusable bundle should fail closed where an operator sees it. - callable_digest is recorded but never checked: verify_callable exists upstream and nothing calls it at dispatch. Computing the true digest is still worth it — it is signed, and it becomes enforced the moment core wires the check — and a test asserts it matches the deployed source, so the field is not decoration in the meantime. ## Operator console A separate image that imports no REMORA module and holds no policy. It shells out to the product CLI for the scenarios rather than keeping its own copy: a console that could show green while the CLI showed red would leave nobody able to say which one described the product. The posture panel is the reason it exists. A console that only showed activity would let a deployment look healthy while running unpinned, unenforced, or with a chain that no longer verifies. Running it found a real config bug: Config.from_env demanded AAE_READER_PASSWORD even when the full reader DSN was supplied, so the console's scenario run failed on a credential it does not need. ## The CLI's execute was broken, and the docs caught it cmd_execute passed a dict where the SDK wants a ToolCall — the same mistake the scenarios made, left behind in the CLI. Writing the onboarding walkthrough and then RUNNING it is what found it. It now takes the review item id, the tool name, the arguments, the intent and the environment, because the approval is bound to a hash over all of them. Restating the whole call is what lets the server detect that you are executing something other than what was approved. It also exits 1 on a refusal. binding_refused arrives as an OUTCOME, not an exception, so a script that only checked for a raised error would treat it as success — exactly the mistake this product's own scenario made. ## Documentation README rewritten for a product that runs, with a Known gaps section naming five, two of them upstream and open. docs/ARCHITECTURE.md — what runs where and why each boundary is where it is, with the test that checks each claim named at the end of its section. docs/ONBOARDING.md — 30 minutes, no research repo required, and half of it is trying to get past the controls rather than watching them work. Every command in it was executed against the running stack before it was written down; the CLI bug above is what that caught. docs/OPERATIONS.md — signing, rotation (and what rotation does to evidence already signed), evidence export including how to verify an archive WITHOUT this product, and what each non-terminal effect status means for an incident.
…achine
Wiped .venv, dist, .env and every volume, then followed the onboarding
guide as written. It failed three times, and all three were real.
1. `make` is not installed on a default Windows machine. The install
instruction for a product whose pitch is "one command" cannot be
"first install make". Every target now delegates to run.py, so there
is one implementation and the Makefile cannot drift from the docs.
2. The runner was named aae.py, which SHADOWS the aae package in src/ —
`from aae.config import ...` inside a script resolved to the runner
instead. Renamed to run.py. A latent trap for anything else importing
aae from the repository root, and only a from-scratch run finds it.
3. bootstrap_env.py hardcoded AAE_API_PORT=8080, which was already taken
on this machine, so `up` died on "port is already allocated". It now
probes for a free port and prints the URLs it chose. A fixed default
is wrong for everyone already running something on 8080, and that is
a poor first experience.
Verified from a genuinely clean slate afterwards:
python run.py up → one command, ends with the two URLs
python run.py scenarios → 6/6
python run.py verify → 71 compatibility + 55 end-to-end
python run.py doctor → pinned core, execution surface only
console /api/posture → toolspec signed + pinned, chain valid
Docs now use the commands that exist, and say the venv note the `aae`
console script needs. Every command in ONBOARDING.md was executed against
the running stack before it was written down — which is how the CLI's
broken `execute` was found in the previous commit, and these three here.
Two jobs. The contract job needs no Docker and fails in seconds if the pin, the manifest agreement or the import boundary is wrong. The vertical job runs `python run.py up` — the same command the README gives a new developer — then the four decisions, the end-to-end suite and the ToolSpec verification. If that step breaks, the install instructions are broken, which is exactly what we want CI to notice. Running it is what found everything interesting this product got wrong: an approver token that could also execute, a refusal that arrives as an outcome rather than an exception, a CRLF entrypoint failing as "no such file or directory", a hardcoded port, and a runner module shadowing the package it was meant to run. None of those were reachable by unit tests.
Measured before cutting. Documentation was 847 lines across four files, which is more than the product it describes deserves and more than anyone evaluating it will read. ## Docs: 847 → 325 lines, four files → two README.md (146) is now the single front door: what it is, two commands to install, the trace it prints, three ways to try to break it, the boundary table, and an honest maturity section. It leads with the thing that is actually interesting — a production write held, released by the identity the DECISION named, then confirmed in the database by a process holding SELECT and nothing else. docs/HOW-IT-WORKS.md (179) absorbs the design reasoning and the operator essentials from the three deleted files, and keeps the property that made ARCHITECTURE.md worth having: every section names the test that checks its claim. Deleted ONBOARDING.md, OPERATIONS.md and ARCHITECTURE.md. The onboarding guide's best half — trying to get PAST the controls — is now in the README where someone evaluating this will actually meet it. ## scenarios.py: 385 → 300 lines Eighteen `return Outcome(...)` calls each repeated the scenario's own title. A small `Run` object carries the name and the trace, so a scenario now reads as a sequence of steps and one verdict. Behaviour unchanged: 6/6. The scenario functions also lost their `scenario_` prefix and their unused third parameter, left over from when a single approver client was passed in. ## One entry point Deleted the Makefile. It delegated every target to run.py, so it was a second name for the same thing — and `make` is not installed on a default Windows machine anyway. ## docker-compose.yml: 249 → 198 lines The comments were written while working out why each setting mattered. That reasoning belongs in docs/HOW-IT-WORKS.md; what stays is what an operator needs while reading the file itself. ## Ten dangling `make` references, all in error messages `run.py doctor` said "try: make up". `sign_toolpack.py` said "run `make sign`". Four test skips and two config errors said the same. Every one of them is a message a user meets when something is already wrong, telling them to run a command that no longer exists. All now name `python run.py ...`. Verified after: 126 tests pass (71 contract, 55 end-to-end), 6/6 scenarios, compose config valid, no `make` reference left anywhere.
…raw dict Two commands had zero references anywhere — not in docs, not in tests, not in the console. Untested surface looks like a capability, and nobody finds out otherwise until an operator reaches for it during an incident. Both turned out to be broken in ways that matter. ## verify-effect exited non-zero on a VERIFIED effect It branched on `EffectStatus.is_terminal`, which asks "is this a settled answer?" — and VERIFIED and UNSUPPORTED are both settled and both fine. Only MISMATCH is a finding. A CI job wired to this command would have failed on every effect it successfully confirmed. Conflating "final" with "bad" is precisely the class of mistake this product exists to prevent, and docs/HOW-IT-WORKS.md had the same error in its status table — UNSUPPORTED listed as non-terminal when it is terminal. Both fixed, and the table now separates the two questions instead of implying they are one. ## lifecycle printed the whole event mapping on one line Every hash at full length, every internal key. That is a record, not a trail. It now names what happened, to which tool, under which authority, which signed ToolSpec allowed it, the hash the approval is welded to, and its position in the chain — hashes truncated to eight characters, because in a trail a hash carries one useful bit: whether it matches the line above. `--json` keeps the unabridged record for reconciliation. The rendered trail is now in the README, because it is the clearest single artifact this product produces. ## Also `evidence._plain` became `evidence.plain` — the CLI needed it, and importing a private name across modules is a smell that outlives the convenience. tests/e2e/test_cli.py (8 tests) covers doctor, propose, lifecycle in both views, all three verify-effect outcomes, and an evidence export whose every file is re-hashed against its manifest. 134 tests pass (71 contract, 63 end-to-end), 6/6 scenarios.
…ics to
anyone who can reach the port
A survey of the RUNNING containers, not the compose file. Everything below
was inherited from an upstream pilot whose own comments said "local pilot
only", and carried into a product that describes itself as governance.
## Found
readonly_rootfs=false capdrop=[] no_new_priv=[] mem=0 pids=<nil>
/metrics → 200 without a token
ports bound 0.0.0.0
Writable root filesystems, the full default capability set (CHOWN, SETUID,
NET_RAW…), no memory or PID ceiling, every published port on every
interface, and a Prometheus endpoint disclosing decision volumes and
auth-failure counts to anyone who could reach it. That last pair is the
worst combination: on a laptop on a shared network, the governance API and
the work-order data were reachable by everyone on the segment, and the
metrics told them how much traffic this deployment governs and how often
auth fails.
## Fixed
- `cap_drop: ALL` and `no-new-privileges` on all four services.
- `read_only: true` + a 64 MB tmpfs on the two application containers. The
databases keep a writable data volume and are bounded by capabilities
instead; each gets back only the five capabilities postgres needs to drop
from root in its own entrypoint.
- Memory and PID ceilings everywhere. Without either, one runaway process
takes the host down with it — including the databases holding the audit
chain.
- Every published port binds 127.0.0.1. The governance state database
publishes nothing at all.
- `REMORA_PROMETHEUS_PUBLIC` defaults to 0. `/v1/metrics` is the
authenticated view; `AAE_PUBLIC_METRICS=1` opts back in for a local
scrape.
## Asserted, not just configured
tests/e2e/test_hardening.py (26 tests) checks `docker inspect` and live
HTTP, never the compose file — what is running is the only thing that
protects anyone. It also pins the two properties that must NOT change:
`/` and `/v1/health` stay open, because a liveness probe that needs a
credential fails for the wrong reason during an incident; and no configured
secret may appear in the console's posture response.
160 tests pass (71 contract, 89 end-to-end), 6/6 scenarios, full stack
rebuilt from scratch under the new limits.
A governance system whose tamper-evident chain cannot survive a disk failure has an expiry date. `run.py backup` takes both databases and the signed ToolSpec bundle with a manifest hashing all of it; `run.py restore` verifies every digest BEFORE applying anything, because restoring a corrupted dump into a live deployment turns a detectable problem into a silent one. Signing keys are deliberately excluded. An archive holding both the audit chain and the key that signs it lets whoever has it forge a history that verifies perfectly — a test asserts no configured key appears anywhere in the archive. The test worth having destroys governance state, restores, and asks REMORA — not this product — whether the chain is intact. A chain that verifies after a round trip was not tampered with in between. ## What building it found The first test run HUNG. `pg_dump` takes an ACCESS SHARE lock on every table, and one session left idle-in-transaction blocks it indefinitely. The session was `SELECT count(*) FROM work_order_events`, open for fifteen minutes — psycopg3 holds a transaction for the whole `with connect()` block, and a killed test run had leaked one. Two real defects behind one symptom: - **Every read-only connection held a transaction open.** The postcondition reader, the console, `doctor`, the test fixture. In production that means a verification that stalls blocks `pg_dump`, which is exactly how a scheduled backup ends up hanging in silence. All are `autocommit=True` now: a reader has nothing to keep a transaction for. - **The backup would have hung forever.** It now sets `lock_timeout=15s`, `statement_timeout=300s` and a wall clock, and the failure message names the query to run to find what is holding the lock. Neither was reachable without actually taking a backup against a live system, which is the argument for having done it this way. 168 tests pass (71 contract, 97 end-to-end), 6/6 scenarios.
…leaves out `python run.py sbom` produces CycloneDX 1.5 per image, read from installed package metadata inside the container — so it lists what is actually there, not what a requirements file claims. Each image is identified by digest, because two builds of one Dockerfile are not the same artifact. CI publishes it as a build artifact on every run. 51 Python components across the two images, and the control plane's document names `remora 0.10.0` — the pinned wheel this entire product is built on. An inventory of everything except that would have been an odd inventory. ## Honesty is the load-bearing part An SBOM presented as complete when it is partial converts an unknown into a false assurance, which is worse than shipping none. So the document states its own limits and the tests assert it keeps stating them: - Debian packages in the base layer are NOT itemised. That needs a scanner this product does not ship. The base image is recorded by digest instead, so the OS layer is identified where it is not enumerated. - This is NOT build provenance. Provenance is an attestation by the builder that an output came from a given input; this is an observation of a built image, made afterwards, by whoever ran the command. Both are useful and they are not the same claim. `test_sbom.py::test_the_manifest_states_what_the_inventory_does_not_cover` fails if either caveat is ever dropped — the person reading the document in six months will not have this conversation. ## What building it found The first version pip-installed a generator inside the container and failed: the container runs with a read-only root filesystem, which is the hardening from the previous commit working exactly as intended. An inventory tool that has to modify the thing it inventories is the wrong tool, so it now reads `importlib.metadata` and writes nothing. The module docstring still claimed `cyclonedx-py` after that rewrite. Fixed — a doc describing behaviour the code does not have is the second-highest signal on this loop's own list, and it is not exempt when the author is me. 176 tests pass (71 contract, 105 end-to-end), 6/6 scenarios.
An external audit found seven P0/P1 items. Verified each in code before
acting; all seven held, and chasing one of them uncovered something worse.
## The one that mattered most
The fault test asserted `outcome != "executed"` — a value the field never
holds, so it could not fail. Correcting it to `"execute"` made it fail, and
that failure was the real finding:
outcome = "execute"
tool_execution = {"executed": False,
"refusal_reason": "tool_failed_nonce_burned",
"error": "work order WO-1202 is closed, not open"}
`outcome` describes the GOVERNED STEP — authorized, dispatched, grant burned.
`tool_execution.executed` describes whether the tool did anything. They
disagree exactly when it matters, and this product read the first and meant
the second in three places: the fault test, the VERIFY scenario, and the CLI's
exit code.
Consequence: the VERIFY scenario PASSED on every repeat run while the close
was failing. The postcondition read a work order closed by an earlier run and
reported EFFECT_VERIFIED for an action that never happened. A green demo for a
failed governed write is the worst failure mode this product could have.
src/aae/execution.py now owns the reading of both fields, and nothing else
reads them. `aae execute` exits non-zero unless the tool actually acted, and
prints the refusal reason and error rather than a bare outcome.
## `verify` could report more than it verified
Proven, not argued. With the stack down:
103 tests → "14 passed, 91 skipped", exit 0
The e2e fixtures skip when the stack is unreachable, and pytest exits 0 when
everything skips. `run.py verify` sets AAE_REQUIRE_LIVE=1, which turns those
skips into failures (91 errors, exit 1). Plain pytest still skips, so a
contributor with no stack gets a readable result.
Split `check` (contract, no Docker) from `verify` (live, must run). The two
remaining runtime skips became a visible parametrize exclusion — "2 skipped"
that nobody can explain is indistinguishable from something hidden.
## The demo was not repeatable
The scenarios genuinely mutate the system of record; that is the point. But a
second run met the leftovers. `run.py reseed` returns the reference work
orders to their seeded state, `scenarios` calls it first, and a `seeded`
fixture does the same for the tests that need it.
The audit chain is deliberately NOT reseeded: erasing the record of what was
decided to make a demonstration repeatable would defeat the thing being
demonstrated. `down` now stops without destroying; `reset` is the one that
removes volumes.
A CLI test also depended on execution order — it verified WO-1202, which is
only closed if a scenario ran first. Now uses WO-1150, which the seed closes.
## Claims brought back to what is enforced
- `network_policy: egress: none` declares what a tool REQUIRES. Nothing in
this profile blocks egress. Recorded in the ToolSpec source and in the new
security model under "declared, not enforced".
- "product code may only import remora.sdk" was not literally true. The
message now says what the test checks, and names the one quarantined
exception.
- The system-of-record event log cannot carry proposal_id or execution_id:
the dispatcher calls a tool as fn(arguments). The columns stay NULL and say
so, because an unfilled column is a visible gap where a removed one is a
hidden one. Correlation is by work order, tool, actor and time — useful, and
not the unambiguous join the docs implied.
- A missing row now reports EFFECT_MISMATCH, not EFFECT_UNOBSERVABLE. The
reader passed None in both cases — "we looked and it is absent" and "we
could not look" — collapsing the one distinction the whole status vocabulary
exists for. It passes {} now; the `looked` field is no longer dead.
## Also
- `.dockerignore`. The Dockerfiles never COPY `.env`, but a build context is
uploaded whole, and "not referenced" is not "never sent".
- `tool_specs.signed.json` is untracked. It is generated per installation with
that installation's key; committing it shipped a signature nobody else's key
can verify and made a routine install dirty the working tree.
## Documentation, restructured
The audit's central point was that the README argued instead of explaining,
and that the same argument appeared in five files. One home per kind of
information now:
README what it is, how to run it, how mature it is
architecture components, data flow, boundaries
security-model enforced vs declared, and trust assumptions
limitations known gaps, each with an owner
operations runbooks
tutorials/ try to get past the controls
adr/ why, including the costs each decision carries
ADR 0004 records the honest version of the control-plane boundary: the lock
calls `servers` internal and the Dockerfile's entrypoint names it, and the
import-boundary test scans Python files, not Dockerfiles. Stated rather than
implied.
174 tests pass (71 contract, 103 end-to-end), 6/6 scenarios, and both are now
stable across repeated runs.
Adopts a cleaner structure and a calmer voice: a decision table instead of prose, a mermaid flow instead of ASCII, capabilities as a list, and no argument for why each boundary is correct — that belongs in the ADRs. Four corrections to the draft, each one something a reader would have hit: - `docs/ARCHITECTURE.md`, `ONBOARDING.md` and `OPERATIONS.md` do not exist. The first two links resolve locally only because Windows is case-insensitive; on GitHub and in CI they 404. Now the real lowercase filenames, plus the two the draft omitted — `security-model.md` and `limitations.md`. - "Equivalent Make targets are also available" — the Makefile was deleted when `run.py` became the single implementation. `make up` fails. - `console/` was described as an operator console. It has no inbound authentication, no CSRF protection and a POST endpoint that mutates state. Calling it an operator console is the exact class of overclaim this repository is trying not to make. It is a local demonstration dashboard. - No maturity statement. A front page with none reads as production-ready, and this is a local reference vertical with no external security review. Restored as a short block near the top, with the non-claims paragraph at the bottom. Every relative link and every command in the file was checked case-sensitively against the repository before committing. 174 tests pass, 6/6 scenarios.
The console audit's two open P0s. Verified both against the running
container first; the other findings it raised — `down` meaning `down -v`,
missing capability drops, no read-only root, no resource ceilings, no SBOM —
are already closed on this branch, which the audit could not see because it
read `main`.
## It held every credential
Compose handed the dashboard `operator`, `reviewer`, `domain_expert`,
`senior_authority` and `viewer` — five tokens, four of which can approve or
execute. It needed them for one reason: an unauthenticated
`POST /api/scenarios` that shelled out to the CLI and ran the scenarios,
which perform real writes under those roles. No login, no CSRF protection,
no confirmation step, on a published port.
A presentation surface carrying an approver credential is a high-value
target wearing a low-value label.
## What changed
Running the scenarios is a CLI action now — `python run.py scenarios` —
which is where a privileged credential belongs. The endpoint is gone, not
guarded: 404 because the route does not exist, rather than 403 from a live
handler someone could re-enable with a config change. The page shows the
command instead.
With the scenario runner gone, the dashboard needs exactly one token. The
`viewer` role grants `read`, which turns out to cover every panel including
the audit-chain verification — checked, not assumed.
And it no longer needs REMORA at all. The image installed the pinned wheel
and the product package solely to reach the CLI; both went with it. A
read-only dashboard has no business shipping the governance engine it
displays.
tokens 5 → 1 (viewer)
write endpoints 1 → 0
remora in image yes → no
## Held in place
tests/e2e/test_console.py (15 tests) checks the running container, not the
compose file: one token and which one, no privileged credential and no
signing key present, the DSN connects as aae_reader, every mutating verb on
every path answers 404 or 405, the panels still report, and the image does
not contain remora.
The last one matters most. Everything else could be undone by an env change;
that one would need someone to put the dependency back.
189 tests pass (71 contract, 118 end-to-end), 6/6 scenarios.
PR #4 was squash-merged while this branch kept moving, so main holds the early work as one commit and this branch holds it as its originals plus everything since. Every file main has that this branch lacks is a deliberate removal here: Makefile run.py became the single implementation docs/ARCHITECTURE.md replaced by docs/architecture.md docs/ONBOARDING.md folded into the README and the attack tutorial docs/OPERATIONS.md replaced by docs/operations.md tool_specs.signed.json generated per install, now gitignored Checked with `git diff --diff-filter=A feat/gate-c-vertical origin/main` before merging: that list is exhaustive, so `-s ours` drops nothing unintentional. The doc renames are lowercase-only changes that a case-insensitive filesystem hides — verified against `git ls-tree`, not the working directory.
darklordVirtual
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything on this branch after PR #4 was squash-merged. Two external audits
drove most of it; each finding was verified in code before anything changed.
Security
The dashboard held every credential. Compose handed it
operator,reviewer,domain_expert,senior_authorityandviewer— four of whichcan approve or execute — because an unauthenticated
POST /api/scenariosranthe scenarios under those roles. No login, no CSRF protection, on a published
port. Running them is a CLI action now; the endpoint is removed, not
guarded (404 because the route does not exist), the dashboard holds one
read-only token, and the image no longer ships REMORA at all.
Nothing was hardened. A survey of the running containers found writable
root filesystems, the full default capability set, no memory or PID ceiling,
ports on every interface, and
/metricsserved without a token — allinherited from an upstream pilot whose own comments said "local pilot only".
All fixed and asserted against
docker inspect, not the compose file.Correctness
verifycould report more than it verified. With the stack down the e2esuite reported "14 passed, 91 skipped" and exited 0.
AAE_REQUIRE_LIVEturnsthose skips into failures;
checkis now the static variant.A test that could never fail hid a real defect. It asserted
outcome != "executed", a value the field never holds. Correcting it exposedthe finding:
outcomedescribes the governed step and reportsexecuteeven when the tool raised and the grant was burned.
tool_execution.executedis the field that answers whether anything happened. The product read the
first and meant the second in three places — including the VERIFY scenario,
which passed on every repeat run while the close was failing.
A missing row reported UNOBSERVABLE instead of MISMATCH, collapsing the
one distinction the whole status vocabulary exists for.
The demo was not repeatable.
reseedreturns the reference work orders totheir seeded state; the audit chain is deliberately never reset.
downnowpreserves data,
resetdestroys it.Claims brought back to what is enforced
network_policy: egress: nonedeclares what a tool requires; nothing hereblocks egress. The event log cannot carry
proposal_id— the dispatcher callsfn(arguments)— so the columns stay NULL and say why. "product code may onlyimport remora.sdk" was not literally true, and now names its one quarantined
exception.
Added
Backup and restore with a verified manifest and a test that destroys
governance state, restores, and asks REMORA whether the chain still verifies.
A CycloneDX SBOM per image, published by CI, stating what it does not cover.
aae lifecyclerenders a readable trail instead of a raw dict.Building the backup exposed a live hazard: every read-only connection held a
transaction open, so one stalled verification blocks
pg_dump— which is howa scheduled backup hangs in silence.
Documentation
Rebuilt around one home per kind of information: README, architecture,
security-model, limitations, operations, an attack tutorial, and four ADRs.
ADR 0004 records the honest version of the control-plane boundary — the lock
calls
serversinternal, the Dockerfile's entrypoint names it, and theimport-boundary test scans Python files, not Dockerfiles.
Verification
189 tests (71 contract, 118 end-to-end), 6/6 scenarios, stable across repeated
runs. CI brings the whole stack up with the same command the README gives a
new developer.
The merge commit uses
-s oursbecause main holds #4 as a squash: every filemain has that this branch lacks is a deliberate removal here, listed and
checked with
--diff-filter=Abefore merging.