| type | Concept | ||
|---|---|---|---|
| title | Database evidence and state-store boundary | ||
| okf_status | active | ||
| tags |
|
||
| freshness | 2026-09-29 | ||
| resource | https://github.com/alphazede/bran | ||
| public_boundary | public |
This is the design for issue #27. BRAN touches databases for two unrelated reasons, and they get two unrelated ports:
- Evidence source. Read-only access to a database catalog and, when policy allows it, bounded query results that enter BRAN provenance and packets.
- State store. Optional durable storage for BRAN-owned derived state: objects, indexes, caches, job metadata, and receipts.
The code is crates/bran-core/src/database/. It is standard-library only, like
the rest of bran-core. Default tests make no database-server or network
calls.
Where the issue offered alternatives, the owner chose the smallest safe option the issue prefers:
| Question | Decision | Reason |
|---|---|---|
| One connector or two ports? | Two ports: EvidenceEngine and StateStore |
Different direction (read vs write), credentials, retention, and threat model. |
| Evidence engine order | SQLite first, PostgreSQL second, others deferred | SQLite is a bounded local file with no network. PostgreSQL is the first remote engine. MySQL/MariaDB, SQL Server, Oracle, DuckDB, and warehouse APIs wait until the contract is proven. |
| Default evidence access | Introspection only | Row access needs an explicit access: bounded-rows plus relation allowlist. |
| Free SQL | Never passthrough | SQL text must pass one deterministic gate and match an allowlisted digest. Projections are generated by BRAN and pass the same gate. |
| State-store backends | Local file store first | It needs no dependency. SQLite and PostgreSQL state backends are declared but report unavailable (see Not done). |
Both descriptors are small YAML documents parsed by BRAN's existing
frontmatter parser. Each parser accepts exactly one kind, an exact key set,
and one credential scope:
| Evidence source | State store | |
|---|---|---|
kind |
evidence-source |
state-store |
| Rust type | EvidenceSourceDescriptor |
StateStoreDescriptor |
| Port | EvidenceEngine (read catalog, stream rows) |
StateStore (put, get, usage) |
| Keys | engine, database, access, relations, queries, snapshot, max_rows, max_bytes, timeout_ms |
backend, location, namespace, quota_bytes, encryption_key |
| Shared keys | kind, id, credential |
kind, id, credential |
| Credential scope | credref:evidence/<name> |
credref:state/<name> |
| Writes | never | only inside its own namespace directory |
| Retention | none; evidence lives in the packet that cites it | quota-bounded, operator-managed |
So:
- An evidence descriptor given to the state parser fails with
WrongKind, and the reverse also fails. - A key that belongs to the other port fails with
UnknownKey. - A state credential in an evidence descriptor fails with
CredentialScope, and the reverse also fails. One credential reference can never authorize both ports. - The Rust types have no conversion between them, and the two traits share no
method or supertrait.
FileStateStore::opentakes only aStateStoreDescriptor.extracttakes only anEvidenceSourceDescriptor.
Test: database::tests::p8_database_boundary_config.
kind: evidence-source
id: synthetic-orders
engine: sqlite
database: data/orders.sqlite
access: bounded-rows
relations: [main.customers, main.orders]
queries: [sha256:<64 lowercase hex>]
snapshot: sha256:<64 lowercase hex>
max_rows: 100
max_bytes: 65536
timeout_ms: 5000engine:sqliteorpostgresql.database: forsqlite, a relative POSIX path of[A-Za-z0-9._-]segments with no.or..segment, at most 256 bytes. Forpostgresql, an opaque host-registered handle name. Connection strings, URLs, hosts, ports, and query parameters are rejected, so they cannot carry credentials.credential: forbidden forsqlite(a local file needs none), required forpostgresql.access:introspection(the default when absent) orbounded-rows.relations: required, non-empty, sorted, and unique. Each entry isschema.relationin lowercase[a-z_][a-z0-9_]*parts. It bounds both introspection and row access.queries: optional digests of allowlisted SQL (see the SQL gate below).snapshot: optional pinned snapshot identity, as asha256:digest. A different or unattested observed snapshot isstale-snapshot.- Limits:
max_rows1..=10000 (default 100),max_bytes1..=4194304 (default 65536),timeout_ms1..=60000 (default 5000).
kind: state-store
id: local-state
backend: file
location: state
namespace: default
quota_bytes: 1048576backend:file,sqlite, orpostgresql. Onlyfileis available in this build.location: a safe relative path under the host-provided state root, with the same rules as a SQLitedatabasepath. It cannot name an absolute path or climb out of the state root.namespace: a name. A store handle is bound to one namespace when it opens. No API reads across namespaces.quota_bytes: 1..=1073741824.credential: forbidden forfile, required forpostgresql.encryption_key: an optionalcredref:state/<name>. The file backend cannot encrypt, so a requested key makesopenfail withUnavailable. BRAN never stores plaintext after encryption was requested.
The only accepted credential form is credref:<scope>/<name>, with scope
evidence or state and name matching [a-z0-9][a-z0-9._-]{0,63}. The host
resolves the reference. BRAN never reads, stores, or prints credential
material.
Rejected, with a content-free error: inline passwords, user:password@host
URLs, env: or file references, a missing or foreign scope, and a name with
/ or other characters. Descriptor errors carry at most a static field name.
They never echo the value, so a secret pasted into the wrong field cannot be
reflected into logs, receipts, packets, or error messages.
Test: database::tests::p8_database_boundary_config.
database::sql::gate is the only way to build a GatedQuery. The engine port
accepts only a GatedQuery, so model-generated SQL, operator SQL, and
BRAN-generated projections all pass through the same deterministic gate. The
gate accepts a deliberately small grammar and rejects everything else:
- exactly one
SELECT, first: no;, noUNION/INTERSECT/EXCEPT, noWITH, no subqueries - no backslash anywhere, so escape-string dialects cannot tokenize differently from the gate
- no comments (
--,/*),$$quoting,@,:,&, or non-ASCII outside string literals - no write or DDL keyword (
INSERT,UPDATE,DELETE,MERGE,REPLACE,CREATE,DROP,ALTER,TRUNCATE,GRANT,REVOKE,COPY,INTO,RETURNING, and similar) - no session or transaction keyword (
SET,RESET,PRAGMA,BEGIN,COMMIT,ROLLBACK,SAVEPOINT,LOCK,FOR, and similar) - no procedure execution (
CALL,EXEC,EXECUTE,DO) - no external reference (
ATTACH,DETACH,LOAD,IMPORT,OPENROWSET,OPENQUERY,OPENDATASOURCE,dblink,table@link, relation names with more than two parts, or column names with more than three) - function calls only from
abs,avg,coalesce,count,length,lower,max,min,round,sum, andupper, named directly. Everything else, such asload_extension,readfile,pg_read_file,pg_sleep, orrandomblob, isUnsafeFunction. So is a schema-qualified call such asother.count(x), which could resolve to a user-defined function, and a call through a grammar word, literal, or quoted name, such asfirst(x),like(x, y),'randomblob'(4), or"abs"(x). Any other(must follow an operator,,,(, or one ofSELECT,DISTINCT,WHERE,AND,OR,NOT,IN,ON,HAVING,BY,BETWEEN,LIMIT, orOFFSET, where it can only group an expression or list. Sox LIKE ('a')must be writtenx LIKE 'a'. INmust be followed by(.x IN schema.relationwould read a relation outside theFROMlist, so it isMalformed.- every relation in the
FROMlist is fully qualifiedschema.relationand present in the descriptor allowlist. Unqualified names are rejected, because a server search path could resolve them to another schema. At parenthesis depth zero,FROM,,, andJOINeach introduce a relation untilWHERE,GROUP,ORDER,HAVING,LIMIT, orOFFSETends the list, including a,after a join'sONexpression. A relation may be followed only by an alias,,, a join keyword,ON, or one of those clause words. Modifiers such asNOT INDEXED,INDEXED BY, orUSINGareMalformed, so they cannot hide a later relation from the allowlist check. - row queries need
ORDER BY, so truncation and result digests are deterministic - at most 8192 bytes and 1024 tokens
The first forbidden keyword in token order decides the rejection class. A
column whose name is also a listed keyword (for example comment or start)
must be double-quoted.
Parameters (? or $1) are accepted for prepared execution. The gate
normalizes the token stream (single spaces; unquoted words lowercased; literals
and quoted identifiers verbatim) and records sha256 of that text as the query
digest. Whitespace and keyword case do not change the digest.
Free SQL must also match one of the descriptor's queries digests. A relation
projection (relation plus column list) needs no digest. BRAN generates
SELECT "c1", "c2" FROM "schema"."relation" ORDER BY 1, 2 and gates it like
any other SQL.
SqlRejection is a content-free enum. It never contains the input.
Adapters must also enforce read-only access in the engine, as defense in depth:
| Engine | Required engine-level guard |
|---|---|
| SQLite | open mode=ro; PRAGMA query_only=ON; an authorizer that permits only SELECT, READ on allowlisted relations, and allowlisted functions; extension loading off; a progress handler for timeout and cancellation |
| PostgreSQL (future) | SET TRANSACTION READ ONLY in a dedicated transaction; statement_timeout; a role without write, dblink, postgres_fdw, or large-object grants; server-side cancel |
tools/ci/database_contract_check.py proves the SQLite column against a real
SQLite engine (Python's standard sqlite3), using the synthetic fixture
database.
Tests: database::sql::tests::p8_database_sql_gate, plus
tools/ci/database_contract_check.py.
database::evidence::extract checks policy before it contacts the engine. A
refused request (rows under introspection, a gate rejection, or an SQL
digest that is not allowlisted) returns ExtractionRefusal. The engine is
never called. An accepted request always yields exactly one outcome.
The effective deadline is the earlier of the caller's deadline and the
descriptor's timeout_ms. The adapter receives it with the cancellation flag
(Interrupt) on every catalog and row call and must stop in-flight engine
work when either fires. BRAN also rechecks both after every engine call: the
opening snapshot, the catalog, the row stream (even an empty one), and the
closing snapshot. After an interruption it makes no further engine call, and a
blocking call that overran or was cancelled never yields admissible evidence.
| Outcome | When | Rows kept | Packet admission |
|---|---|---|---|
schema-only |
no row request | none | admitted |
bounded-rows |
every row fit max_rows and max_bytes |
all | admitted |
truncated |
the engine had more rows or bytes than the limits | the bounded prefix | admitted, with truncated: true |
cancelled |
the cancellation flag was set at any point, or the engine reported cancel | none | refused |
timeout |
the effective deadline passed at any point, or the engine reported timeout | none | refused |
unavailable |
no adapter for this engine, or the adapter reported unavailable | none | refused |
stale-snapshot |
the observed snapshot differs from the pinned one, or changed during extraction | none | refused |
dlp-rejected |
the DLP check reported findings | none | refused |
Refused outcomes (cancelled, timed out, unavailable, stale, and DLP-rejected)
keep no partial rows and no catalog. Their record has "columns":[], so a
column name that DLP flagged, or that DLP never checked, cannot reach the
sealed record. They produce a receipt and nothing a packet can cite. An engine for which no
adapter is compiled in (PostgreSQL today) is reported as unavailable. BRAN
never simulates it.
Test: database::evidence::tests::p8_database_evidence_packet.
Every extraction produces one canonical JSON record (sorted keys, no whitespace) with:
adapterandenginesource_id,database(the validated SQLite path or PostgreSQL handle, which cannot contain credentials), andcredential(the opaque reference, ornull). Changing the configured database changes the record, the evidence digest, and the packet node id.access,relations, andcolumns(schema.relation.columnlocators; empty for refused outcomes)derivation:schema-only,projection, orqueryquery_digest: the normalized gate digest, ornullsnapshot:{"state":"attested","value":…}or{"state":"unavailable","value":null}. Engine snapshot text is untrusted, so anything other than asha256:digest is sealed as its digest.row_countandtruncatedextracted_at, supplied by the caller so replays are deterministiclimits:max_rows,max_bytes, andtimeout_mssource_digest(the canonical catalog) andresult_digest(the canonical rows)outcome,dlp(not-evaluated,passed, orfindings), andclassificationandredaction. Both areunavailablein this version. BRAN does not claim a classification or redaction it did not perform.
evidence_digest is sha256 of that record. The same source, request, and
snapshot always give the same record and digest. The record never contains
credential material, connection strings, or rows.
DatabaseEvidence::packet_evidence turns an admitted extraction into the
existing packet inputs:
- a graph
NodeInputwith roleExternal, iddb:<source_id>:<first 32 hex of evidence_digest>, provenance sourcedatabase-evidence, locatordatabase:<source_id>#<evidence_digest>, and facts for outcome, truncation, row count, and result digest - an
EvidenceContentwhose content is the provenance record plus one canonical JSON array per row, with preservation anchors fordb.outcome,db.truncated,db.row_count, anddb.result_digest
PacketAssembler then carries database evidence like any other node. Its
provenance and truncation state stay in the packet items and payload.
Non-admitted outcomes return NotAdmitted and never become packet items.
Today bran packet and bran query still read repository files only. No CLI
command reads database descriptors yet (see Not done).
Test: database::evidence::tests::p8_database_evidence_packet.
StateStore is the port. FileStateStore is the first implementation. The
existing process-local MemoryResultStore is unchanged and stays the default.
A durable store is opt-in through a state-store descriptor.
Layout under <state root>/<location>/<namespace>/:
VERSION: the layout version,1objects/<sha256 hex>: content-addressed bytestmp/: staging for writes in progress
Behaviour:
- Atomic writes.
putwrites the bytes totmp/, syncs them, then renames intoobjects/. A crash leaves a temp file, never a partial object.opendeletes leftover temp files and reports how many it removed asrecovered_partial_writes. - Corruption.
getandverifyrecompute SHA-256. A modified or truncated object isCorrupt. It is never returned as data. - Migration. A missing
VERSIONin an empty namespace is initialised to1. Any other version isMigrationFailed, and the store refuses to open without writing anything. A malformedVERSIONisCorrupt. Version 1 is the first layout, so there is no migration yet. A future migration must keep this refuse-without-writing rule. - Quota. A
putthat would exceedquota_bytesisQuotaExceeded. It leaves no temp file and does not change usage. Repairing a damaged object recounts usage from disk, subtracts the damaged length, adds the true length, and enforces the quota on that total. - Confinement. Every directory from the state root down to the namespace,
objects/,tmp/,VERSION, and every object and staged entry is inspected without following symlinks before it is read, written, or removed. A symlink or unexpected entry isUnconfined, andopenrefuses before it creates, writes, or deletes anything. The host-supplied state root itself is trusted. - Isolation. A handle can read and write only its own namespace.
- Canonical data. The store writes only under its own namespace directory. It never reads or writes repository content. The state store is never a source of truth for canonical repository documents; everything in it can be rebuilt or discarded.
Errors are content-free: no paths, bytes, or credentials.
Limits: a namespace allows one writer at a time; two open handles each track
their own usage and can together exceed the quota. The confinement checks run
before each use, because portable std has no openat or O_NOFOLLOW, so a
process that can already write inside the state root could race them.
Concurrent server workloads are what a PostgreSQL backend would be for.
Test: database::state::tests::p8_state_store_failures.
fixtures/database/synthetic.sql builds a small synthetic SQLite database
(customers, orders, and a view). All names start with Synthetic, and all
email addresses use the reserved .invalid domain. No customer data is
included. Adversarial data is included too: injection text stored as a value,
non-ASCII text, a long value for byte truncation, and a synthetic key-shaped
note for DLP.
fixtures/database/queries.tsv is the shared SQL corpus, one
class<TAB>effect<TAB>rejection<TAB>sql row per statement. rejection is the
exact SqlRejection the Rust gate must return (- for allowed queries). It holds allowed queries plus
adversarial statements by class: write, multi-statement, session,
procedure, unsafe-function, external-link, compound,
secret-reflection, and policy. The effect column says what the engine
does without the gate:
mutates: the statement changes an unguarded copy of the database. These negative tests are real attacks, not strawmen.gate-only: the statement is a harmless read for SQLite, so the engine guard runs it. Only the BRAN gate refuses it, to keep the grammar small and results deterministic. Examples are compound queries, a trailing;, a comment, and unqualified or unordered queries.-: the engine guard refuses it too (or cannot parse it).
tools/ci/database_contract_check.py builds the database twice and requires
identical dumps. It checks that every row is synthetic. It runs every allowed
and gate-only query under the SQLite guard, and requires every other
adversarial statement to be refused with the database unchanged. It also shows
that a progress-handler interrupt stops a long query, which is how a SQLite
adapter implements timeout and cancellation. The Rust gate test reads the same
corpus and requires the gate to return the exact listed rejection for all 47
adversarial rows.
No part of this design opens a listener or a network connection. The default
test suite needs no database server. SQLite evidence is a local file.
PostgreSQL stays unavailable until an adapter is explicitly compiled and
selected.
| Acceptance item | Evidence |
|---|---|
| Separate EvidenceSource and StateStore; not interchangeable | This document; database::tests::p8_database_boundary_config |
| Only validated references to host-managed credentials | database::tests::p8_database_boundary_config |
| Write SQL, multi-statements, unsafe functions, external links, and secret reflection rejected | database::sql::tests::p8_database_sql_gate; tools/ci/database_contract_check.py |
| Eight outcomes typed separately | database::evidence::tests::p8_database_evidence_packet |
| Evidence enters packets with stable provenance and truncation/derivation state | Partial. Library level: database::evidence::tests::p8_database_evidence_packet and provenance_binds_database_origin. No real database evidence reaches bran packet yet (see Not done). |
| Corruption, migration failure, quota exhaustion, and partial writes fail visibly without losing canonical data | database::state::tests::p8_state_store_failures, repair_accounts_quota, and state_paths_refuse_symlink_escape |
| Deterministic synthetic fixture database | fixtures/database/; tools/ci/database_contract_check.py |
| No cloud database or network for the default suite | All of the above run offline in ./tools/ci/check.sh --fast |
- Rust SQLite and PostgreSQL adapters (deferred by owner decision). A
Rust SQLite binding needs an engine dependency, and the owner's delegate
deferred adding one (2026-09-30). The trade-offs recorded for that decision:
rusqlite0.31 bundles SQLite 3.45 (older than current security fixes) and needs a C compiler, which conflicts with CONTRIBUTING's "stable Rust toolchain, nothing else";rusqlite0.33 and later pullfoldhash(Zlib), whichdeny.tomldoes not allow. Until an adapter lands, the port and gate are proven here and the engine guard is proven through Python's standardsqlite3. PostgreSQL needs a server and a network adapter, and is not attempted. So the packet-entry acceptance item is partial: packet entry is proven throughPacketAssemblerwith a synthetic engine, but no real database evidence can reach a packet yet. - SQLite and PostgreSQL state backends. Same dependency decision. The
descriptors accept them, and
openreportsunavailable. - CLI wiring.
bran packetdoes not yet load descriptors or run extraction. Packet entry is proven at the library level. - State-store retention and garbage collection, backup and restore tooling,
encryption at rest, and deterministic state export. The file layout is
plain content-addressed files. Restore means copying the files back, then
running
verify. The rest is follow-up work. - Classification and redaction receipts. These are recorded as
unavailableuntil a classifier or redactor exists.