Skip to content

chore: move to oneiriq-surql 0.34 and SurrealDB 3.3 - #114

Merged
albedosehen merged 2 commits into
mainfrom
chore/surql-0.34
Oct 1, 2026
Merged

albedosehen merged 2 commits into
mainfrom
chore/surql-0.34

Conversation

@albedosehen

@albedosehen albedosehen commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Moves Copal to oneiriq-surql 0.34 (lockfile: 0.34.1) and SurrealDB 3.3, with kayak at main 268ac8d, which moved to surql 0.34 in Oneiriq/kayak#54. Kayak and Copal have to name the same surql minor, or kayak's schema types and Copal's are different crates.

What changes

Floors

  • Workspace oneiriq-surql 0.34; MSRV 1.95 (surql's).
  • surrealdb 3.3 in copal-store, copal-flow, and copal-server.
  • The compose, conformance, and scale-bench engines move to surrealdb/surrealdb:v3.3.0.

Boot schema apply (copal-store/src/store.rs)

  • A changed index rebuilds in the background again. surql 0.34 reports a changed index as ModifyIndex, where it used to be an add. Boot only sent adds down the CONCURRENTLY path, so a new embedding width would have rebuilt the HNSW index synchronously and held boot for the whole rebuild. AddIndex | ModifyIndex now both go through index_add_statement. The routing moved into a pure plan_schema(), and a unit test diffs a 768-wide snapshot against a 384-wide one and asserts that the rebuild ends in CONCURRENTLY;.
  • "Still being reclaimed" is retried. After an index is removed or overwritten, SurrealDB 3.3 can refuse a DEFINE INDEX while it reclaims the table's document ids. run_ddl retries that refusal on a backoff of 0.5, 1, 2, 4, 8, and 15 s.
  • Schema statements are applied one at a time. Re-running the whole script would overwrite every index ahead of the refused one again, and that restarts the reclaim. With one statement per request, a retry repeats only the refused statement. It also lets surql 0.34.1's lone-statement retry on write conflicts apply to boot DDL.

Engine assumption (copal-store/tests/engine_assumptions.rs)

  • On 3.3, a schemafull option<array> field accepts object items and keeps their keys. The test that pinned the refusal now pins acceptance. file_version.markers and file_text.withheld stay TYPE any until a migration moves them.

What kayak main brought along (generators changed between 29f24b0 and 268ac8d)

  • MCP sub-collection tools. The manifest now lists file_versions_list and webhook_deliveries_list. mcp.rs gains Route::SubList, dispatched through Dispatcher::sub_list, and the route-parity test failed until it was added. The end-to-end agent test now lists the version its upload created.
  • Re-blessed artifacts.
    • clients/client.ts, and so the TypeScript SDK, names fields the way the wire does (content_type, created_at, …). The old camelCase names were never on a response. Nothing in sdks/ or console/ read them, and the assembled TypeScript SDK passes tsc --noEmit.
    • docs/openapi.json lists the values the state filter takes.

Audit

Local verification

The ci.yml rust job's gates, plus a build of copal-server alone. They ran on Windows with rustc 1.95.0 at 13bdab5, against the committed lock: kayak 268ac8d, surql 0.34.1, surrealdb 3.3.0.

Gate Result
cargo fmt --all --check pass
cargo clippy --workspace --all-targets -- -D warnings pass
cargo build -p copal-server (alone) pass
cargo test --workspace --no-fail-fast pass: 530 passed, 0 failed, 13 ignored, 78 binaries
cargo audit pass (with the ignores above)
TypeScript SDK tsc --noEmit pass
conformance/run.sh against v3.3.0 blocked by the MinIO image (below); live check instead

Caveat: on Windows the build compiles aws-lc-sys from source and wants NASM, so it ran with AWS_LC_SYS_PREBUILT_NASM=1. The Linux runners don't need that.

Live check against a real SurrealDB v3.3.0 server

conformance/run.sh built the server image: a Linux cargo build --release --locked -p copal-server against the committed lock, done in 10m17s. It then stopped at minio/minio:latest, which no longer pulls from Docker Hub ("pull access denied … repository does not exist"). This PR doesn't cause that, and it stops the conformance workflow on main too. Swapping the migration source is its own change.

Without MinIO, the conformance stack's surrealdb/surrealdb:v3.3.0 and the built image (engine sessions on, engine access key set) showed:

  • First boot. It applied 280 schema statements one at a time over ws://, with every non-unique index logged as index build backgrounded (CONCURRENTLY). It was ready in about 2 s, with no errors or warnings.
  • Restart. It applied 1 statement: the engine access definition, which is re-applied by design whenever a key is configured because the engine redacts keys in its echo. The schema diff was empty, so surql 0.34 reads 3.3's echo back as equal.
  • MCP. tools/list includes file_versions_list and webhook_deliveries_list. file_create, followed by file_versions_list on the new draft file, returned {"items":[],"next_cursor":null} with isError: false.

Workspace surql 0.34, MSRV 1.95 (surql's), surrealdb floors 3.3, and
the compose, conformance, and scale-bench engines at v3.3.0.

- Boot routes ModifyIndex, which surql 0.34 reports for a changed
  index, through the backgrounded CONCURRENTLY path with AddIndex.
  Routing only adds sent a new embedding width's HNSW rebuild down
  the synchronous path. The routing is a pure plan_schema(), tested.
- run_ddl retries SurrealDB 3.3's "still being reclaimed" refusal on a
  ~30 s backoff, and apply_schema runs statement by statement so a
  retry repeats only the refused statement instead of overwriting the
  indexes ahead of it again, which would restart the reclaim.
- The engine assumption that a schemafull option<array> refuses object
  items now pins 3.3's acceptance.
- The MCP face routes the sub-collection tools kayak's manifest now
  declares (file_versions_list, webhook_deliveries_list) through the
  dispatcher's sub_list.
- Re-blessed artifacts from kayak main: wire-named TypeScript client
  fields, the files state filter's enum in OpenAPI, the two new tools.
- .cargo/audit.toml carries RUSTSEC-2026-0194/0195 (quick-xml through
  object_store 0.13 in surrealdb-core 3.3.0); upstream
  surrealdb/surrealdb#7568.
Kayak main moved to surql 0.34 in Oneiriq/kayak#54; the lock now takes
it, surql 0.34.1, and surrealdb 3.3.0, so one surql version serves both.
@albedosehen
albedosehen marked this pull request as ready for review October 1, 2026 19:02
@albedosehen
albedosehen merged commit 3cff1b9 into main Oct 1, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant