Skip to content

fix(compat): route teardown flushes through the policy and wake refresh - #4373

Queued
pmoust wants to merge 1 commit into
mainfrom
land/4362-teardown-policy
Queued

pmoust wants to merge 1 commit into
mainfrom
land/4362-teardown-policy

Conversation

@pmoust

@pmoust pmoust commented Nov 17, 2026

Copy link
Copy Markdown
Member

The #4243 drill showed v0.33.0's ingest teardown publishing a data object
and commit record into a fenced bucket. In v0.34.0 every metrics, log and
span flush, the teardown drain's included, is spawned from one site per
signal that asks compat_grant before run_flush, and every PUT and fence
attempt asks check_attempt with the bucket's cached compat epoch. That
holds for the Shutdown drain, the channel-closed drain and the bypass
passes that write a flush the scan-set or floor check (NoView) could not
confirm. No path bypassed the policy, so no routing changed. Each signal
gains two tests: after an activation, with the bucket fenced, a teardown
by shutdown() and by a drop under a view past the horizon waits out its
max_flush_lifetime, is refused view_stale and counted, and leaves every
key under the tenant byte for byte as it was; on a store that never ran
compat init, with the view read absent or blind, the teardown publishes
at the baseline with nothing refused and no sys/compat object.

The #4355 round-2 review items:

  • Admission refused every ingest write while the view was stale, and
    only the 60 s refresh tick cleared it. Every view_stale answer of the
    ingest policy now notifies a refresh wake, and so does a first touch
    refused for a stale view. ravel-server's refresh loop refreshes on the
    wake at once, then at most once per second, with the wakes that arrive
    in the gap served by one read. The 5xx-at-startup e2e test now holds
    the woken read to keep the view unread for both refused requests, and
    asserts that releasing it makes the view fresh with no refresh by the
    test.
  • The grant re-checked max_publish_lifetime_ns and dropped rows already
    acknowledged. ADR-4233 decision 7 never lowers that bound, so it falls
    below a live router's lifetime only when compat init records it while
    the router runs. Admission still refuses, and admitted rows publish
    inside the router's own lifetime. The docs name that trigger.
  • A flush waiting out a stale view now asks it once more at its deadline
    instead of giving up to one second early.
  • A first touch no longer creates a provisioning record while sys/compat
    bounds publishes below the ingest lifetime: it refuses with a retryable
    PublishLifetimeAboveCompat, counted under publish_lifetime_above_compat,
    ahead of the admission refusal that would follow.
  • The stale-view admission tests in tests/compat_policy.rs are renamed and
    assert the refusal comes at admission: a buffered write gets the error
    instead of an acknowledgement, with nothing buffered and no flush opened.
  • docs/consistency-model.md names both ways view_stale can drop rows on a
    never-activated store (a read view past the horizon, and a blind view
    after one non-AccessDenied failure) and the shutdown timeout bound.

Gates run on the result: fmt, clippy for ravel-ingest and ravel-server, nextest ravel-ingest (776) and ravel-server lib plus the compat, audit, provisioning and metrics integration binaries (1497), check_docs. CI runs the feature lanes.

Fixes: #4362
Refs: #4233

The #4243 drill showed v0.33.0's ingest teardown publishing a data object
and commit record into a fenced bucket. In v0.34.0 every metrics, log and
span flush, the teardown drain's included, is spawned from one site per
signal that asks compat_grant before run_flush, and every PUT and fence
attempt asks check_attempt with the bucket's cached compat epoch. That
holds for the Shutdown drain, the channel-closed drain and the bypass
passes that write a flush the scan-set or floor check (NoView) could not
confirm. No path bypassed the policy, so no routing changed. Each signal
gains two tests: after an activation, with the bucket fenced, a teardown
by shutdown() and by a drop under a view past the horizon waits out its
max_flush_lifetime, is refused view_stale and counted, and leaves every
key under the tenant byte for byte as it was; on a store that never ran
compat init, with the view read absent or blind, the teardown publishes
at the baseline with nothing refused and no sys/compat object.

The #4355 round-2 review items:

- Admission refused every ingest write while the view was stale, and
  only the 60 s refresh tick cleared it. Every view_stale answer of the
  ingest policy now notifies a refresh wake, and so does a first touch
  refused for a stale view. ravel-server's refresh loop refreshes on the
  wake at once, then at most once per second, with the wakes that arrive
  in the gap served by one read. The 5xx-at-startup e2e test now holds
  the woken read to keep the view unread for both refused requests, and
  asserts that releasing it makes the view fresh with no refresh by the
  test.
- The grant re-checked max_publish_lifetime_ns and dropped rows already
  acknowledged. ADR-4233 decision 7 never lowers that bound, so it falls
  below a live router's lifetime only when compat init records it while
  the router runs. Admission still refuses, and admitted rows publish
  inside the router's own lifetime. The docs name that trigger.
- A flush waiting out a stale view now asks it once more at its deadline
  instead of giving up to one second early.
- A first touch no longer creates a provisioning record while sys/compat
  bounds publishes below the ingest lifetime: it refuses with a retryable
  PublishLifetimeAboveCompat, counted under publish_lifetime_above_compat,
  ahead of the admission refusal that would follow.
- The stale-view admission tests in tests/compat_policy.rs are renamed and
  assert the refusal comes at admission: a buffered write gets the error
  instead of an acknowledgement, with nothing buffered and no flush opened.
- docs/consistency-model.md names both ways view_stale can drop rows on a
  never-activated store (a read view past the horizon, and a blind view
  after one non-AccessDenied failure) and the shutdown timeout bound.

Refs: #4362
Refs: #4233
Signed-off-by: Panagiotis Moustafellos <pmoust@nofire.ai>
@pmoust
pmoust added this pull request to the merge queue Nov 17, 2026
Any commits made after this event will not be merged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

T22 (epic #4233): teardown flush obeys the write policy on every signal

1 participant