adapter: notify SystemVars callbacks at the catalog commit boundary (… - #38046
Open
mtabebe wants to merge 1 commit into
Open
adapter: notify SystemVars callbacks at the catalog commit boundary (…#38046mtabebe wants to merge 1 commit into
mtabebe wants to merge 1 commit into
Conversation
…SQL-525) Problem: `SystemVars::register_callback` lets something mirror a system var's value into out-of-band state. Today the only user is the connection limit in `ConnectionCounter`. However, these callbacks fired straight from `SystemVars::set`/`reset`/`set_default`, which run inside `transact_inner` on the speculative cloned candidate state. So if a transaction applied a system-config op and then aborted, the mirror was left ahead of the catalog. `transact_incremental_dry_run`, which never commits anything notified for changes that get thrown away by design. Solution: Move the notification to the commit boundary. `SystemVars::notify_all_callbacks` fires from `Catalog::transact` right after the durable commit and state swap, and only when the transaction actually contained a system-config op. Since the callbacks are idempotent reads of the `SystemVars` they're handed, firing all of them on any system-config commit is cheap, and it saves us from threading a set of changed var names back out of `transact_inner`. With that in place, `set`/`reset`/`set_default` no longer notify at all. Callbacks now only run from `register_callback` (the once-at-registration guarantee) and from the commit-boundary `notify_all_callbacks`, so whatever they mirror can only ever reflect committed catalog state. Drop the dead `SystemVars::reset_all`. Testing: Two regression tests - a `transact_incremental_dry_run` carrying a `max_connections` op must not notify, even though the base-state clone does carry the callbacks; - a `transact` whose second op fails after the first already applied must not notify, and must leave the committed value alone. An mzcompose workflow against a real environmentd, `test-connection-limit-tracks-committed-vars` it holds one counted connection open and walks the enforced connection limit through every op that can change a system var (`ALTER SYSTEM SET`, `RESET`, `RESET ALL`) plus a rejected `SET`, requiring a new connection to be refused or allowed exactly as the committed value dictates. Neither sqllogictest nor testdrive can express this, since neither can assert that establishing a connection fails.
mtabebe
force-pushed
the
ma/sql-525-systemvars-commit-boundary
branch
from
August 4, 2026 15:20
4e84297 to
6b6be85
Compare
mtabebe
marked this pull request as ready for review
August 4, 2026 16:41
aljoscha
reviewed
Aug 5, 2026
| self.shared_transient_revision | ||
| .store(self.transient_revision, atomic::Ordering::SeqCst); | ||
| self.state = new_state; | ||
|
|
Contributor
There was a problem hiding this comment.
I don't know the details of this is wired right now, but could we instead move this to the catalog implications framework, so that this also works when another envd does the change and we listen to it? For the future multi-envd world.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…SQL-525)
Problem:
SystemVars::register_callbacklets something mirror a system var's value into out-of-band state. Today the only user is the connection limit inConnectionCounter. However, these callbacks fired straight fromSystemVars::set/reset/set_default, which run insidetransact_inneron the speculative cloned candidate state. So if a transaction applied a system-config op and then aborted, the mirror was left ahead of the catalog.transact_incremental_dry_run, which never commits anything notified for changes that get thrown away by design.Solution:
Move the notification to the commit boundary.
SystemVars::notify_all_callbacksfires fromCatalog::transactright after the durable commit and state swap, and only when the transaction actually contained a system-config op. Since the callbacks are idempotent reads of theSystemVarsthey're handed, firing all of them on any system-config commit is cheap, and it saves us from threading a set of changed var names back out oftransact_inner.With that in place,
set/reset/set_defaultno longer notify at all. Callbacks now only run fromregister_callback(the once-at-registration guarantee) and from the commit-boundarynotify_all_callbacks, so whatever they mirror can only ever reflect committed catalog state.Drop the dead
SystemVars::reset_all.Testing:
Two regression tests
transact_incremental_dry_runcarrying amax_connectionsop must not notify, even though the base-state clone does carry the callbacks;transactwhose second op fails after the first already applied must not notify, and must leave the committed value alone.Plus an mzcompose workflow against a real environmentd,
test-connection-limit-tracks-committed-varsintest/cluster: it holds one counted connection open and walks the enforced connection limit through every op that can change a system var (ALTER SYSTEM SET,RESET,RESET ALL) plus a rejectedSET, requiring a new connection to be refused or allowed exactly as the committed value dictates. Neither sqllogictest nor testdrive can express this, since neither can assert that establishing a connection fails.