Skip to content

refactor(membership): retire row reads in favor of snapshots - #11389

Draft
ReubenBond wants to merge 5 commits into
dotnet:mainfrom
ReubenBond:rb-refactor-retire-membership-row-reads
Draft

ReubenBond wants to merge 5 commits into
dotnet:mainfrom
ReubenBond:rb-refactor-retire-membership-row-reads

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

Problem

The built-in membership manager reads coherent full-table snapshots. A separate point-row read contract adds provider behavior and test complexity without a runtime consumer.

Solution

Retain and deprecate IMembershipTable.ReadRow and ReadRowAsync, preserving their signatures and wire identity. First-party implementations report NotSupportedException after observing pre-cancellation. The default interface adapter remains available to custom legacy providers.

Migrate first-party tests and conformance observations to ReadAllAsync with MembershipTableData.TryGet, selecting the row, ETag and table version from the same snapshot. Keep the 959 generated histories, 18 internal operation identities, and exact hosted partition multiplicities; present/absent observations now use snapshots. Public G15/G22 scenario names remain obsolete aliases while normal discovery avoids duplicate full-read scenarios.

Scope and setup cost

Per review feedback, the proposed receipt-returning mutation APIs, DTOs, provider plumbing and receipt-only tests have been removed. Existing bool-returning mutation APIs, native write behavior, and RPC identities remain unchanged. No artificial runtime consumers or replacement test hooks are added.

Large conformance setup uses existing bool inserts and full snapshots to obtain the current ETag. For N rows it performs N inserts and N+3 full reads, returning N(N+1)/2 + 2N membership rows. This explicitly accepted test-only cost is 8,398,848 returned rows at N=4096. Expected canonical entries remain independently constructed, with per-insert and final verification.

The landed provider read fences, ZooKeeper single-attempt mutation/session ownership, Firestore transactional full reads, cancellation behavior and fixture ownership remain intact. Migration guidance and regenerated API surfaces accompany the behavioral change. No database/catalog or package-validation policy changes are introduced.

Copilot AI balanced review requested due to automatic review settings October 2, 2026 18:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes public contracts and atomic-write behavior across many external storage providers requiring final human validation.

Review effort: Balanced
Findings: None

What changed in this PR

Adds receipt-returning membership writes while retiring built-in point-row reads in favor of coherent full snapshots.

Changes:

  • Introduces immutable write-result and commit-receipt contracts.
  • Implements native receipts across membership providers and preserves legacy adapters.
  • Updates conformance coverage, provider tests, documentation, and generated APIs.
File Description
test/​TestInfrastructure/​Orleans.TestingHost.Tests/​InProcessMembershipTableCancellationTests.cs Covers cancellation for richer writes.
test/​Orleans.Runtime.Internal.Tests/​MembershipTests/​MembershipTableTestsBaseLifecycleTests.cs Verifies snapshot-only legacy scenarios.
test/​Orleans.Runtime.Internal.Tests/​MembershipTests/​MembershipTableTestsBase.cs Adds shared receipt provenance tests.
test/​Orleans.Runtime.Internal.Tests/​MembershipTests/​MembershipTableSystemTargetTests.cs Tests retired reads and system-target receipts.
test/​Orleans.Runtime.Internal.Tests/​MembershipTests/​MembershipTableConformanceTestsBase.cs Replaces point-read conformance cases.
test/​Orleans.Core.Tests/​Membership/​MembershipTableWriteResultTests.cs Tests result and receipt contracts.
test/​Orleans.Core.Tests/​Membership/​MembershipTableWriteResultSerializationTests.cs Tests receipt serialization.
test/​Orleans.Core.Tests/​Membership/​MembershipTableWriteResultDefaultTests.cs Tests default provider adapters.
test/​Orleans.Core.Tests/​Membership/​MembershipTableCancellationTests.cs Verifies wire identities and cancellation.
test/​Orleans.Core.Tests/​Membership/​InMemoryMembershipTable.cs Retires test point reads.
test/​Orleans.Clustering.TestKit.Tests/​MembershipTableConformanceTests.cs Updates conformance scenarios and counts.
test/​Orleans.Clustering.TestKit.Tests/​MembershipTableCleanupLifetimeTests.cs Retires fixture point reads.
test/​Orleans.Clustering.TestKit.Tests/​IdealizedMembershipTable.cs Adds idealized write receipts.
test/​Orleans.Clustering.TestKit.Tests/​FaultyMembershipTableTests.cs Updates fault-injection coverage.
test/​Orleans.Clustering.TestKit.Tests/​FaultyMembershipTable.cs Adapts faults to snapshots and receipts.
test/​Extensions/​Orleans.Redis.Tests/​Clustering/​RedisMembershipTableTests.cs Tests Redis receipt chaining.
test/​Extensions/​Orleans.Cosmos.Tests/​CosmosMembershipTestStorage.cs Adds receipt-aware Cosmos fakes.
test/​Extensions/​Orleans.Cosmos.Tests/​CosmosMembershipTableTests.cs Tests Cosmos receipt provenance.
test/​Extensions/​Orleans.Clustering.ZooKeeper.Tests/​ZooKeeperReadRetryTests.cs Tests retired reads and native receipts.
test/​Extensions/​Orleans.Clustering.ZooKeeper.Tests/​ZooKeeperReadResilienceTests.cs Uses full snapshots for resilience tests.
test/​Extensions/​Orleans.Clustering.ZooKeeper.Tests/​ZooKeeperNativeFake.cs Returns native transaction results.
test/​Extensions/​Orleans.Clustering.ZooKeeper.Tests/​ZookeeperMembershipTableTests.cs Tests ZooKeeper receipt chaining.
test/​Extensions/​Orleans.Clustering.ZooKeeper.Tests/​ZooKeeperBasedMembershipTableUnitTests.cs Expands native receipt coverage.
test/​Extensions/​Orleans.Clustering.Firestore.Tests/​FirestoreMembershipTableTests.cs Adds Firestore receipt tests.
test/​Extensions/​Orleans.Clustering.Firestore.Tests/​FirestoreClusteringTests.cs Migrates tests to snapshots.
test/​Extensions/​Orleans.Clustering.Consul.Tests/​ConsulMembershipTableTest.cs Tests Consul receipt chaining.
test/​Extensions/​Orleans.Clustering.Cassandra.Tests/​Clustering/​CassandraMembershipTableInitializationTests.cs Tests pre-initialization behavior.
test/​Extensions/​Orleans.Clustering.Cassandra.Tests/​Clustering/​CassandraMembershipStatementTests.cs Tests Cassandra receipts and snapshots.
test/​Extensions/​Orleans.Clustering.Cassandra.Tests/​Clustering/​CassandraClusteringTableTests.cs Migrates provider integration tests.
test/​Extensions/​Orleans.Azure.Tests/​AzureMembershipTableTests.cs Adds Azure receipt provenance tests.
test/​Extensions/​Orleans.AWS.Tests/​MembershipTests/​DynamoDBMembershipTableTest.cs Adds DynamoDB receipt tests.
test/​Extensions/​Orleans.AdoNet.Tests/​StorageTests/​Relational/​RelationalOrleansQueriesUnitTests.cs Tests relational receipt behavior.
test/​Extensions/​Orleans.AdoNet.Tests/​StorageTests/​Relational/​Fakes/​ScriptedRelationalStorage.cs Supports delayed read completion tests.
test/​Extensions/​Orleans.AdoNet.Tests/​StorageTests/​Relational/​AdoNetMembershipUpgradeTests.cs Validates receipts with existing schemas.
src/​Redis/​Orleans.Clustering.Redis/​Storage/​RedisMembershipTable.cs Returns Redis commit receipts.
src/​Orleans.TestingHost/​InProcess/​InProcessMembershipTable.cs Adds in-process receipts.
src/​Orleans.Runtime/​MembershipService/​SystemTargetBasedMembershipTable.cs Exposes system-target receipts.
src/​Orleans.Runtime/​MembershipService/​InMemoryMembershipTable.cs Produces in-memory receipts.
src/​Orleans.Core/​SystemTargetInterfaces/​MembershipTableWriteResult.cs Defines result and receipt contracts.
src/​Orleans.Core/​SystemTargetInterfaces/​IMembershipTable.cs Adds richer write APIs and retires reads.
src/​Orleans.Clustering.ZooKeeper/​ZooKeeperBasedMembershipTable.cs Derives receipts from ZooKeeper results.
src/​Orleans.Clustering.TestKit/​README.md Documents revised conformance behavior.
src/​Orleans.Clustering.TestKit/​MembershipTableTestRunner.cs Uses receipts and full snapshots.
src/​Orleans.Clustering.TestKit/​MembershipTableModel.cs Maps row observations to snapshots.
src/​Orleans.Clustering.Consul/​ConsulBasedMembershipTable.cs Derives Consul transaction receipts.
src/​Google/​Orleans.Clustering.Firestore/​FirestoreMembershipTable.cs Returns Firestore write timestamps.
src/​Cassandra/​Orleans.Clustering.Cassandra/​CassandraClusteringTable.cs Returns Cassandra version receipts.
src/​Azure/​Orleans.Clustering.Cosmos/​Membership/​CosmosMembershipTable.cs Returns Cosmos batch receipts.
src/​Azure/​Orleans.Clustering.AzureStorage/​OrleansSiloInstanceManager.cs Returns Azure transaction ETags.
src/​Azure/​Orleans.Clustering.AzureStorage/​AzureBasedMembershipTable.cs Exposes Azure commit receipts.
src/​AWS/​Orleans.Clustering.DynamoDB/​Membership/​DynamoDBMembershipTable.cs Returns DynamoDB counter receipts.
src/​api/​Orleans.Clustering.ZooKeeper/​Orleans.Clustering.ZooKeeper.cs Updates ZooKeeper API surface.
src/​api/​Orleans.Clustering.TestKit/​Orleans.Clustering.TestKit.cs Updates TestKit API surface.
src/​api/​Orleans.Clustering.Consul/​Orleans.Clustering.Consul.cs Updates Consul API surface.
src/​api/​AdoNet/​Orleans.Clustering.AdoNet/​Orleans.Clustering.AdoNet.cs Updates ADO.NET API surface.
src/​AdoNet/​Orleans.Clustering.AdoNet/​Messaging/​AdoNetClusteringTable.cs Infers relational commit receipts.
docs/​site/​src/​content/​docs/​implementation/​provider-authoring.md Documents provider migration guidance.
docs/​site/​src/​content/​docs/​implementation/​cluster-management.md Documents snapshot and receipt semantics.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI balanced review requested due to automatic review settings October 2, 2026 18:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

ADO.NET rich writes can commit successfully and then throw while constructing an overflowing receipt token.

Review effort: Balanced
Findings: None

Previously missed (1)

In code that hasn't changed since last review

Medium severity Validate receipt token before insert to prevent post-commit overflow

src/​AdoNet/​Orleans.Clustering.AdoNet/​Messaging/​AdoNetClusteringTable.cs:115

The receipt token is computed only after the native insert has completed. With an expected ETag of int.MaxValue, the SQL write can commit the next database version and then CreateWriteResult throws OverflowException, so the caller receives a deterministic failure despite a committed mutation and no receipt (the new unit test explicitly exercises this). Validate that the next token is representable before issuing the write, then return the precomputed receipt only when the write succeeds.

This issue also appears on line 158 of the same file.

@ReubenBond

Copy link
Copy Markdown
Member Author

Review disposition: ADO.NET post-commit receipt overflow

Review 5395407025 contains one medium Previously missed finding recommending prevalidation, despite its top-level “Findings: None”. I reviewed that finding and am retaining the approved caller/native boundary rather than adding another provider guard.

  • The public TableVersion constructor accepts an arbitrary integer/string pair; Int32 typing alone does not prevent a fabricated candidate. However, Next() explicitly uses checked(Version + 1). The runtime obtains the successor before calling insert/update (status path, vote path, death path). ADO.NET maps a read's numeric version and ETag from the same value. Thus a valid observed int.MaxValue has no valid protocol successor. Next_AtVersionLimit_ThrowsOverflowException pins this caller boundary.
  • The risk is real for manually fabricated, out-of-protocol candidates: Oracle's version column is NUMBER(*,0), and its successful native mutation can commit a number beyond Int32. SQL Server/MySQL use INT, and PostgreSQL uses integer; their native numeric limits apply instead. Oracle here is source/scripted evidence, not a live Oracle result.
  • Existing bool entrypoints still return their existing native outcome. Rich entrypoints await the same operation, then construct the actual committed expected-ETag-plus-one receipt. MembershipReceipt_UnrepresentableOracleCommitToken_ThrowsAfterNativeSuccess explicitly verifies the post-native exception and unchanged bool outcome. False writes carry no receipt. There is no wrapped token, replay, or later read presented as originating-commit metadata.
  • Prevalidation would introduce a new malformed-input policy: for example, a stale maximum-valued ETag which the database would reject with false would instead throw before the native condition is evaluated; native error ordering would also change. The approved scope deliberately retains caller-valid successors and native failure semantics rather than introducing that guard or changing SQL catalogs.

Disposition: retain the explicit overflow outcome for this out-of-protocol/unrepresentable native result, acknowledge that it does not imply rollback, and preserve the unchanged bool contract. The finding exists only in the review body (no inline thread to resolve). Final native CI evidence will be reported separately and will not be described as Oracle execution.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage

Metric Pull request
Lines 82.88% (115,560 / 139,427)
Branches 72.21% (33,581 / 46,505)

Report-only conclusion: current-main baseline stale.

The newest successful coverage run tested 6bf11ad, not current main 9bb744a.

Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.

The comparison remains report-only while normal line and branch variance is calibrated.

Coverage details

@ReubenBond
ReubenBond marked this pull request as ready for review October 2, 2026 19:04
@ReubenBond

Copy link
Copy Markdown
Member Author

Final implementation and validation handoff

Head: 793181549c79f26c141353559b4df2677930554a. The exact-head gate is 85 successful checks, one expected Pages deployment skip, zero failed/pending checks. Native/full CI, documentation, static analysis and CodeQL all completed. The PR is ready for human review and remains unmerged.

Requirement Evidence
Optional receipts without breaking existing providers or bool RPCs DefaultWriteResult_InvokesBoolOnceWithoutReading, generated receipt roundtrips, GeneratedProxyCalls_PreserveLegacyWireCompatibility, and additive rich-RPC tests.
Exact originating-commit metadata Actual native receipt cases passed for Azure Table, Cosmos no-content batches, DynamoDB, Firestore, Consul, Redis, Cassandra and ZooKeeper; ADO.NET frozen-original/current-fresh cases passed on SQL Server, PostgreSQL and MariaDB. Selected exact-head TRX outcomes were checked directly. Oracle remains source/scripted coverage only.
ZooKeeper ambiguous outcomes remain exceptions Rich and bool wrapper commit/close-loss cases, callback-plus-close failure cases, owned Multi/close lifetime cases, and the landed sequential/read-fence regressions passed. Rich writes use the single-attempt session owner.
Row-read retirement and compatibility All built-in signatures remain and return unsupported after pre-cancellation; zero-I/O/zero-RPC cases pass. The custom legacy-provider default adapter and old generated request identities remain covered. Ordinary callers use full snapshots and TryGet.
Linear setup and explicit absent-receipt behavior ConcurrentReadSetup_UsesCommitReceiptsAndIndependentFinalView passes at N=5, 1001 and 4096, including provider-token variants: N actual richer inserts, three full snapshots, 2N returned membership rows, zero point reads. Missing-receipt coverage proves one successful write/one initial read, then an explicit prerequisite error without replay or readback.
Preserve history coverage while changing observations Manifest_DefaultGenerationPreservesOriginal959Cases and exact partition/multiplicity checks pass. All 18 internal operation identities remain; present/absent observations now use snapshot lookup. Normal discovery has 26 direct scenarios plus generated conformance (four generated partitions for the hosted target); G15/G22 remain tested obsolete aliases without duplicate normal discovery.
Generated surfaces and compatibility Core, TestKit, ADO.NET, Consul and ZooKeeper surfaces regenerated via GenAPI. Normal Release pack checks passed with no new compatibility suppressions. Documentation validation passed, including 240 site tests and link/output audits.

Review disposition: review 5395407025 contains a medium “Previously missed” ADO.NET overflow finding despite its top-level “Findings: None”. The source/test-linked disposition records the checked Next()/runtime caller boundary, unchanged bool behavior, and the explicit manual out-of-protocol Oracle post-commit exception caveat. No prevalidation guard, SQL expansion, replay or readback receipt was added. There are no inline review threads to resolve; this is not a claim that no concern was raised.

@ReubenBond
ReubenBond marked this pull request as draft October 3, 2026 15:02
@ReubenBond ReubenBond changed the title feat(membership): return commit receipts and retire row reads refactor(membership): retire row reads in favor of snapshots Oct 3, 2026
Add the result contract, compatibility defaults, runtime cores, and disjoint provider receipts while retaining existing bool APIs and row reads. Azure/Dynamo receipt patches and ADO receipt tests remain deferred for the separately owned wiring fixes; final retirement integration follows the wiring landing.
Copilot AI balanced review requested due to automatic review settings October 3, 2026 15:25
@ReubenBond
ReubenBond force-pushed the rb-refactor-retire-membership-row-reads branch from 7931815 to 68817c0 Compare October 3, 2026 15:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Retiring public read behavior across providers requires maintainer validation of compatibility and backend test results.

Review effort: Balanced
Findings: None

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants