Skip to content

Replace durable-stream attachment polling with lifecycle recovery - #3958

Merged
vigoo merged 30 commits into
mainfrom
durable-stream-lifecycle-cleanup
Sep 25, 2026
Merged

vigoo merged 30 commits into
mainfrom
durable-stream-lifecycle-cleanup

Conversation

@vigoo

@vigoo vigoo commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Behavior

Follow-up to #3928 and #3938. #3928 has merged, so this PR targets main; the original expiry branch merge remains in this branch ancestry.

  • Remove periodic maintenance of healthy stream attachments and active-attachment renewal. Prepared attachments retain deadlines.
  • Recover on producer load, committed session changes and stream terminals, and before deletion. Park after recovery; retry timers exist only for unfinished or failed work.
  • Repair a lost producer activation on read only with exact committed Active consumer authority. Healthy reads remain read-only.
  • Preserve replay/resume dependencies. Invocation completion or forwarding alone must not release producer history still needed by a reader.
  • Make forwarding ownership explicit through canonical intents, exact destination bindings and containing publication records, with admission/publication barriers.

Deliberate tradeoff

Stale producer-side attachment bookkeeping is lazy: a healthy resident producer need not discover a remote consumer disappearance until later lifecycle activity or deletion. Deletion reconciles dependencies before deciding safety. This does not add atomic storage transactions, an outbox redesign, a global cleanup scan, or payload copying. It does change attachment/forwarding records within the existing oplog model; no compatibility path is provided, per repository policy.

The durable-stream maintenance poll is gone, not every timer in the executor. Expiry scheduling, protocol long polls, and bounded retries of unfinished/failed recovery retain their separate responsibilities.

Validation

  • Full common library suite: 1,364 passed, 1 ignored.
  • Full executor library suite: 2,411 passed, 8 ignored.
  • 98 selected executor integration tests passed: durable streams/sessions, RPC, worker initialization, fork/expiry, active-agent lifecycle and oplog sweeping.
  • cargo clippy -p golem-common -p golem-worker-executor --all-targets --no-deps -- -D warnings passed.
  • Package-scoped Rust formatting and git diff --check passed.
  • cargo make generate-configs and direct executor config drift checks passed.
  • Docs formatting/version links checked; operations documentation and executor walkthrough rendered and inspected.
  • Oracle approved implementation stages, both upstream merges, generated-config follow-through, and the guest-trap failure wakeup fix. Final-stage bug finder reported no bugs.

Full GitHub CI run passed, including all five worker-executor groups, service and CLI integration groups, workspace unit/drift checks, and SDK checks. The entire executor/service/CLI integration matrix was run in CI rather than locally.

vigoo and others added 28 commits September 22, 2026 08:54
…m-expiry

Amp-Thread-ID: https://ampcode.com/threads/T-01a0c3f9-2a89-745e-a286-486bd9969a77
Co-authored-by: Amp <amp@ampcode.com>

# Conflicts:
#	golem-worker-executor/src/services/worker_fork/export.rs
#	golem-worker-executor/src/storage/indexed/mod.rs
Remove renewal records and operations while preserving preparation, exact authority checks, epoch fencing, and deletion dependencies.

Oracle approved this bounded protocol stage. Targeted stream, session, projection, worker, and expiry tests: 339 passed, 1 existing ignored. Lifecycle delivery and poller removal follow separately.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0cd74-b013-75dd-b5bf-9d5f8a21e245
Co-authored-by: Amp <amp@ampcode.com>
Keep per-reader publication identity exact while allowing independent readers to share the same destination binding.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0cd74-b013-75dd-b5bf-9d5f8a21e245
Co-authored-by: Amp <amp@ampcode.com>
Use committed publication and attachment evidence, preserving fork and revert boundaries. Retry only on retained-history changes, not ordinary append progress.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0cd74-b013-75dd-b5bf-9d5f8a21e245
Co-authored-by: Amp <amp@ampcode.com>
Prove historical destination acceptance outside the source writer, then revalidate the retained intent and preserve its attribution when committing one idempotent receipt. Keep reader disposition separate from attachment history retention.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0cd74-b013-75dd-b5bf-9d5f8a21e245
Co-authored-by: Amp <amp@ampcode.com>
Park healthy attachment maintenance until committed lifecycle changes. Repair lost activation on an authorized read, preserve failure retries, and remove unused forwarding acceptance receipts and inspector-only indexes.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0cd74-b013-75dd-b5bf-9d5f8a21e245
Co-authored-by: Amp <amp@ampcode.com>
Preserve the lifecycle-driven stream recovery changes alongside upstream deferred ephemeral completion and retired-worker metadata handling.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a0cd74-b013-75dd-b5bf-9d5f8a21e245
@vigoo
vigoo requested a review from a team September 24, 2026 10:06
@netlify

netlify Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for golemcloud canceled.

Name Link
🔨 Latest commit 175b426
🔍 Latest deploy log https://app.netlify.com/projects/golemcloud/deploys/6ab4fbde414bd90008ccb10c

vigoo and others added 2 commits September 24, 2026 10:21
Preserve executor-shutdown cancellation and joining alongside event-driven stream recovery.

Amp-Thread-ID: https://ampcode.com/threads/T-01a0cd74-b013-75dd-b5bf-9d5f8a21e245
Co-authored-by: Amp <amp@ampcode.com>
@github-actions

Copy link
Copy Markdown

📖 Docs preview: https://docs-4vp93f2gv-golem-cloud.vercel.app

Built from commit 175b426a331f169e4fb531e15116e68ef93fbad8 by docs.yaml.

@vigoo
vigoo merged commit 3a02cf4 into main Sep 25, 2026
71 checks passed
@vigoo
vigoo deleted the durable-stream-lifecycle-cleanup branch September 25, 2026 10:52
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 25, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants