Skip to content

test(ipsec-lb): investigate encrypted SQLite retirement replay failure after adapter restart #863

Description

@VerifiedOrganic

The kernel-independent conformance job failed in session_repin::tests::encrypted_sqlite_retirement_survives_adapter_restart when replaying retirement after reopening a standalone SQLite backend. The first retirement succeeded; the second retire call returned Io { operation: "session_repin_journal", kind: ConnectionRefused, raw_os_error: None } at crates/opc-ipsec-lb/src/session_repin.rs:7795.

Observed evidence:

  • First failing job, attempt 1: 458 tests passed and this test failed. The complete first log is preserved. No rerun has been requested.
  • PR head 3f000b68a50687b536d784dc88b72e46376db8e9; tested merge b97de11feb925b23e3050411e4634ddb5d409ce2; verified tree acdd22b3aa4c663993149f5e3602563a1d4e5749.
  • The failing harness is the first cargo test invocation in .github/workflows/ipsec-lb.yml's kernel-independent conformance job, using its original all-feature selection and four test threads. Preserve that workflow profile for reproduction.
  • The fixture reopens the same local database with the same in-memory key provider. It does not use a remote session store or start a Raft engine.

ConnectionRefused here is the journal's generic mapping of backend unavailability or replication-watch catch-up, not evidence of a failed network connection. The panic does not retain the underlying store error. The standalone SQLite read can report unavailability from its transaction/read/prune/commit path or bounded worker execution; none has been established as the cause. Capability rejection and crypto/serialization rejection have different journal outcomes.

This failure happened before the workflow's separately selected #856 compaction test. It does not reproduce #856. #811 concerns native WAL cancellation/cache validation; a shared store implementation does not establish that issue as the cause of this standalone failure. Track this failure separately until evidence establishes a relationship.

Acceptance:

  • Preserve the first failing log, exact source/tree, command profile and any subsequent observations. Passing retries alone are not a fix.
  • Establish the failing store operation and a bounded, value-free error class. Do not print record contents, keys, file paths, raw error strings or private operational data.
  • Produce a deterministic detector for the established cause before changing behavior. Retain the replay assertions: AlreadyRetired, unchanged retention deadline, and no active record returned by load.
  • Preserve the existing two-second SQLite worker bound, authority checks, durable retirement and cancellation semantics. Do not widen deadlines, skip the test, add blanket retries or substitute a different passing profile.
  • Prove the correction with fix-removal and adversarial controls, focused tests, the failing workflow profile, full required gates and independent review. Leave causality explicitly unresolved until demonstrated.

This issue records an observed qualification blocker, not a production failure claim or a diagnosed regression in the diagnostic-only PR #857.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions