Skip to content

fix(e2e): remove epoch gap flakiness - #3445

Draft
jpraynaud wants to merge 6 commits into
mainfrom
jpraynaud/fix-e2e-flakiness
Draft

fix(e2e): remove epoch gap flakiness#3445
jpraynaud wants to merge 6 commits into
mainfrom
jpraynaud/fix-e2e-flakiness

Conversation

@jpraynaud

Copy link
Copy Markdown
Member

Content

This PR includes...

Pre-submit checklist

  • Branch
    • Tests are provided (if possible)
    • Crates versions are updated (if relevant)
    • CHANGELOG file is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • All check jobs of the CI have succeeded
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested
  • Documentation
    • Update README file (if relevant)
    • Update documentation website (if relevant)
    • Add dev blog post (if relevant)
    • Add ADR blog post or Dev ADR entry (if relevant)
    • No new TODOs introduced

Comments

Issue(s)

Relates to #YYY or Closes #YYY

The exponential poll backoff exceeded short devnet epochs, so target
epochs could be detected up to a full epoch after their boundary.
A restart crossing an epoch boundary with a certificate chain two epochs
behind blocked the aggregator on an unrecoverable epoch gap.
A single epoch left no slack for certificate sealing latency, causing
spurious 'Timeout exhausted waiting for...' failures.
@jpraynaud jpraynaud self-assigned this Jul 30, 2026
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

Test Results

     5 files  ± 0     209 suites  ±0   59m 27s ⏱️ - 2h 15m 45s
 3 331 tests  - 75   3 331 ✅  - 75  0 💤 ±0  0 ❌ ±0 
11 093 runs   - 82  11 093 ✅  - 82  0 💤 ±0  0 ❌ ±0 

Results for commit 4847826. ± Comparison against base commit bf1c9b3.

This pull request removes 76 and adds 1 tests. Note that renamed tests count towards both.
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::index_out_of_bounds
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::index_too_large_for_circuit_range
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::indices_not_increasing
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_merkle_path_mismatch
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_swap_keep_merkle_path
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::leaf_wrong_verification_key
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_corrupt_sibling
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_flip_position
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_length_long
mithril-stm ‑ circuits::halo2::tests::golden::cases::negative::slow::merkle_path_length_short
…
mithril-end-to-end ‑ toolkit::context::tests::tenth_of_epoch_poll_backoff_polls_at_a_constant_delay

♻️ This comment has been updated with latest results.

@jpraynaud
jpraynaud temporarily deployed to testing-preview July 30, 2026 18:00 — with GitHub Actions Inactive
@jpraynaud
jpraynaud temporarily deployed to testing-2-preview July 30, 2026 18:00 — with GitHub Actions Inactive
@jpraynaud
jpraynaud temporarily deployed to testing-preview July 30, 2026 19:23 — with GitHub Actions Inactive
@jpraynaud
jpraynaud temporarily deployed to testing-2-preview July 30, 2026 19:23 — with GitHub Actions Inactive
@jpraynaud
jpraynaud temporarily deployed to testing-preview July 30, 2026 20:45 — with GitHub Actions Inactive
@jpraynaud
jpraynaud temporarily deployed to testing-2-preview July 30, 2026 20:45 — with GitHub Actions Inactive
@jpraynaud
jpraynaud temporarily deployed to testing-2-preview July 31, 2026 07:58 — with GitHub Actions Inactive
@jpraynaud
jpraynaud temporarily deployed to testing-preview July 31, 2026 07:58 — with GitHub Actions Inactive
Set RUST_LIB_BACKTRACE=0 so anyhow errors logged by spawned nodes
do not embed full backtraces.
@jpraynaud
jpraynaud force-pushed the jpraynaud/fix-e2e-flakiness branch 2 times, most recently from c6a1600 to 18be50a Compare July 31, 2026 11:18
Keep 'm' unchanged to avoid a signer computation spike at the change
epoch, and let a lone signer reach 'k' so a late signer cannot miss an epoch.
@jpraynaud
jpraynaud force-pushed the jpraynaud/fix-e2e-flakiness branch from 18be50a to 4847826 Compare July 31, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant