Skip to content

test(e2e): wait for pruner in RocksDB pruning tests (#27601), harden Osaka blob conversion (#27687) - #35

Merged
geoknee merged 6 commits into
af/act-stub-secure-runnerfrom
af/backport-e2e-race-fixes
Nov 11, 2026
Merged

geoknee merged 6 commits into
af/act-stub-secure-runnerfrom
af/backport-e2e-race-fixes

Conversation

@aakoshh

@aakoshh aakoshh commented Nov 5, 2026

Copy link
Copy Markdown

Stacked on #34. Both PRs together make reth-act pass on op-v2.7.0; see Caveats for how they can merge.

What

Cherry-picks six upstream test-only commits from v2.8.0 with -x. They fix two tests that race on CircleCI's runners:

  • f269bccbc test(e2e): wait for pruner in RocksDB pruning tests (paradigmxyz/reth#27601)
  • 0fc98b4d8 test(e2e): replace fixed sleeps with pool waits (paradigmxyz/reth#27500)
  • e29c3433c feat(e2e): control the next payload timestamp (paradigmxyz/reth#27596)
  • cdc4e096f feat(e2e): wait for the pool to process a new head (paradigmxyz/reth#27597)
  • 569c770ca feat(e2e): add import_payload helper (paradigmxyz/reth#27599)
  • 7a5c817ab test: harden Osaka blob conversion e2e (paradigmxyz/reth#27687)

Each pick has the same git patch-id --stable as its upstream commit.

Why

With #34's stub in place, the act jobs run again. Two of them fail on tests whose code is identical in v2.7.0 and in our branch:

  • act-e2e.yml: test_rocksdb_storage_history_pruning failed all three tries, and test_rocksdb_account_history_pruning failed once (job 3582). The test reads the history shards once the block is visible, which can be before the pruner has committed for it:
    Expected [16, 17, 18, 19, 20], got [15, 16, 17, 18, 19, 20]
    
  • act-integration.yml: blobs::blob_conversion_at_osaka failed all three tries:
    assertion failed: matches!(prague_payload.sidecars(), BlobSidecars::Eip4844(_))
    
    The test puts Osaka 24 seconds after the wall clock. That makes its first block exactly two slots before Osaka, so importing that block starts the pool's sidecar conversion (crates/transaction-pool/src/maintain.rs) 4 seconds later. On a slow runner, building and verifying the blob transactions takes longer than that. The first blob transaction has then been removed for conversion before the Prague payload is built. Both tests pass locally.

How

  • f269bccbc (upstream 27601) adds wait_for_prune_checkpoint and calls it in both RocksDB tests before they read the shards.
  • 7a5c817ab (upstream 27687) replaces the wall-clock timestamps with fixed ones, so the conversion window opens only when the test imports the Prague payload. That happens after it has checked both legacy sidecars. The fix uses harness helpers that the other four commits add, so those come along. They also add tests of their own in custom_genesis.rs, pool.rs and p2p.rs.

Caveats

Alternatives

  • Port test: harden Osaka blob conversion e2e #31 (the same fix, hand-ported to op-v2.6.0's harness as one commit) instead of the five upstream commits. That's a smaller diff, but it's fork-only: it has no cherry picked from line, so the next bump's audit can't drop it automatically.
  • Move Osaka further out in blob_conversion_at_osaka: a one-line change, but it would be fork-only. The next bump would have to drop it by hand, while the audit drops upstream picks automatically.

mattsse and others added 5 commits November 5, 2026 10:41
Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com>
(cherry picked from commit f269bcc)
Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com>
(cherry picked from commit 0fc98b4)
Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com>
(cherry picked from commit e29c343)
Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com>
(cherry picked from commit cdc4e09)
Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com>
(cherry picked from commit 569c770)
@aakoshh
aakoshh requested a review from a team as a code owner November 5, 2026 12:27
@aakoshh
aakoshh added this pull request to stack #36 November 5, 2026 12:27
@aakoshh
aakoshh force-pushed the af/backport-e2e-race-fixes branch from 3746976 to d7a863a Compare November 5, 2026 12:37
@aakoshh aakoshh changed the title test: backport upstream e2e race fixes test(e2e): wait for pruner in RocksDB pruning tests (#27601), harden Osaka blob conversion (#27687) Nov 10, 2026
aakoshh added a commit to ethereum-optimism/optimism that referenced this pull request Nov 10, 2026
Pin the reth crates to the head of ethereum-optimism/reth#35, which
stacks on ethereum-optimism/reth#34 on top of op-v2.7.0. Both are
unmerged, so the pin is a rev until op-v2.7.0-0 is tagged. Sync the
shared dependencies to the versions reth v2.7.0 requires: alloy 2.5.0,
alloy-hardforks 0.4.9, reth-core 0.8.1, revm 43 and revm-inspectors
0.44.

Adapt to the upstream API changes:

- StateProviderDatabase now takes an EvmStateProvider
  (paradigmxyz/reth#27109). Wrap each provider with
  into_evm_state_provider, on a reference where it is used again.
- CancelOnDrop moved from reth-revm to reth-tasks
  (paradigmxyz/reth#27466).
- StateProviderFactory gained a Primitives type
  (paradigmxyz/reth#26923). Where a bound also names
  NodePrimitivesProvider, the debug witness API now says which one it
  means.
- BlockAndReceipts holds SharedReceipts (paradigmxyz/reth#27427).
- The e2e setup functions folded into E2ETestSetupBuilder
  (paradigmxyz/reth#27432).
- MemoryOverlayStateProvider was removed in favour of the overlay
  manager (paradigmxyz/reth#26923). The test engine records every
  validated block in the manager it shares with its provider, so it now
  appends a block to its real parent with state_with_block_appended.
  The flashblock pending state does the same, but on the canonical
  anchor: a speculative block's parent is the pending block before it,
  which the overlay does not know. The next commit shows that this
  breaks for speculative builds.
aakoshh added a commit to ethereum-optimism/optimism that referenced this pull request Nov 10, 2026
develop's op-v2.6.0-2 carries upstream's iterator seek boundary fix
(paradigmxyz/reth#28264), which v2.7.0 lacks. ethereum-optimism/reth#40
cherry-picks it onto ethereum-optimism/reth#35, so pin its head until
op-v2.7.0-0 is tagged.

Only the reth git source changes in Cargo.lock, and the SP1 guest
workspace doesn't use reth. The fix only touches the trie node iterator,
which no UPSTREAM-MIRROR covers, so the reth mirror tags move to the new
rev unchanged.
@aakoshh
aakoshh requested a review from sebastianst November 10, 2026 15:27
@geoknee
geoknee enabled auto-merge (rebase) November 11, 2026 17:39
auto-merge was automatically disabled November 11, 2026 17:39

Pull request removed from stack

@geoknee
geoknee removed this pull request from stack #36 November 11, 2026 17:39
@geoknee
geoknee merged commit 3b69ad1 into af/act-stub-secure-runner Nov 11, 2026
30 checks passed
@geoknee
geoknee deleted the af/backport-e2e-race-fixes branch November 11, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants