Skip to content

op-reth: unit test the proofs storage startup reconciliation - #23419

Closed
einar-oplabs wants to merge 1 commit into
developfrom
einar/23387-proofs-reconcile-tests
Closed

einar-oplabs wants to merge 1 commit into
developfrom
einar/23387-proofs-reconcile-tests

Conversation

@einar-oplabs

@einar-oplabs einar-oplabs commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Pins down the proofs storage startup reconciliation that #23358 added, over every relation between the storage's window and reth's chain, so a regression in the reconcile or in the pruner cap that keeps the window reconcilable fails a unit test instead of a devnet. Closes #23387.

rust/op-reth/crates/exex/src/tests/startup_reconciliation.rs calls the production startup step (OpProofsExEx::startup, extracted from run) against a real BlockchainProvider holding blocks 1..=60, with reth's last 7 or 50 blocks kept in its canonical in-memory state so fork points land in the database or in memory, and the Finish stage checkpoint seeded at the persisted head. The canonical blocks are empty blocks whose state is the accumulated block reward, so a real EthEvmConfig in the ExEx context reproduces their state roots.

  • startup_reconciles_storage_with_reth_chain: rstest table over reth split {7, 50} x backend {MDBX, in-memory} for the cases that probe a block below reth's persisted head: storage behind reth, fork point below the window.
  • startup_reconciles_storage_with_reth_tip: the cases that probe only genesis or reth's tip, over backend: storage equal to reth, initialized at the tip, ahead on reth's chain, ahead from the tip, ahead on a fork above the tip, a single foreign block at height 0, window above reth's tip; plus an uninitialized storage.
  • startup_unwinds_storage_to_fork_point: the on-fork cases below reth's tip, over reth split {7, 50} x backend. They exercise the state-root criterion, which a match by height would miss, and check the unwound block's state root.
  • run_reconciles_storage_then_syncs: run itself, over backend: the startup error returns from run, and an unwind to fork point 50 is followed by run's sync_to, whose engine executes 51..=60 on the proofs storage's state and ends at reth's tip with the canonical state root.
  • startup_unwind_lets_reth_reimport_be_indexed: after the unwind, reth re-imports the blocks and the ExEx indexes them to the canonical state root (the Implement CI release pipeline #464 sequence), over persistence thresholds of 1 and 3 blocks for the 6 re-imported ones.
  • pruner_caps_earliest_at_reth_persisted_head: one case proving the real provider's Finish stage checkpoint reaches the pruner; the cap arithmetic is pinned by the pruner's own unit tests.

The reconcile_tests! suite and the helpers only it used are deleted; the tables cover its six scenarios. Production changes: the startup method, and the in-memory proofs store's cursors now wait for a writer instead of failing with TryLockError, which made the in-memory engine-sync cases flaky (mirrors #23449), and the variant, constructed nowhere any more, is removed.

Breakdown

added removed net
implementation 24 14 +10
tests 666 257 +409
docs 0 0 0

AI Reviews

  • 2026-11-10T13:47:41Z e1a6caa Fable 5.1 xhigh
  • 2026-11-10T13:42:06Z e1a6caa GPT-6.1-Sol xhigh

🤖 Generated with Claude Code

@einar-oplabs einar-oplabs self-assigned this Oct 14, 2026
@sebastianst
sebastianst force-pushed the seb/proofs-startup-reconcile branch from b601541 to 4e0fb2d Compare October 19, 2026 12:29
Base automatically changed from seb/proofs-startup-reconcile to develop October 22, 2026 09:29
@einar-oplabs
einar-oplabs force-pushed the einar/23387-proofs-reconcile-tests branch 3 times, most recently from c9231e5 to e1a6caa Compare November 10, 2026 13:19
@einar-oplabs
einar-oplabs marked this pull request as ready for review November 10, 2026 13:19
@einar-oplabs
einar-oplabs requested a review from a team as a code owner November 10, 2026 13:19
@einar-oplabs

Copy link
Copy Markdown
Contributor Author

Claude: GPT-6.1-Sol (xhigh): No actionable findings in the review of commit e1a6caaf73d762371f65d072a454dc7560f04677.

Reviewed fixture construction, production reconciliation and pruning behavior, startup error cases, and reimported state-root assertions.

Validation: all 82 focused tests passed, covering both storage backends, both reth persistence splits, and both engine flush cadences where applicable. The 27 unrelated tests were excluded by the filter. The tests ran outside the sandbox because the reth test context requires local network sockets.

cargo nextest run --locked --offline -p reth-optimism-exex startup_reconciliation

@einar-oplabs
einar-oplabs force-pushed the einar/23387-proofs-reconcile-tests branch from 510b9ec to 216004c Compare November 10, 2026 15:33
@einar-oplabs
einar-oplabs requested review from a team as code owners November 10, 2026 15:33
@mintlify

mintlify Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
Optimism-Docs 🟢 Ready View Preview Nov 10, 2026, 7:25 PM

@wiz-0f98cca50a

wiz-0f98cca50a Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities -
Data Finding Sensitive Data -
Secret Finding Secrets -
IaC Misconfiguration IaC Misconfigurations -
SAST Finding SAST Findings -
Software Management Finding Software Management Findings -
Total -

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try the Wiz Code extension for VS Code, JetBrains, or Visual Studio.

Covers the startup reconciliation of the proofs storage against reth's chain
as rstest tables over a real BlockchainProvider: storage behind, equal to or
ahead of reth on its chain, on a fork above, at or below reth's tip, a fork
point below the window, a window above reth's tip and an uninitialized
storage, with reth's last few or most blocks held in memory; the re-import
after an unwind, which must leave the canonical state root in the storage;
the pruner cap at reth's Finish stage checkpoint, and two cases driven
through run with a real EVM: a startup error returns promptly with the
storage untouched, and an unwind is followed by run's sync_to up to reth's
tip.

The ExEx's startup prologue moves into OpProofsExEx::startup so the tests
call what run calls, the reconcile_tests! suite and the helpers the tables
supersede are removed from lib.rs, and the in-memory proofs store's cursors
take the read lock with read() instead of try_read(), so a reader that lands
while the engine's save thread holds the write lock waits instead of failing
(the same change sits on optimism#23449 and becomes a no-op there), which
leaves OpProofsStorageError::TryLockError without a producer, so it goes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@einar-oplabs
einar-oplabs removed request for a team and maurelian November 11, 2026 12:29
@joshklop joshklop closed this Nov 21, 2026

This branch was successfully deployed

1 active deployment
staging - docs/public-docs — 69cd9da1 Deployed Nov 10, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

op-reth: unit tests for the proofs store startup reconciliation

2 participants