Repository navigation
op-acceptance-tests,op-devstack: cover proofs history across an unclean shutdown - #23421
einar-oplabs wants to merge 1 commit into
Conversation
|
This pr has been automatically marked as stale and will be closed in 5 days if no updates |
c474193 to
35282e5
Compare
35282e5 to
2d72947
Compare
|
Claude: Detail moved out of the PR description when the branch was squashed into 2d72947. It is the description as it stood before the squash, unedited, so it describes the earlier commits. WhyAfter a SIGKILL, OOM or power loss, an op-reth proofs-history store usually holds blocks reth's datadir has not persisted (reth persists every What
Matrix, as table-driven subtests: threshold {7, 50} x proofs path {index =
Unit tests: Local verificationDebug op-reth built from this branch, The second test, run once with the skip removed, fails in all eight cells the same way: the sequencer rebuilds the store-tip block with a different hash, the restarted follower imports the chain up to the store's tip, and its op-reth exits on the next block: "expected" is the hash of the store's tip block on the chain that was killed, "got" the rebuilt block at the same height; the ExEx had skipped the rebuilt blocks below the tip by height. The test's own failure is the sync wait losing the follower's RPC, with the panic in the log right above it. After the review changes (both op-reths killed back to back before the CLs stop, store tip re-read right before the kill, workload setup-block precondition, recorder contract in the DSL), one cell of each was re-run: CI budget: with Not run: the kona-node CL variant ( Skipped cells
Findings and deviations from the issue
Stacked on #23418. Closes #23380 |
a1ecc68 to
af31ccb
Compare
8f5a66c to
272c097
Compare
272c097 to
259913d
Compare
…an shutdown SubProcess.Kill sends SIGKILL, waits a bounded time for the exit and fails unless the process died of that signal, so a crash before a deliberate kill fails the caller; OpReth.StopUngracefully, stack.UngracefulLifecycle and L2ELNode.StopUngracefully expose it. MixedSingleChainNodeSpec gains per-node L2CLOpts, with L2CLVerifierSyncMode and L2CLPostFinalizationELSync, and kona-node rejects options it cannot honor. The DSL gains Enode and ForgetPeer, so a node can drop a peer's trusted entry before it dials again, and BlockRecorderWorkload, the transfer and block-recorder load the restart, upgrade and crash tests share. Four top-level tests, one per test and kill schedule, run four cells each: the index proofs path under cl_sync, el_sync (post-finalization EL sync on) and el_sync_after_finality, and the execute path under el_sync. A proofs-history follower is held at a block its store has not flushed and killed there, restarted, and must catch up and serve proofs matching the sequencer's across the gap; under op-node the EL-sync cells also require op-node's log to show the sync path the cell names. In the rebuilt-chain tests both op-reths are killed back to back so the lost heights, which include the last workload block, are rebuilt with different contents, and the follower must serve the rebuilt chain's proofs from below the sequencer's persisted tip. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
259913d to
7f21b30
Compare
After a SIGKILL an op-reth proofs-history store can hold blocks reth's datadir never persisted, and nothing tested that a node comes back serving correct proofs.
Adds an unclean op-reth stop (
StopUngracefully, which fails unless the process died of the SIGKILL), per-node L2 CL options and a shareddsl.BlockRecorderWorkload, and four top-level tests (one per test and kill schedule) of four cells each (index proofs undercl_sync,el_sync,el_sync_after_finality; execute underel_sync): a follower held at an unflushed block and killed must catch up and serve the sequencer's proofs, with op-node's log showing the sync path each EL-sync cell names; and after both op-reths are killed the follower must serve the rebuilt chain's proofs (un-skipped, #23358 is in the base).Stacked on #23418. Closes #23380. Local verification, findings and notes: detail comment.
Breakdown
AI Reviews
<ISODATETIME><COMMITID>Fable 5.1 xhigh<ISODATETIME><COMMITID>GPT-6.1-Sol xhigh<ISODATETIME><COMMITID>Opus 5.5 xhigh🤖 Generated with Claude Code