Skip to content

rpc: replay block transactions through the block executor (backport of paradigmxyz#28266) - #37

Merged
einar-oplabs merged 1 commit into
op-v2.6.0from
einar/backport-28266-485
Nov 6, 2026
Merged

einar-oplabs merged 1 commit into
op-v2.6.0from
einar/backport-28266-485

Conversation

@einar-oplabs

@einar-oplabs einar-oplabs commented Nov 5, 2026 •

Copy link
Copy Markdown

Backport of paradigmxyz#28266 to op-v2.6.0, for ethereum-optimism/protocol-team#485.

Every replay of a block's own transactions runs through the chain's block executor instead of a bare EVM: debug_trace* (canonical, raw and bad blocks), trace_*, ots_getTransactionError, debug_accountAt, debug_intermediateRoots, and the prefix replay before debug_traceCall at an index, eth_callMany and debug_traceCallMany. Replayed results therefore include the per-transaction changes the executor applies on top of the EVM result, which is what makes op-reth's SDM refund settlement visible to RPC with no op-reth override. Call gains block_executor and replay_transactions_until_with_executor; Trace::inspect_transaction_in_block takes the target index instead of a transaction environment and rejects an out-of-range index; TracingCtx borrows the result and owns the inspector, as upstream.

API changes: Trace::inspect_transaction_in_block loses its transaction-environment parameter, Call::replay_transactions_until_with_evm is replaced by replay_transactions_until_with_executor, and TracingCtx changes shape. Nothing in op-reth or op-reth-premium uses the old forms. Raw and bad blocks that the executor rejects error instead of returning pre-refund traces; that is upstream's semantics, taken as is.

Two differences from upstream, both forced by alloy-evm 0.39: the inspector is a type parameter of the EVM, so tracing paths build an inspected executor and enable the inspector only for the target transaction instead of calling upstream's inspect_transaction_without_commit; and there is no set_block_access_index, so the cached-BAL path still executes the target transaction on a bare EVM. Op-reth rejects computed BALs, so that path is unreachable there.

Supersedes #33, which kept raw debug_traceBlock and debug_traceBadBlock on the bare EVM. Consumer: ethereum-optimism/optimism#23942, whose raw-block cases already encode the executor semantics. Base: #38 is merged and merged into this branch, so the CI commits carried earlier are gone from the diff.

Landing: this merges on the fork without waiting for paradigmxyz#28266 upstream, since nothing downstream depends on the new API names; it is tagged op-v2.6.0-4 on its merge commit (op-v2.6.0-3 marks the base after #38 and carries no replay change). On evm2 bases the backport is dropped for a cherry-pick of paradigmxyz#28266 itself; see UPDATING-RETH.md in the monorepo for the carry rule.

Tests: the e2e test from upstream (replays_include_block_executor_settlement), a node whose executor credits a settlement to every sender, asserts that prestate diffs, block traces and intermediate roots agree with committed state; plus the out-of-range index unit test. Locally the e2e test passes and all-features clippy on reth-rpc and reth-rpc-eth-api is clean; reth-node-ethereum tests were linted with default features, since jit needs system LLVM.

Review: approved by George at ab7e467b; his three non-blocking nits are answered in the thread (base merged; n+1 inspector constructions per block is the floor without a Default bound leaking into the callback type; executor bookkeeping cost unmeasured). Upstream's later TracedInspector fix is evm2-only.

added removed net
implementation 254 177 +77
tests 418 0 +418
docs 0 0 0
total 672 177 +495

🤖 Generated with Claude Code

@geoknee geoknee left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. This is the right path for protocol-team#485. It gives a comprehensive fix with zero production code in op-reth, and the fork carries +257/−180, compared with the hooks plan's +416/−221 in the fork plus +437 in OP. Reviewed at 5141e03557.

Correctness

  • Settlement before commit. The approach depends on the executor applying per-transaction changes inside execute_transaction_without_commit. OpBlockExecutor does: it canonicalizes result gas and applies the SDM refund to result.state before returning (rust/alloy-op-evm/src/block/mod.rs, execute_transaction_without_commit). So the traced target transaction is settled, not just the prefix.

  • Refunds come from the block itself. executor_for_block / context_for_block derives PostExecMode::Verify(payload) from the block's own 0x7D transaction (rust/op-reth/crates/evm/src/lib.rs). Historical replays therefore apply the recorded refunds by index, exactly as import does.

  • Coverage. At this head every replay of a block's own transactions goes through the executor:

    • debug_trace by tx, block, raw block, bad block and chain;
    • trace_* and ots_*;
    • the prefix replays behind eth_callMany, debug_traceCall at an index, debug_traceCallMany and debug_accountAt;
    • debug_intermediateRoots.

    The remaining bare-EVM transact calls are synthetic calls and estimates, which correctly get no recorded refund, plus the cached-BAL branch, which op-reth can't reach because it denies BALs.

  • Settled fields: default and callTracer top-level gas, prestate and diff tracers, parity stateDiff, prefix state, and per-tx intermediate roots. Per-frame and per-opcode gas stays EVM-level by construction.

  • No downstream API exposure. Nothing in op-reth or op-reth-premium uses TracingCtx, the old inspect_transaction_in_block signature or replay_transactions_until_with_evm, so the breaking API change costs us nothing. Downstream is a pin bump plus optimism#23942's tests.

Decisions to make explicit

  1. Landing order. The PR body says this waits for paradigmxyz#28266 to merge. Nothing downstream depends on the new API names, so upstream churn during review can only affect us through behaviour. I'd land on the fork without waiting, re-synced to paradigmxyz#28266's latest head before tagging. Either way, please state the choice.
  2. Raw and bad blocks. debug_traceBlock and debug_traceBadBlock now error on blocks the executor rejects (malformed SDM payload, pre-refund gas over the limit), instead of returning pre-refund traces. I'd accept upstream's semantics: the #33-style carve-out would become permanent drift once paradigmxyz#28266 lands. If anyone needs pre-refund raw-block traces, raise it on paradigmxyz#28266 while it's in review. debug_replaySDMBlock only takes canonical refs, so it doesn't cover this.

Nits (non-blocking)

  • Merge #38 first; this PR carries its two CI commits.
  • trace_block_until_with_inspector calls inspector_setup() one or two more times than it needs per block.
  • Each traced transaction now also pays for executor bookkeeping (Jovian DA-footprint estimate, receipt building). It's probably fine, but ranged trace_filter gets slightly slower. I haven't measured it.

Forward-porting

  • op-v2.7.0 / v2.8.0. These are still revm 43 / alloy-evm 0.39, so this carries forward, but not cleanly. git merge-tree of 92fd8ee gives:

    • onto op-v2.7.0: conflicts in helpers/call.rs and rpc/src/trace.rs;
    • onto v2.8.0: conflicts in helpers/call.rs, helpers/trace.rs and debug.rs.

    The test commit is clean on op-v2.7.0. The OP repin and optimism#23942 must land before any OP bump to op-v2.7.0, otherwise the fix silently disappears.

  • v2.9.0+ (evm2). Drop this backport and cherry-pick paradigmxyz#28266 itself until a release includes it. For OP's evm2 port:

    • OP keeps the SDM refund-policy observer in the EVM inspector slot (PostExecCompositeInspector). paradigmxyz#28266's inspect_transaction_without_commit default installs and takes the tracer through that slot, which is the panic class its security bot flagged. OpBlockExecutor needs an override or composition there.
    • Upstream's BAL branch still skips per-tx settlement, so keep op-reth's no-BAL invariant under test.

Tests

  • op-reth (the real lock): optimism#23942. It runs on every reth pin bump. It needs to be:

    • repinned from #33's bff7be24 to this PR;
    • updated so the raw-block pre-refund assertions (debug_trace_post_exec.rs, "raw block traces are pre-refund") and the exhausted-budget case flip.

    Gaps worth adding:

    • debug_traceChain, which has its own loop and error policy and no coverage anywhere;
    • debug_accountAt after a refund;
    • one trace_filter or ots_* case;
    • tracers on the 0x7D transaction itself;
    • a synthetic call at an index asserting no refund is applied;
    • per-boundary intermediate roots;
    • the error shape for executor-rejected blocks.
  • alloy-op-evm: a unit test next to canonicalized_result_gas_applies_refund_exactly_once asserting that in Verify mode, with an inspector enabled, execute_transaction_without_commit returns settled gas and state. That's the exact property this PR relies on, and nothing pins it today.

  • This fork: keep only the upstream e2e test, so it travels with the carry. OP-specific tests belong in op-reth. Contributing trace_* and call-many prefix cases to paradigmxyz#28266 would bring them back for free on rebase.

@einar-oplabs einar-oplabs changed the title rpc!: replay block transactions through the block executor (backport of #28266) rpc: replay block transactions through the block executor (backport of #28266) Nov 6, 2026
@einar-oplabs
einar-oplabs marked this pull request as ready for review November 6, 2026 03:59
@einar-oplabs
einar-oplabs requested review from a team as code owners November 6, 2026 03:59
@einar-oplabs einar-oplabs changed the title rpc: replay block transactions through the block executor (backport of #28266) rpc: replay block transactions through the block executor (backport of paradigmxyz#28266) Nov 6, 2026
@einar-oplabs

Copy link
Copy Markdown
Author

Fable 5.1 (high): Thanks. (1) #38 is in and the base is merged, so the carried CI commits are gone from the diff. (2) trace_block_until_with_inspector builds n+1 inspectors for n transactions: the executor needs one at creation, and each transaction's tracer is moved out by replacing it. On alloy-evm 0.39 the inspector is a type parameter and components_mut only lends it, so dropping the last replacement would need a Default bound or an Option in the callback type; one spare construction per block seemed the cheaper wart. (3) Unmeasured; the per-transaction extra is the DA-footprint check and receipt building, the same work import does.

einar-oplabs added a commit to ethereum-optimism/optimism that referenced this pull request Nov 6, 2026
Pins reth to ethereum-optimism/reth#37, under which raw and bad blocks also
replay through the block executor: the raw-block trace now asserts settled
gas, and a raw block whose transfer exceeds the remaining block gas is
rejected with the executor's error instead of being traced.

Closes the coverage gaps from that PR's review: per-boundary intermediate
roots, callTracer and prestate diffs on the post-exec transaction, a
synthetic call at an index that earns no refund, trace_filter over an SDM
block, and an alloy-op-evm unit test pinning that a Verify-mode result is
settled before commit with an inspector enabled. The settlement test no
longer calls the fork's Trace API directly, so it survives API reshapes.
UPDATING-RETH.md records how the carried patch moves to later bases.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@einar-oplabs

Copy link
Copy Markdown
Author

Fable 5.1 (high): Decisions, per Einar: this lands on the fork without waiting for paradigmxyz#28266 to merge upstream, re-synced to that PR's head before the tag is cut; and raw and bad blocks keep upstream's semantics as is, so a block the executor rejects errors instead of returning pre-refund traces.

…paradigmxyz#28266)

Backports paradigmxyz#28266 to op-v2.6.0. Every replay of a block's own
transactions, canonical, raw or bad, runs through the chain's block executor
instead of a bare EVM, so traces, replayed state diffs, `debug_accountAt` and
`debug_intermediateRoots` include the per-transaction changes the executor
applies on top of the EVM result. `Call` gains `block_executor` and
`replay_transactions_until_with_executor`; `Trace::inspect_transaction_in_block`
takes the target transaction's index instead of a transaction environment and
rejects an out-of-range index; `TracingCtx` borrows the result and owns the
inspector, as upstream. Raw and bad blocks the executor rejects error instead
of returning pre-refund traces.

Differences from upstream, both forced by alloy-evm 0.39: the inspector is a
type parameter of the EVM, so tracing paths build an inspected executor and
enable the inspector only for the target transaction instead of calling
`inspect_transaction_without_commit`; and there is no `set_block_access_index`,
so the cached-BAL path still executes the target transaction on a bare EVM.

Ports upstream's end-to-end test: a node whose block executor credits a
settlement to every sender shows that prestate diffs, block traces and
intermediate roots agree with committed state.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@einar-oplabs
einar-oplabs force-pushed the einar/backport-28266-485 branch from ab7e467 to 65a3dde Compare November 6, 2026 05:01
@einar-oplabs
einar-oplabs merged commit 58c6608 into op-v2.6.0 Nov 6, 2026
30 checks passed
@einar-oplabs
einar-oplabs deleted the einar/backport-28266-485 branch November 6, 2026 05:33
einar-oplabs added a commit to ethereum-optimism/optimism that referenced this pull request Nov 6, 2026
The fork tag cut from ethereum-optimism/reth#37's merge commit replaces the
rev pin on that PR's head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
einar-oplabs added a commit to ethereum-optimism/optimism that referenced this pull request Nov 6, 2026
…replay

Pins reth to the fork tag op-v2.6.0-4, which replays every block
transaction through the block executor (ethereum-optimism/reth#37, a
backport of paradigmxyz/reth#28266), so debug, trace and otterscan replays
report SDM-settled gas and balances with no override in op-reth. Related to
ethereum-optimism/protocol-team#485.

Adds the integration tests that pin the behavior on a sequenced Lagoon
block with refunded, unrefunded, refund-dependent and reverting
transactions: per-transaction gas against receipts, prestate and parity
state diffs against `debug_accountAt`, intermediate roots per boundary, the
post-exec transaction as a replay target, a synthetic call at an index that
earns no refund, call-many on the settled prefix, `trace_filter`,
`traceChain` with its per-block error shape, and otterscan revert data.
Raw blocks trace with settled gas, and a raw block the executor rejects
returns the executor's error. An alloy-op-evm unit test pins that a
Verify-mode result is settled before commit with an inspector enabled.

Shared fixtures and node helpers live in the Lagoon fixture module. The
reth update guide documents the carried patch and the fork tagging rules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
aakoshh added a commit to ethereum-optimism/optimism that referenced this pull request Nov 10, 2026
develop now pins op-v2.6.0-4 (#23942), which
carries ethereum-optimism/reth#37: RPC block replays through the block
executor, which op-reth's SDM refund settlement depends on. Carry it
onto op-v2.7.0 as ethereum-optimism/reth#41, stacked on
ethereum-optimism/reth#40, and pin its head.

Resolve the conflicts with #23942:

- Cargo.toml and Cargo.lock: keep the op-v2.7.0 rev, now
  ethereum-optimism/reth#41's head.
- UPDATING-RETH.md: keep both sides, and update "Carried patches" for
  op-v2.7.0: where ethereum-optimism/reth#37's carry conflicted on
  v2.7.0, and the trie seek boundary fix carried as
  ethereum-optimism/reth#40.
- invalid_post_exec.rs: develop's crate::lagoon helpers in the
  E2ETestSetupBuilder call reth v2.7.0 needs.

Adapt #23942's new tests to reth v2.7.0: trace_replayTransaction returns
an optional TraceResultsWithTransactionHash, and StateProviderDatabase
takes an EvmStateProvider. Advance the reth mirror tags;
ethereum-optimism/reth#41 touches no mirrored symbol.
op-will pushed a commit that referenced this pull request Nov 11, 2026
…paradigmxyz#28266) (#37)

Backports paradigmxyz#28266 to op-v2.6.0. Every replay of a block's own
transactions, canonical, raw or bad, runs through the chain's block executor
instead of a bare EVM, so traces, replayed state diffs, `debug_accountAt` and
`debug_intermediateRoots` include the per-transaction changes the executor
applies on top of the EVM result. `Call` gains `block_executor` and
`replay_transactions_until_with_executor`; `Trace::inspect_transaction_in_block`
takes the target transaction's index instead of a transaction environment and
rejects an out-of-range index; `TracingCtx` borrows the result and owns the
inspector, as upstream. Raw and bad blocks the executor rejects error instead
of returning pre-refund traces.

Differences from upstream, both forced by alloy-evm 0.39: the inspector is a
type parameter of the EVM, so tracing paths build an inspected executor and
enable the inspector only for the target transaction instead of calling
`inspect_transaction_without_commit`; and there is no `set_block_access_index`,
so the cached-BAL path still executes the target transaction on a bare EVM.

Ports upstream's end-to-end test: a node whose block executor credits a
settlement to every sender shows that prestate diffs, block traces and
intermediate roots agree with committed state.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 58c6608)
op-will pushed a commit that referenced this pull request Nov 11, 2026
…paradigmxyz#28266) (#37)

Backports paradigmxyz#28266 to op-v2.6.0. Every replay of a block's own
transactions, canonical, raw or bad, runs through the chain's block executor
instead of a bare EVM, so traces, replayed state diffs, `debug_accountAt` and
`debug_intermediateRoots` include the per-transaction changes the executor
applies on top of the EVM result. `Call` gains `block_executor` and
`replay_transactions_until_with_executor`; `Trace::inspect_transaction_in_block`
takes the target transaction's index instead of a transaction environment and
rejects an out-of-range index; `TracingCtx` borrows the result and owns the
inspector, as upstream. Raw and bad blocks the executor rejects error instead
of returning pre-refund traces.

Differences from upstream, both forced by alloy-evm 0.39: the inspector is a
type parameter of the EVM, so tracing paths build an inspected executor and
enable the inspector only for the target transaction instead of calling
`inspect_transaction_without_commit`; and there is no `set_block_access_index`,
so the cached-BAL path still executes the target transaction on a bare EVM.

Ports upstream's end-to-end test: a node whose block executor credits a
settlement to every sender shows that prestate diffs, block traces and
intermediate roots agree with committed state.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 58c6608)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants