Repository navigation
rpc: replay block transactions through the block executor (backport of paradigmxyz#28266) - #37
Conversation
geoknee
left a comment
There was a problem hiding this comment.
Approving. This is the right path for protocol-team#485. It gives a comprehensive fix with zero production code in op-reth, and the fork carries +257/−180, compared with the hooks plan's +416/−221 in the fork plus +437 in OP. Reviewed at 5141e03557.
Correctness
-
Settlement before commit. The approach depends on the executor applying per-transaction changes inside
execute_transaction_without_commit.OpBlockExecutordoes: it canonicalizes result gas and applies the SDM refund toresult.statebefore returning (rust/alloy-op-evm/src/block/mod.rs,execute_transaction_without_commit). So the traced target transaction is settled, not just the prefix. -
Refunds come from the block itself.
executor_for_block/context_for_blockderivesPostExecMode::Verify(payload)from the block's own 0x7D transaction (rust/op-reth/crates/evm/src/lib.rs). Historical replays therefore apply the recorded refunds by index, exactly as import does. -
Coverage. At this head every replay of a block's own transactions goes through the executor:
- debug_trace by tx, block, raw block, bad block and chain;
trace_*andots_*;- the prefix replays behind
eth_callMany,debug_traceCallat an index,debug_traceCallManyanddebug_accountAt; debug_intermediateRoots.
The remaining bare-EVM
transactcalls are synthetic calls and estimates, which correctly get no recorded refund, plus the cached-BAL branch, which op-reth can't reach because it denies BALs. -
Settled fields: default and callTracer top-level gas, prestate and diff tracers, parity
stateDiff, prefix state, and per-tx intermediate roots. Per-frame and per-opcode gas stays EVM-level by construction. -
No downstream API exposure. Nothing in op-reth or op-reth-premium uses
TracingCtx, the oldinspect_transaction_in_blocksignature orreplay_transactions_until_with_evm, so the breaking API change costs us nothing. Downstream is a pin bump plus optimism#23942's tests.
Decisions to make explicit
- Landing order. The PR body says this waits for paradigmxyz#28266 to merge. Nothing downstream depends on the new API names, so upstream churn during review can only affect us through behaviour. I'd land on the fork without waiting, re-synced to paradigmxyz#28266's latest head before tagging. Either way, please state the choice.
- Raw and bad blocks.
debug_traceBlockanddebug_traceBadBlocknow error on blocks the executor rejects (malformed SDM payload, pre-refund gas over the limit), instead of returning pre-refund traces. I'd accept upstream's semantics: the #33-style carve-out would become permanent drift once paradigmxyz#28266 lands. If anyone needs pre-refund raw-block traces, raise it on paradigmxyz#28266 while it's in review.debug_replaySDMBlockonly takes canonical refs, so it doesn't cover this.
Nits (non-blocking)
- Merge #38 first; this PR carries its two CI commits.
trace_block_until_with_inspectorcallsinspector_setup()one or two more times than it needs per block.- Each traced transaction now also pays for executor bookkeeping (Jovian DA-footprint estimate, receipt building). It's probably fine, but ranged
trace_filtergets slightly slower. I haven't measured it.
Forward-porting
-
op-v2.7.0 / v2.8.0. These are still revm 43 / alloy-evm 0.39, so this carries forward, but not cleanly.
git merge-treeof92fd8eegives:- onto
op-v2.7.0: conflicts inhelpers/call.rsandrpc/src/trace.rs; - onto
v2.8.0: conflicts inhelpers/call.rs,helpers/trace.rsanddebug.rs.
The test commit is clean on
op-v2.7.0. The OP repin and optimism#23942 must land before any OP bump toop-v2.7.0, otherwise the fix silently disappears. - onto
-
v2.9.0+ (evm2). Drop this backport and cherry-pick paradigmxyz#28266 itself until a release includes it. For OP's evm2 port:
- OP keeps the SDM refund-policy observer in the EVM inspector slot (
PostExecCompositeInspector). paradigmxyz#28266'sinspect_transaction_without_commitdefault installs and takes the tracer through that slot, which is the panic class its security bot flagged.OpBlockExecutorneeds an override or composition there. - Upstream's BAL branch still skips per-tx settlement, so keep op-reth's no-BAL invariant under test.
- OP keeps the SDM refund-policy observer in the EVM inspector slot (
Tests
-
op-reth (the real lock): optimism#23942. It runs on every reth pin bump. It needs to be:
- repinned from #33's
bff7be24to this PR; - updated so the raw-block pre-refund assertions (
debug_trace_post_exec.rs, "raw block traces are pre-refund") and the exhausted-budget case flip.
Gaps worth adding:
debug_traceChain, which has its own loop and error policy and no coverage anywhere;debug_accountAtafter a refund;- one
trace_filterorots_*case; - tracers on the 0x7D transaction itself;
- a synthetic call at an index asserting no refund is applied;
- per-boundary intermediate roots;
- the error shape for executor-rejected blocks.
- repinned from #33's
-
alloy-op-evm: a unit test next to
canonicalized_result_gas_applies_refund_exactly_onceasserting that in Verify mode, with an inspector enabled,execute_transaction_without_commitreturns settled gas and state. That's the exact property this PR relies on, and nothing pins it today. -
This fork: keep only the upstream e2e test, so it travels with the carry. OP-specific tests belong in op-reth. Contributing
trace_*and call-many prefix cases to paradigmxyz#28266 would bring them back for free on rebase.
|
Fable 5.1 (high): Thanks. (1) #38 is in and the base is merged, so the carried CI commits are gone from the diff. (2) |
Pins reth to ethereum-optimism/reth#37, under which raw and bad blocks also replay through the block executor: the raw-block trace now asserts settled gas, and a raw block whose transfer exceeds the remaining block gas is rejected with the executor's error instead of being traced. Closes the coverage gaps from that PR's review: per-boundary intermediate roots, callTracer and prestate diffs on the post-exec transaction, a synthetic call at an index that earns no refund, trace_filter over an SDM block, and an alloy-op-evm unit test pinning that a Verify-mode result is settled before commit with an inspector enabled. The settlement test no longer calls the fork's Trace API directly, so it survives API reshapes. UPDATING-RETH.md records how the carried patch moves to later bases. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Fable 5.1 (high): Decisions, per Einar: this lands on the fork without waiting for paradigmxyz#28266 to merge upstream, re-synced to that PR's head before the tag is cut; and raw and bad blocks keep upstream's semantics as is, so a block the executor rejects errors instead of returning pre-refund traces. |
…paradigmxyz#28266) Backports paradigmxyz#28266 to op-v2.6.0. Every replay of a block's own transactions, canonical, raw or bad, runs through the chain's block executor instead of a bare EVM, so traces, replayed state diffs, `debug_accountAt` and `debug_intermediateRoots` include the per-transaction changes the executor applies on top of the EVM result. `Call` gains `block_executor` and `replay_transactions_until_with_executor`; `Trace::inspect_transaction_in_block` takes the target transaction's index instead of a transaction environment and rejects an out-of-range index; `TracingCtx` borrows the result and owns the inspector, as upstream. Raw and bad blocks the executor rejects error instead of returning pre-refund traces. Differences from upstream, both forced by alloy-evm 0.39: the inspector is a type parameter of the EVM, so tracing paths build an inspected executor and enable the inspector only for the target transaction instead of calling `inspect_transaction_without_commit`; and there is no `set_block_access_index`, so the cached-BAL path still executes the target transaction on a bare EVM. Ports upstream's end-to-end test: a node whose block executor credits a settlement to every sender shows that prestate diffs, block traces and intermediate roots agree with committed state. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ab7e467 to
65a3dde
Compare
The fork tag cut from ethereum-optimism/reth#37's merge commit replaces the rev pin on that PR's head. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…replay Pins reth to the fork tag op-v2.6.0-4, which replays every block transaction through the block executor (ethereum-optimism/reth#37, a backport of paradigmxyz/reth#28266), so debug, trace and otterscan replays report SDM-settled gas and balances with no override in op-reth. Related to ethereum-optimism/protocol-team#485. Adds the integration tests that pin the behavior on a sequenced Lagoon block with refunded, unrefunded, refund-dependent and reverting transactions: per-transaction gas against receipts, prestate and parity state diffs against `debug_accountAt`, intermediate roots per boundary, the post-exec transaction as a replay target, a synthetic call at an index that earns no refund, call-many on the settled prefix, `trace_filter`, `traceChain` with its per-block error shape, and otterscan revert data. Raw blocks trace with settled gas, and a raw block the executor rejects returns the executor's error. An alloy-op-evm unit test pins that a Verify-mode result is settled before commit with an inspector enabled. Shared fixtures and node helpers live in the Lagoon fixture module. The reth update guide documents the carried patch and the fork tagging rules. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
develop now pins op-v2.6.0-4 (#23942), which carries ethereum-optimism/reth#37: RPC block replays through the block executor, which op-reth's SDM refund settlement depends on. Carry it onto op-v2.7.0 as ethereum-optimism/reth#41, stacked on ethereum-optimism/reth#40, and pin its head. Resolve the conflicts with #23942: - Cargo.toml and Cargo.lock: keep the op-v2.7.0 rev, now ethereum-optimism/reth#41's head. - UPDATING-RETH.md: keep both sides, and update "Carried patches" for op-v2.7.0: where ethereum-optimism/reth#37's carry conflicted on v2.7.0, and the trie seek boundary fix carried as ethereum-optimism/reth#40. - invalid_post_exec.rs: develop's crate::lagoon helpers in the E2ETestSetupBuilder call reth v2.7.0 needs. Adapt #23942's new tests to reth v2.7.0: trace_replayTransaction returns an optional TraceResultsWithTransactionHash, and StateProviderDatabase takes an EvmStateProvider. Advance the reth mirror tags; ethereum-optimism/reth#41 touches no mirrored symbol.
…paradigmxyz#28266) (#37) Backports paradigmxyz#28266 to op-v2.6.0. Every replay of a block's own transactions, canonical, raw or bad, runs through the chain's block executor instead of a bare EVM, so traces, replayed state diffs, `debug_accountAt` and `debug_intermediateRoots` include the per-transaction changes the executor applies on top of the EVM result. `Call` gains `block_executor` and `replay_transactions_until_with_executor`; `Trace::inspect_transaction_in_block` takes the target transaction's index instead of a transaction environment and rejects an out-of-range index; `TracingCtx` borrows the result and owns the inspector, as upstream. Raw and bad blocks the executor rejects error instead of returning pre-refund traces. Differences from upstream, both forced by alloy-evm 0.39: the inspector is a type parameter of the EVM, so tracing paths build an inspected executor and enable the inspector only for the target transaction instead of calling `inspect_transaction_without_commit`; and there is no `set_block_access_index`, so the cached-BAL path still executes the target transaction on a bare EVM. Ports upstream's end-to-end test: a node whose block executor credits a settlement to every sender shows that prestate diffs, block traces and intermediate roots agree with committed state. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 58c6608)
…paradigmxyz#28266) (#37) Backports paradigmxyz#28266 to op-v2.6.0. Every replay of a block's own transactions, canonical, raw or bad, runs through the chain's block executor instead of a bare EVM, so traces, replayed state diffs, `debug_accountAt` and `debug_intermediateRoots` include the per-transaction changes the executor applies on top of the EVM result. `Call` gains `block_executor` and `replay_transactions_until_with_executor`; `Trace::inspect_transaction_in_block` takes the target transaction's index instead of a transaction environment and rejects an out-of-range index; `TracingCtx` borrows the result and owns the inspector, as upstream. Raw and bad blocks the executor rejects error instead of returning pre-refund traces. Differences from upstream, both forced by alloy-evm 0.39: the inspector is a type parameter of the EVM, so tracing paths build an inspected executor and enable the inspector only for the target transaction instead of calling `inspect_transaction_without_commit`; and there is no `set_block_access_index`, so the cached-BAL path still executes the target transaction on a bare EVM. Ports upstream's end-to-end test: a node whose block executor credits a settlement to every sender shows that prestate diffs, block traces and intermediate roots agree with committed state. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 58c6608)
Backport of paradigmxyz#28266 to
op-v2.6.0, for ethereum-optimism/protocol-team#485.Every replay of a block's own transactions runs through the chain's block executor instead of a bare EVM:
debug_trace*(canonical, raw and bad blocks),trace_*,ots_getTransactionError,debug_accountAt,debug_intermediateRoots, and the prefix replay beforedebug_traceCallat an index,eth_callManyanddebug_traceCallMany. Replayed results therefore include the per-transaction changes the executor applies on top of the EVM result, which is what makes op-reth's SDM refund settlement visible to RPC with no op-reth override.Callgainsblock_executorandreplay_transactions_until_with_executor;Trace::inspect_transaction_in_blocktakes the target index instead of a transaction environment and rejects an out-of-range index;TracingCtxborrows the result and owns the inspector, as upstream.API changes:
Trace::inspect_transaction_in_blockloses its transaction-environment parameter,Call::replay_transactions_until_with_evmis replaced byreplay_transactions_until_with_executor, andTracingCtxchanges shape. Nothing in op-reth or op-reth-premium uses the old forms. Raw and bad blocks that the executor rejects error instead of returning pre-refund traces; that is upstream's semantics, taken as is.Two differences from upstream, both forced by alloy-evm 0.39: the inspector is a type parameter of the EVM, so tracing paths build an inspected executor and enable the inspector only for the target transaction instead of calling upstream's
inspect_transaction_without_commit; and there is noset_block_access_index, so the cached-BAL path still executes the target transaction on a bare EVM. Op-reth rejects computed BALs, so that path is unreachable there.Supersedes #33, which kept raw
debug_traceBlockanddebug_traceBadBlockon the bare EVM. Consumer: ethereum-optimism/optimism#23942, whose raw-block cases already encode the executor semantics. Base: #38 is merged and merged into this branch, so the CI commits carried earlier are gone from the diff.Landing: this merges on the fork without waiting for paradigmxyz#28266 upstream, since nothing downstream depends on the new API names; it is tagged
op-v2.6.0-4on its merge commit (op-v2.6.0-3marks the base after #38 and carries no replay change). On evm2 bases the backport is dropped for a cherry-pick of paradigmxyz#28266 itself; seeUPDATING-RETH.mdin the monorepo for the carry rule.Tests: the e2e test from upstream (
replays_include_block_executor_settlement), a node whose executor credits a settlement to every sender, asserts that prestate diffs, block traces and intermediate roots agree with committed state; plus the out-of-range index unit test. Locally the e2e test passes and all-features clippy onreth-rpcandreth-rpc-eth-apiis clean;reth-node-ethereumtests were linted with default features, sincejitneeds system LLVM.Review: approved by George at
ab7e467b; his three non-blocking nits are answered in the thread (base merged; n+1 inspector constructions per block is the floor without aDefaultbound leaking into the callback type; executor bookkeeping cost unmeasured). Upstream's laterTracedInspectorfix is evm2-only.🤖 Generated with Claude Code