Repository navigation
fix: pick upstream RPC, p2p and storage fixes onto op-v2.7.0 - #44
Conversation
…adigmxyz#27462) Picked as a prerequisite of the eth_callBundle bound (paradigmxyz#28253), whose transaction loop stops on reth_tasks::cancel::is_cancelled(). Adapted: the fork's backport of paradigmxyz#28266 replays block transactions through the block executor. The cancellation checks upstream placed in the replaced EVM loops go into the executor loops instead (replay_transactions_until_with_executor, trace_block_until_with_inspector and debug intermediate roots), as upstream did when paradigmxyz#28266 landed. Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com> (cherry picked from commit 13a59de)
) Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com> (cherry picked from commit 0d1324f)
Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com> (cherry picked from commit 22e3708)
Adapted: the test imports StorageSlot from reth_revm::db::states (revm 43) instead of reth_trie_common::bundle_state, which wraps evm2 types upstream. (cherry picked from commit 3c8b4e8)
Adapted to revm 43 / alloy-evm 0.39, which this base still runs instead of evm2: - EIP-8037 detection reads `cfg_env.enable_amsterdam_eip8037` and the cap `Cfg::tx_gas_limit_cap()` instead of `version().feature(EIP8037)` and `version().tx_gas_limit_cap`, the same values alloy-evm's block executor admits transactions with. - Per-transaction regular and state gas come from `ResultGas::block_regular_gas_used()` / `block_state_gas_used()`, which compute what evm2's `execution_gas_spent()` / `state_gas_spent()` do. - The block gas limit is a u64 on the base's block env, and the loop keeps the base's peekable iteration and deferred commit. (cherry picked from commit 329fcfc)
…aradigmxyz#28314) Adapted to revm 43 / alloy-evm 0.39, which this base still runs instead of evm2. The limits, error variants, messages and codes are upstream's. - `CallManyBudget::admit` takes the call's `TransactionEnvMut` tx env directly; the base's call env is not wrapped in `Recovered`. - `CallManyBudget::charge` takes revm's `ExecutionResult<H>`. evm2's "out of gas, except MemoryLimitOOG" check becomes a halt with any `OutOfGasError` except `MemoryLimit`, matched through `H::from(HaltReason)` so it also covers wrapping halt reasons such as op-revm's. - The base's trace_callMany and debug_traceCallMany pass the tx env by value and an `&mut` inspector, so the budget calls wrap those `inspect` calls. - The budget's unit tests build revm `TxEnv`s and `ExecutionResult`s instead of evm2 envelopes and `TxResultExt`s; the core.rs test imports TraceType, BlockingTaskGuard and Runtime, which upstream's test module got through imports this base lacks. Not picked: 13a59de (paradigmxyz#27462) is not needed by this bound; it is in this stack only for paradigmxyz#28253's cancellation check. (cherry picked from commit faf670e)
…xyz#27716) Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com> (cherry picked from commit b5dc94c)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> (cherry picked from commit 9e10691)
Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com> (cherry picked from commit b5e7d63)
aakoshh
left a comment
There was a problem hiding this comment.
TL;DR: I think this PR can be merged as is. Below is a list of other upstream PRs we could cherry-pick alongside the included ones; I couldn't tell from the description why they were left out. Could you say for each group whether it was considered and why it was left out? Any you agree should go in can be a follow-up PR, so they don't hold this one up.
The picks themselves check out. I range-diffed each commit against its upstream original:
- 7 picks are identical apart from the
(cherry picked from …)line and hunk context. - Each of the 4 hand-ports changes only what its message says:
- paradigmxyz#27462 covers every cancellation site; the two missing ones are folded into
replay_transactions_until_with_executor. - paradigmxyz#28253's admission uses the same
block_regular_gas_used/block_state_gas_used/tx_gas_limit_capas alloy-evm 0.39's block executor. - paradigmxyz#28314's out-of-gas list maps 1:1 onto evm2's
is_out_of_gas()minusMemoryLimitOOG. - paradigmxyz#28148 only changes a test import.
- paradigmxyz#27462 covers every cancellation site; the two missing ones are folded into
No upstream commit reverts or fixes any of the picks.
I couldn't see the forward-sweep report, so I rebuilt the candidate list: 505 upstream commits newer than v2.7.0 (--cherry-pick against this branch), 263 of them fix/perf, about 210 after dropping CI/bench/build/test-only ones. Can you share the report, or say why these were left out? "Left for v2.8.1" can't apply to anything newer than v2.8.1.
Follow-ups to picked work
- paradigmxyz#28453 enforces the
eth_callBundletimeout. It finishes paradigmxyz#28253: a bundle is still bounded only by its gas pool. Main-only. - paradigmxyz#28386 bounds
mev_simBundlethe same way as paradigmxyz#28253. v2.11.0. - paradigmxyz#28312 and paradigmxyz#28390 extend paradigmxyz#27462's cancellation to
eth_simulateV1. v2.11.0. - paradigmxyz#28428 is the error-close companion of paradigmxyz#28423:
on_active_session_droppedat this base doesn't remove a peer thatremove_peermarked. Main-only. - paradigmxyz#28454 adds a 30s timeout to
debug_traceBlock. Main-only.
Same class as the HIGH picks (resource bounds, liveness), newer than v2.8.1
- paradigmxyz#28254 and paradigmxyz#28404: lock inversion in the libmdbx read-tx timeout monitor. At this base the monitor takes the tx mutex while holding a
DashMapshard guard (txn_manager.rs,start_monitor), so a read tx that is committed or dropped just as it expires can deadlock. - paradigmxyz#28251: a cancelled
pendingrequest keeps building the pending block. - paradigmxyz#28364: pending pool imports are bounded by count only, not by bytes.
- paradigmxyz#28360: banned or untrusted inbound peers are activated before they are rejected.
- paradigmxyz#28280: malformed Disconnect payloads aren't bounded.
- paradigmxyz#28332: pending sessions and the listener can starve under load.
- paradigmxyz#28326: IPC socket permissions.
- paradigmxyz#28198:
--netrestrictisn't applied to startup peers. - paradigmxyz#28345: discv5 ENR conversion can panic.
- paradigmxyz#28420: stages unwind on unrecoverable senders.
- paradigmxyz#28359: the txpool validator head goes stale after deep canonical commits.
Snap serving. Does op-reth serve snap at this base? If so: paradigmxyz#28375 (a range proof can reserve GBs), paradigmxyz#28367 (alloy-trie 0.9.9 panic fix; this base locks 0.9.7), paradigmxyz#28378, paradigmxyz#28392 (serves partially downloaded state) and paradigmxyz#28401 (a peer can grow the shared changeset cache).
In v2.8.1 (so they come with the next bump), but they look urgent enough to ask whether waiting was decided per fix: paradigmxyz#27270 (sparse trie reuse fork safety), paradigmxyz#27952 (WS origin allowlist), paradigmxyz#27864 and paradigmxyz#27866 (IPC oversized requests), paradigmxyz#27903 (JS tracer timeout), paradigmxyz#27465 (otterscan tracing permits), paradigmxyz#27842 (pending inbound rate limit), paradigmxyz#27868 and paradigmxyz#27722 (discv4 pong/neighbours source IP).
Exclusions I checked and agree with
- paradigmxyz#28249 can't be reached here: every production overlay caller passes
trie_changesets = true, because of the carried paradigmxyz#27634. - paradigmxyz#28246 was reverted upstream by paradigmxyz#28272.
- paradigmxyz#28162, paradigmxyz#28311 and paradigmxyz#28330 are evm2-only.
- The Amsterdam/EIP-8037-gated fixes don't apply.
- The pre-London
expectfix is indeed only in paradigmxyz#25002.
Description nits
- paradigmxyz#28423: before the fix, a second
remove_peeron a disconnecting peer dropped its entry, becauseis_connected()excludesDisconnecting*. The close handler then never released the slot. So the slot leaked rather than being "freed twice". - Where does the "roughly 240 MB per message" figure for paradigmxyz#28403 come from? Upstream doesn't give one.
Carrying forward. Only paradigmxyz#27462, paradigmxyz#27719 and paradigmxyz#27678 are in v2.8.1. The other 8 picks (paradigmxyz#28148 v2.9.0, paradigmxyz#28253 v2.10.0, the rest v2.11.0 only) have to be carried again on op-v2.8.1. Could they go in the guide's "Carried patches" list, together with the op-reth follow-ups for get_proof and build_pending_env? I couldn't find issues for those two.
|
🤖 Opus 5.5 says: Thanks for the review. Merged, and Forward sweep. It covered v2.7.0 up to upstream That leaves two gaps you found: paradigmxyz#28453 and paradigmxyz#28454 merged after that tip, and several of your candidates were in range with no recorded verdict (paradigmxyz#28251, paradigmxyz#28312/paradigmxyz#28390, paradigmxyz#28326, paradigmxyz#28198, paradigmxyz#28420, paradigmxyz#27952, paradigmxyz#27864/paradigmxyz#27866). A second sweep, up to current Nits. Both fixed in the description. paradigmxyz#28423 now says the slot leaked. The 240 MB was our own estimate for a 10 MiB message, not an upstream figure, and the description now says so. Carrying forward. Agreed. A monorepo PR adds the picks newer than v2.8.1 to the "Carried patches" list in |
Picks upstream fixes newer than v2.7.0 onto
op-v2.7.0, for theop-v2.7.0-1tag. A forward sweep of the reth update review (ethereum-optimism/optimism#24024) found them. Please rebase-merge, so each pick keeps its own commit and(cherry picked from commit …)line.Fixes:
eth_callMany,trace_callManyanddebug_traceCallManyhad no call-count or gas budget (fix(rpc): bound eth_callMany, trace_callMany and debug_traceCallMany paradigmxyz/reth#28314).eth_callBundlehad no gas bound and took no permit (fix(rpc): bound eth_callBundle work paradigmxyz/reth#28253).eth_getProof/eth_getMultiProoftook any number of keys (fix(rpc): cap eth_getProof and eth_getMultiProof key counts paradigmxyz/reth#27716).remove_peeron a disconnecting peer dropped its entry, so the close handler never released its connection slot and the slot leaked (fix(net): keep disconnecting peers on repeated removal paradigmxyz/reth#28423).BlockOverrides.time(fix(rpc): select simulated block fork from its timestamp paradigmxyz/reth#28431).paradigmxyz#27462 (cancel long blocking jobs once their request is dropped) is included because paradigmxyz#28253's loop calls its
is_cancelled().Upstream wrote paradigmxyz#27462, paradigmxyz#28253 and paradigmxyz#28314 on top of its evm2 refactor, so they are hand-ported to revm 43 / alloy-evm 0.39. Each of these commits says in its message what was adapted. An independent review against upstream found them faithful: the same limits, errors and gas accounting (EIP-7623 floor, state gas). The out-of-gas classification covers OP's
OpHaltReason, and deposit and post-exec transactions can't reach the budgets.paradigmxyz#28370 alone made
test_engine_tree_fcu_reorg_with_all_blocks_v2_e2efail about 1 run in 6. That was a real race, not a test artifact: during an on-disk reorg, a reader could decode the tip header's last value from stale bytes past its offset, and newPayload rejected a valid block as INVALID. Upstream avoids it with paradigmxyz#27678, so paradigmxyz#27719 and paradigmxyz#27678 are picked before it. With them, the test passes 24 of 24 runs.Not included:
.expect("fee is always valid")ineth_callBundlecan panic on pre-London state when abase_feeoverride exceeds a transaction's fee cap. Every OP chain has London active, and upstream's fix exists only inside the evm2 refactor (refactor!: replace revm with evm2 paradigmxyz/reth#25002), so this PR leaves it.main, picks candidates forop-v2.7.0-2.All picks except paradigmxyz#27462, paradigmxyz#27719 and paradigmxyz#27678 are newer than v2.8.1 and must be carried again onto
op-v2.8.1.op-reth's own copies of
get_proofandbuild_pending_envget the matching fixes in ethereum-optimism/optimism#24077 and ethereum-optimism/optimism#24078.🤖 Co-created with Claude Opus 5.5