Skip to content

op-acceptance-tests,op-devstack,op-service: cover eth version negotiation and transaction propagation between op-reth and op-geth - #23443

Closed
einar-oplabs wants to merge 1 commit into
einar/23399-fcu-stale-safe-finalizedfrom
einar/23402-eth71-opgeth-peering
Closed

einar-oplabs wants to merge 1 commit into
einar/23399-fcu-stale-safe-finalizedfrom
einar/23402-eth71-opgeth-peering

Conversation

@einar-oplabs

@einar-oplabs einar-oplabs commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Nothing checked how op-reth and op-geth peer and exchange transactions, and reth 2.6 changes the default eth version, the transaction fetcher and peer reputation handling.

op_reth_opgeth_peering peers an op-reth and an op-geth follower as untrusted peers and requires the highest common eth version, transaction propagation both ways, and that op-reth, offered more transaction hashes than its per-peer cap, keeps the session, includes at least a cap's worth of the burst and accounts for every burst transaction it pooled. Adds basic EL peering, typed admin_peers, batch submission and pool waits to the devstack.

Stacked on #23440. Closes #23402. Local verification, findings and notes: detail comment.

Breakdown

Added Removed Net
Implementation 743 53 +690
Tests 768 0 +768
Docs 0 0 0

AI Reviews

  • <ISODATETIME> <COMMITID> Fable 5.1 xhigh
  • <ISODATETIME> <COMMITID> GPT-6.1-Sol xhigh
  • <ISODATETIME> <COMMITID> Opus 5.5 xhigh

🤖 Generated with Claude Code

@opgitgovernance

Copy link
Copy Markdown
Contributor

This pr has been automatically marked as stale and will be closed in 5 days if no updates

@opgitgovernance opgitgovernance added the S-stale Status: Will be closed unless there is activity label Oct 30, 2026
@einar-oplabs einar-oplabs reopened this Nov 6, 2026
@einar-oplabs einar-oplabs removed the S-stale Status: Will be closed unless there is activity label Nov 6, 2026
@einar-oplabs
einar-oplabs force-pushed the einar/23399-fcu-stale-safe-finalized branch from 2a9b08c to c7e6250 Compare November 9, 2026 21:32
@einar-oplabs
einar-oplabs force-pushed the einar/23402-eth71-opgeth-peering branch from 56e7d65 to e63e2d9 Compare November 9, 2026 21:32
@einar-oplabs
einar-oplabs force-pushed the einar/23399-fcu-stale-safe-finalized branch from c7e6250 to a3c78e4 Compare November 10, 2026 13:38
@einar-oplabs
einar-oplabs force-pushed the einar/23402-eth71-opgeth-peering branch from e63e2d9 to 3c2a8a0 Compare November 10, 2026 13:38
@einar-oplabs

Copy link
Copy Markdown
Contributor Author

Claude: Detail moved out of the PR description when the branch was squashed into 3c2a8a0. It is the description as it stood before the squash, unedited, so it describes the earlier commits.


Nothing checked how op-reth and op-geth peer and exchange transactions: rust/op-reth/crates/node/tests/it/p2p_version.rs pins the version two op-reth nodes negotiate, and every devstack peering is trusted, which exempts the session from the reputation handling the reth 2.6 bump (#23046) changes along with the default protocol (eth/71) and the transaction fetcher (reth#26924, a per-peer cap of 4096 announced hashes). This adds an acceptance test package that peers an op-reth follower and an op-geth follower as basic peers and checks the negotiated version, transaction propagation in both directions, and op-reth under an announcement burst from op-geth, written against hooks that are the same on develop and on the bump.

What the tests cover

All three tests run on a pairSystem (op-acceptance-tests/tests/op_reth_opgeth_peering/system_test.go): an op-reth sequencer, an op-reth follower and an op-geth follower, 1s blocks. One follower is the sender: it is kept off the mixed runtime's trusted mesh (IsolateFromL2P2P), its consensus client is peered with the sequencer's alone, and it dials the other follower (the receiver) with the new PeerWithBasic, so the receiver is its only execution-layer peer and the session is trusted on neither side. The receiver keeps the trusted peering with the sequencer that the runtime set up. A transaction submitted to the sender can therefore only reach the sequencer through the receiver, so its inclusion proves it crossed the peering under test; with the sequencer paused, the receiver's pool can be observed. The system is returned once the sender has filled its gap to the chain through the receiver.

  • TestOpRethAndOpGethNegotiateTheHighestCommonEthVersion, subtests op-reth_dials_op-geth and op-geth_dials_op-reth: both sides must report the same negotiated eth version through admin_peers, equal to the highest version both advertise (sysgo.HighestCommonEthVersion over the two caps lists), and the advertised versions are logged.
  • TestTransactionCrossesTheOpRethOpGethPeeringIntoABlock, subtests op-reth_to_op-geth and op-geth_to_op-reth: with the sequencer paused, a transfer submitted to the sender must show up in the receiver's pool (eth_getTransactionByHash with no block), and once the sequencer resumes both followers must see it in the same block.
  • TestOpRethKeepsItsOpGethPeerThroughAnAnnouncementBurst: op-geth is the sender. 4112 fresh accounts (the cap plus one fan-out of 16) each submit one transaction with 4096 bytes of zero calldata to op-geth in batch calls while the sequencer is paused. op-geth sends a transaction larger than its txMaxBroadcastSize (4096 bytes) to no peer whole: it announces the hash (eth/handler.go, BroadcastTransactions), so the burst is 4112 NewPooledTransactionHashes entries to op-reth within about a second, which op-reth has to fetch. The test requires op-reth's pool to reach at least 4096 of them, op-reth to keep the same session with op-geth (same remote port in admin_peers; a drop followed by op-geth's static re-dial would show a new one) and to log nothing at error level, and once the sequencer resumes, op-reth's pool to drain down to at most 16 and its head to keep advancing. How many of the burst op-reth fetched and how many each follower still holds afterwards are logged.

What the burst can and cannot show: the number of hashes announced exceeds what op-reth tracks per peer after the bump, but op-reth fetches while the burst is still arriving, and whether its unfetched set from op-geth ever reached 4096 (the point where it drops further announcements) is not observable through RPC. The test therefore proves that op-reth takes a cap-sized announcement burst from an untrusted op-geth peer in its stride, not that the drop path ran. Proving the latter needs reth's announced_hashes_dropped_peer_limit metric, which the mixed runtime does not expose. The drain bound of 16 rather than 0 is for the same cap downstream: op-reth passes the burst to the sequencer, which fetches what op-reth only announced to it, so a cap-sized fetch there can strand up to one fan-out in op-reth's pool; every run below drained to 0.

Each sender has exactly one transaction, so a hash the fetcher drops cannot leave a nonce gap behind; the accounts are funded through a tree of distributors (funding_test.go) because reth holds at most 16 transactions per account (--txpool.max-account-slots, and the runtime passes --txpool.nolocals), which also bounds every funding round to one batch per payer.

Matrix: direction {op-reth to op-geth, op-geth to op-reth} as subtests (the dialing side is the sender, so both dial directions are covered too); CL {op-node, kona-node} through DEVSTACK_L2CL_KIND. Skips on DEVSTACK_L2EL_KIND=op-geth (the sequencer is this tree's op-reth).

Local verification

Debug op-reth built from this stack's Rust tree (identical to develop's), kona-node release binary from develop's rust/kona, -parallel 2, under caffeinate:

cd <worktree> && DEVSTACK_ORCHESTRATOR=sysgo RUST_BINARY_PATH_OP_RETH=<op-reth> RUST_BINARY_PATH_KONA_NODE=<kona-node> \
  go test -count=1 -timeout 40m -parallel 2 -run 'TestOpRethAndOpGethNegotiateTheHighestCommonEthVersion|TestTransactionCrossesTheOpRethOpGethPeeringIntoABlock|TestOpRethKeepsItsOpGethPeerThroughAnAnnouncementBurst' -v ./op-acceptance-tests/tests/op_reth_opgeth_peering/

develop op-reth, op-node (default): all five cells pass, 69s for the package.

--- PASS: TestOpRethKeepsItsOpGethPeerThroughAnAnnouncementBurst (37.56s)
--- PASS: TestTransactionCrossesTheOpRethOpGethPeeringIntoABlock/op-reth_to_op-geth (23.08s)
--- PASS: TestTransactionCrossesTheOpRethOpGethPeeringIntoABlock/op-geth_to_op-reth (23.02s)
--- PASS: TestOpRethAndOpGethNegotiateTheHighestCommonEthVersion/op-reth_dials_op-geth (20.54s)
--- PASS: TestOpRethAndOpGethNegotiateTheHighestCommonEthVersion/op-geth_dials_op-reth (21.94s)

develop op-reth, DEVSTACK_L2CL_KIND=kona-node: all five cells pass, 100s.

--- PASS: TestOpRethKeepsItsOpGethPeerThroughAnAnnouncementBurst (46.95s)
--- PASS: TestTransactionCrossesTheOpRethOpGethPeeringIntoABlock/op-reth_to_op-geth (36.33s)
--- PASS: TestTransactionCrossesTheOpRethOpGethPeeringIntoABlock/op-geth_to_op-reth (33.36s)
--- PASS: TestOpRethAndOpGethNegotiateTheHighestCommonEthVersion/op-reth_dials_op-geth (31.98s)
--- PASS: TestOpRethAndOpGethNegotiateTheHighestCommonEthVersion/op-geth_dials_op-reth (29.74s)

Against #23046's op-reth (reth 2.6.0, built from the bump head 93d9ec9, op-node): all five cells pass, 71s.

--- PASS: TestOpRethKeepsItsOpGethPeerThroughAnAnnouncementBurst (36.82s)
--- PASS: TestTransactionCrossesTheOpRethOpGethPeeringIntoABlock/op-reth_to_op-geth (22.28s)
--- PASS: TestTransactionCrossesTheOpRethOpGethPeeringIntoABlock/op-geth_to_op-reth (28.46s)
--- PASS: TestOpRethAndOpGethNegotiateTheHighestCommonEthVersion/op-reth_dials_op-geth (23.69s)
--- PASS: TestOpRethAndOpGethNegotiateTheHighestCommonEthVersion/op-geth_dials_op-reth (19.67s)

Negotiated versions, as the verification criterion asks: op-geth v1.101702.4-rc.1 advertises eth/69 only (eth/protocols/eth/protocol.go, ProtocolVersions = {ETH69}). develop's op-reth advertises eth/66 to eth/69, the bump's eth/66 to eth/71; the pair negotiates eth/69 on both, in both dial directions, so on the bump op-reth falls back two versions from its own eth/71 to op-geth's highest. The peering is untrusted on both sides (asserted inside PeerWithBasic); op-reth marks op-geth's entry static when op-reth dials, op-geth marks op-reth's static when op-geth dials.

Burst: op-reth's pool held 4107 to 4112 of the 4112 at the first poll after the batches were accepted, on develop and on the bump alike (4105 to 4112 across all runs), the session kept its remote port and eth/69, no op-reth error log, and after the sequencer resumed every burst transaction was included within 6s: 0 left in op-reth and 0 left in op-geth in every run.

Unit: go test ./op-service/txplan/ ./op-devstack/sysgo/ -run 'TestHighest|TestPeerInfo|TestPeerIDMatches|TestPollPeerList|TestPinnedFields' and go test -run TestFunding ./op-acceptance-tests/tests/op_reth_opgeth_peering/ pass; linter/bin/op-golangci-lint run on the changed packages reports no issue.

Observations

  • An isolated op-geth follower under op-node logs Failed to get payload ... err="Unsupported fork" while it is behind the chain: op-node derives attributes from L1 and asks op-geth to seal them, which op-geth refuses under the devnet's Karst-at-genesis config. The follower catches up through the receiver instead, and the same lines appear on develop; noted, not investigated.
  • On this machine, under two concurrent devnets, an op-geth follower that is both gap-filling and peered with two consensus clients once answered its user RPC with dial tcp 127.0.0.1:<port>: i/o timeout for 30s. The pair keeps the sender's consensus client on a single peer (the sequencer's), which never showed it in nine runs since.

Seams

  • sysgo.ConnectP2PBasic / dsl.L2ELNode.PeerWithBasic: a basic peering (static entry on the dialer, nothing on the acceptor, trusted on neither side, asserted after connecting). PeerWith stays trusted on both sides; DisconnectPeerWith undoes PeerWith only.
  • dsl.L2ELNode.VerifyPeeredWith, PeerInfoOf, Peers over sysgo.PeerInfo / ListPeers / ELPeerInfo (caps, negotiated eth version or 0 during the handshake, inbound/static/trusted, session addresses), sysgo.HighestEthVersion and HighestCommonEthVersion.
  • dsl.L2ELNode.SendTransactions (batched eth_sendRawTransaction), WaitForPooledTransaction, TxPoolStatus, TxPoolReached, TxPoolHoldsAtMost, TxPoolDrained.
  • dsl.EOA.PlanOffline, Sign, Submit; txplan.WithStaticChainID. txplan.WithGasFeeCap now drops the default fee-cap computation the way WithGasLimit drops the estimator, so a pinned fee cap stays pinned when the tip cap is pinned after it or a block is fetched (it was silently recomputed from the tip cap and the base fee); the one existing caller, op-acceptance-tests/tests/interop/loadtest/l2.go, now gets the 10,000 gwei cap it asked for, which its budget already accounts for.

go-code-reviewer was run and every finding addressed: the bigint lint, the burst's claim re-scoped as above and its drain bound, both followers gated on the funding blocks before the burst and the transfer, the expected version as the highest common one rather than the smaller maximum, Peers listing handshaking peers too, the session check guarded to the accepting side, per-poll logging in the pool waits, and the setup re-checks, unused Fn variant, duplicated node-info lookups and inaccurate comments removed.

Stacked on #23440, #23438, #23436, #23431, #23427, #23421 and #23418.

Closes #23402

@einar-oplabs einar-oplabs changed the title op-acceptance-tests,op-devstack: cover eth version negotiation and transaction propagation between op-reth and op-geth op-acceptance-tests,op-devstack,op-service: cover eth version negotiation and transaction propagation between op-reth and op-geth Nov 10, 2026
@einar-oplabs
einar-oplabs requested a review from aakoshh November 10, 2026 14:07
@einar-oplabs
einar-oplabs force-pushed the einar/23399-fcu-stale-safe-finalized branch from a3c78e4 to 6cabe52 Compare November 10, 2026 14:53
@einar-oplabs
einar-oplabs force-pushed the einar/23402-eth71-opgeth-peering branch from 3c2a8a0 to 6c242fd Compare November 10, 2026 14:53
@einar-oplabs
einar-oplabs force-pushed the einar/23399-fcu-stale-safe-finalized branch from 6cabe52 to eb73ef7 Compare November 11, 2026 12:28
@einar-oplabs
einar-oplabs force-pushed the einar/23402-eth71-opgeth-peering branch 2 times, most recently from 65ef2e0 to 88cb7c3 Compare November 11, 2026 13:59
@einar-oplabs
einar-oplabs force-pushed the einar/23399-fcu-stale-safe-finalized branch from eb73ef7 to 574c473 Compare November 11, 2026 13:59
@einar-oplabs
einar-oplabs force-pushed the einar/23402-eth71-opgeth-peering branch from 88cb7c3 to a4dc3ae Compare November 11, 2026 15:04
@einar-oplabs
einar-oplabs force-pushed the einar/23399-fcu-stale-safe-finalized branch from 4b951df to a6fb532 Compare November 11, 2026 15:28
@einar-oplabs
einar-oplabs force-pushed the einar/23402-eth71-opgeth-peering branch from a4dc3ae to 8dc8ff7 Compare November 11, 2026 15:28
@einar-oplabs
einar-oplabs force-pushed the einar/23399-fcu-stale-safe-finalized branch from a6fb532 to c4f5482 Compare November 11, 2026 16:28
@einar-oplabs
einar-oplabs force-pushed the einar/23402-eth71-opgeth-peering branch from 8dc8ff7 to d13cbfd Compare November 11, 2026 16:28
@sebastianst

Copy link
Copy Markdown
Member

🤖 Fable says: A note from the op-geth decoupling work (#20257). The new op_reth_opgeth_peering package runs op-geth followers (sysgo.MixedL2ELOpGeth), and the PR extends sysgo/l2_el_opgeth.go.

op-geth is EOL as of Karst, and #21744 removes the in-process op-geth L2 EL from op-devstack/sysgo (l2_el_opgeth.go, MixedL2ELOpGeth, DEVSTACK_L2EL_KIND). It cannot exist after the go.mod flip to upstream go-ethereum (#20266): the same code would silently run upstream geth as the L2 EL. So please don't add new test or CI dependencies on op-geth; they would have to be removed again before the flip.

For this PR that leaves two options: keep the eth-version and propagation coverage between op-reth nodes only, or, if op-reth↔op-geth peering still needs a test for pre-Karst networks, run op-geth as an external pinned binary rather than the in-process sysgo EL. The second is new harness work, so it would be good to settle whether that coverage is still wanted before this leaves draft.

…tion and transaction propagation between op-reth and op-geth

Add an acceptance test package that peers an op-reth and an op-geth
follower as basic peers, requires them to negotiate the highest eth
version both advertise, requires a transaction submitted to either to
reach the other's pool and a block, and hands op-geth twice as many
transactions to announce as op-reth tracks per peer, sent so that
op-reth is offered more than its cap, requiring op-reth to keep the
session, keep syncing, include at least a cap's worth of the burst, and,
once mined transactions have left its pool, account for every burst
transaction it pooled as included or still pooled. Both nodes run pool
limits well above the burst, the funding leaves room for the L1 data
fee, the pair is used only once the sender has caught up with the
sequencer, and the op-geth follower runs a genesis without Karst, which
op-geth v1.101702.4-rc.2 refuses.

The devstack gains basic (untrusted) EL peering, typed admin_peers
lookups including handshaking peers, batch transaction submission,
logged pool waits including waits for pending, pooled and included
transactions and for transactions to leave the pool, offline signing,
and op-geth options on the mixed preset; txplan gains a static chain ID
option, and an explicit fee cap is no longer recomputed when the tip cap
is pinned after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@einar-oplabs
einar-oplabs force-pushed the einar/23399-fcu-stale-safe-finalized branch from c4f5482 to 9ac60ed Compare November 12, 2026 17:25
@einar-oplabs
einar-oplabs force-pushed the einar/23402-eth71-opgeth-peering branch from d13cbfd to 5f79226 Compare November 12, 2026 17:25
@einar-oplabs
einar-oplabs marked this pull request as ready for review November 13, 2026 13:18
@einar-oplabs
einar-oplabs requested a review from a team as a code owner November 13, 2026 13:18
@joshklop joshklop closed this Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants