Repository navigation
op-devstack,op-acceptance-tests: skip presets that start no kona-node in the kona acceptance job - #24245
Merged
Conversation
nonsense
marked this pull request as ready for review
November 18, 2026 12:32
sebastianst
reviewed
Nov 19, 2026
…e in the kona acceptance job The supernode runtimes and the interop runtimes without a supernode start only op-node and op-supernode CLs, whatever DEVSTACK_L2CL_KIND says. Under kona-node their tests repeated the op-node acceptance job exactly: 158 of 289 top-level tests and about 10,200 of 15,400 test-seconds of the kona job on develop. Their sysgo constructors now skip the test under kona-node before starting anything. Tests that start both CLs explicitly still run in both jobs, and SkipOnKonaNode stays the list of real kona-node gaps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the presets that start no kona-node skipped under kona-node, the kona job has about 5,100 test-seconds left, a third of before. Its wall time is bounded by its longest test, about 400 s, so 2 nodes running 16 tests at once finish no later than 4 did. The scheduled shipped-binaries copy of the job inherits the change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move the kona-node skip from six sysgo runtime constructors to the first line of the 14 preset constructors whose runtimes start only op-node and op-supernode CLs (NewSimpleInterop, NewTwoL2SupernodeInterop, NewSingleChainInteropNoSupernode and the rest). A reader of a preset now sees that its tests don't run in the kona-node job. Nothing outside op-devstack/presets calls those runtimes, so the same tests skip. The test lists both sides: the 14 presets must skip under kona-node, and the 11 presets that start the selected CL must not. A probe T stops each preset at its first temp dir, so neither case starts a system. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The two preset lists were written by hand, so a new preset on neither list passed without notice: an op-node-only preset that forgot the skip would repeat the op-node job under kona-node. TestEveryPresetIsClassified parses the package and requires every exported New* func whose first parameter is a devtest.T to be on exactly one of the op-node-only, kona-capable or not-a-preset lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nonsense
force-pushed
the
ae/acceptance-kona-skip-opnode-only-presets
branch
from
November 19, 2026 10:23
0db7942 to
372b117
Compare
sebastianst
approved these changes
Nov 20, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Nov 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The kona-node acceptance job (
memory-all-kona-op-reth-fusaka) repeats most of the op-node job unchanged.DEVSTACK_L2CL_KIND=kona-nodeselects the CL only in the single-chain presets. The supernode runtimes and the interop runtimes without a supernode start op-node and op-supernode CLs whatever the variable says. On develop that is 158 of 289 top-level tests and about 10,200 of 15,400 test-seconds in the kona job. Their kona-job timings match the op-node job's (10,243 s vs 10,316 s). Each memory-all job costs about 3,000 credits per run, and there are about 2,560 runs a month.How the duplication started. Until #19787, both memory-all jobs ran only the
basegate. That was a list of single-chain packages (base,ecotone,fjord,isthmus,jovian) with no interop, supernode or proofs tests. #19787 removed gates so that every implementation variant runs all acceptance tests. The interop and supernode runtimes never readDEVSTACK_L2CL_KIND, so that change also made the kona job re-run every one of their tests on op-node. Each interop and supernode test added since has grown the duplication. This PR reverses that part of #19787 and keeps the rest: the kona job still runs every test that starts a kona-node.With this PR, the 14 presets built on those runtimes (
NewSimpleInterop,NewTwoL2SupernodeInterop,NewSingleChainInteropNoSupernodeand the others) skip the test under kona-node before they start anything. The skip is the first line of each of those preset constructors (skipOpNodeOnlyPresetOnKonaNode), so the preset shows which tests don't run in the kona job. The kona job now runs on 2 nodes instead of 4. The remaining work, about 5,100 test-seconds, is bounded by the longest kona test (TestExecuteStep_CannonKona, about 400 s), so 2 nodes running 16 tests at once should take no longer than 4 did. The estimated saving is 4–5M credits a month. Skipping without the node cut would save about 1M. The op-node job is unchanged, and so is coverage: every skipped run starts the same processes as its op-node twin.Policy change for the acceptance-test owners to sign off.
docs/ai/acceptance-tests.mdsaid that a test pinning every node's CL kind should run in both jobs rather than skip one to save CI time. That rule came from #23856, which removedSkipOnKonaNodefrom the 7 tests that start both CLs and accepted running them twice. It stays for the tests that start both CLs explicitly (safedb_parity,derivation/confirmations), which still run in both jobs. Whether they should also skip one job is left to the owners. The new rule covers presets whose runtime starts no kona-node: they skip themselves under kona-node in the preset constructors, so new interop and supernode tests need no per-test handling, andSkipOnKonaNodekeeps listing only real kona-node gaps. When kona-node or lokahi can run these topologies, remove theskipOpNodeOnlyPresetOnKonaNodecalls from those presets. No code outsideop-devstack/presetscalls these runtimes directly.Not skipped, deliberately:
newDefaultSingleChainWorldwithPresetConfig.InteropAtGenesisalso gives the sequencer an op-node. No preset accepts that option today, and the multi-node variants built on the same runtime add verifiers that followDEVSTACK_L2CL_KIND. A skip there could drop kona coverage.Side effects:
op-up --interopwithDEVSTACK_L2CL_KIND=kona-nodenow stops with the skip message, because op-up treats a skip as an error. Before, it silently started op-node CLs.DEVSTACK_L2CL_KIND=kona-nodeskip them too.op-devstack/presets/op_node_only_test.golists both sides: the 14 presets must skip under kona-node, and the 11 presets that start the selected CL must not. A probe stops each preset at its first temp dir, so neither check starts a system. Removing the call from one preset, or adding it toNewMinimal, fails the test.Review findings that were not applied:
🤖 Generated with Claude Code