Repository navigation
Description
Part of #23473. Depends on #23475, #23476 and #23477 (the two RPCs and the flag that lets the devstack enable a kona-node safedb).
Why
The RPC answers are the compatibility surface between the two implementations, and the byte layout is not (docs/ai/safedb-review.md). Consumers must not be able to tell an op-node from a kona-node behind optimism_safeHeadAtL1Block and superroot_atTimestamp. The only way to know that is to run both on the same chain and compare every answer, through the scenarios that have historically broken the record: resets, reorgs, restarts and resyncs.
Shape
One acceptance package, op-acceptance-tests/tests/safedb_parity, on a MixedSingleChainRuntime with one sequencer plus two verifiers on the same L1 and L2: an op-node verifier and a kona-node verifier, both with a safedb path. op-node is the reference; its answers are the expected values. op-node itself is not changed by this work.
Scenarios, each ending with VerifySafeHeadDatabaseMatches in both directions over every recorded L1 block, plus a superroot_atTimestamp comparison of RequiredL1 and VerifiedRequiredL1 for each L2 block in the window:
- Steady state. Derive through several L1 blocks, including L1 blocks that carry no batch data and span batches whose blocks share a source. Catches a key choice that differs (derivation source versus the block's own L1 origin).
- L1 reorg. Trigger an L1 reorg with the existing
ReorgTriggeredDSL deep enough to unwind batches, then derive past it. Catches a truncation point that differs from the resume point. - CL restart, EL kept. Restart the kona verifier's CL only. Mirrors
TestNotTruncateDatabaseOnRestartWithExistingDatabase(kona-node,safedb,op-acceptance-tests: validate restart persistence #22265). - CL sync resume. Mirrors
TestPreserveDatabaseOnCLResync. - EL resync. Stop both and wipe the EL. Mirrors
TestTruncateDatabaseOnELResync. Until the resume-from-tip follow-up on kona-node: record safe heads into kona-safedb #23474 lands, kona-node clears its history here while op-node keeps and resumes from its tip, so this case documents the known difference and is expected to fail until then. - Disabled. A kona-node without a safedb path answers both RPCs with the same error class op-node does.
The existing safeheaddb_elsync and safeheaddb_clsync packages keep running under DEVSTACK_L2CL_KIND=kona-node (#23479); this package is the direct comparison.
Done
- The
safedb_paritypackage exists with steady-state, L1-reorg, CL-restart, CL+EL-restart, and disabled-database cases (kona-node,op-acceptance-tests: serve the safe head history and compare it with op-node #23590); these were reported passing locally before the final integration revisions. - All six implemented parity cases pass on op-acceptance-tests,safedb: validate parity after Kona resync #23818 commit
53f6a13500in native CI job 5812786: steady state, L1 reorg, Kona CL restart, Kona CL+EL restart, wiped-EL resync, and disabled SafeDB. This does not establish the deferred canonical-tip EL-sync recovery policy. - A deliberate key mix-up (
l1Head = safeHead.L1Originin the op-node reference writer) makes scenario 1 fail at L1 block 5 with the expected parity mismatch; both positive tests pass after exact writer restoration (CI job 5812896). - Implement and pass the canonical-tip EL-sync recovery scenario once kona-node,safedb,gap-detection: resume derivation from the canonical SafeDB tip after EL sync #23658 and persistent EL-data-directory wipe support land. op-acceptance-tests,safedb: validate parity after Kona resync #23818 validates parity after an in-memory op-geth EL is recreated, but does not establish this separate recovery policy.
Status evidence: merged #23590 and the earlier local-run report above. The deliberate wrong-key red/green result is now recorded in #23818 (CI job 5812896). EL-resync history preservation remains long-term work in #23658.
Status — 2026-10-29
24-hour soak passed on 2026-10-29: 7,454/7,454 SafeDB queries matched op-node: 7,422 consecutive queries across L1 12,014,303–12,021,724 plus 32 older queries, with zero RPC errors/mismatches and all 5,758 prior answers unchanged. The upper bound was safe on both nodes at 19:51:48 UTC, beyond the 19:50:49 UTC soak endpoint. Kona retained its original PVC and had zero container restarts after the controlled pod recreation. Final evidence.
This adds a final merged-build full-range live RPC comparison plus a controlled CL restart with the EL/PVC retained. It does not rerun the full parity package, deliberately mutate the key, trigger an L1 reorg, or wipe the EL data directory. The later native suite and mutation evidence is recorded below; canonical-tip EL-sync recovery remains deferred in #23658.
Current validation — 2026-10-30
The final implemented five-case package has passed in native CI job 5811528. Mixed-node tests run once in the op-node matrix because they explicitly start both implementations; the Kona matrix intentionally skips the duplicate topology.
-
Add a direct mixed-node wiped-EL resync test and compile/review it: ready-for-review op-acceptance-tests,safedb: validate parity after Kona resync #23818. The existing five cases retain op-reth; the new fixture uses op-geth, asserts the restarted EL is at genesis, retains Kona's SafeDB path, and checks both RPCs after catch-up.
-
The new mixed-node resync test passes twice in native CI job 5812896, preserving the recorded history floor and matching both RPCs after catch-up (op-acceptance-tests,safedb: validate parity after Kona resync #23818).
The isolated wrong-key control has the required fail/pass result, and the original writer SHA-256 matches after restoration. No intentional mutation is included in op-acceptance-tests,safedb: validate parity after Kona resync #23818. -
Normal required CI passes for op-acceptance-tests,safedb: validate parity after Kona resync #23818: 52 passed checks and 3 skipped, zero failures/pending.
-
Review and merge op-acceptance-tests,safedb: validate parity after Kona resync #23818.
A true canonical-tip EL-sync recovery policy and interval-based gap detection remain deferred in #23658/#23659. The deliberate key-mutation criterion is backed by executed negative and positive controls, rather than implementation alone.
Runtime evidence: #23818 (comment)
All six case results are retained in native CI job 5812786 artifacts. The aggregate acceptance job and the remaining required checks have also passed.
Activity
einar-oplabs commentedon Oct 22, 2026
Scenario 5 (EL resync) is not in the first version of the package. Two blockers: kona-node clears its history when it resumes above the recorded tip (the resume-from-tip follow-up on #23474), and the devstack has no primitive to wipe an op-reth data dir between a stop and a start. Scenarios 1 to 4 and 6 pass locally against the branch; the first run against the previous kona-node failed for the expected reasons (method-not-found read as an empty database, disabled database read as empty).
[-]op-acceptance-tests: op-node and kona-node safedb parity[/-][+]kona-node,safedb,op-acceptance-tests: verify parity with op-node[/+]