Skip to content

kona-node,safedb,op-acceptance-tests: verify parity with op-node #23585

Closed

Description

@einar-oplabs

Part of #23473. Depends on #23475, #23476 and #23477 (the two RPCs and the flag that lets the devstack enable a kona-node safedb).

Why

The RPC answers are the compatibility surface between the two implementations, and the byte layout is not (docs/ai/safedb-review.md). Consumers must not be able to tell an op-node from a kona-node behind optimism_safeHeadAtL1Block and superroot_atTimestamp. The only way to know that is to run both on the same chain and compare every answer, through the scenarios that have historically broken the record: resets, reorgs, restarts and resyncs.

Shape

One acceptance package, op-acceptance-tests/tests/safedb_parity, on a MixedSingleChainRuntime with one sequencer plus two verifiers on the same L1 and L2: an op-node verifier and a kona-node verifier, both with a safedb path. op-node is the reference; its answers are the expected values. op-node itself is not changed by this work.

Scenarios, each ending with VerifySafeHeadDatabaseMatches in both directions over every recorded L1 block, plus a superroot_atTimestamp comparison of RequiredL1 and VerifiedRequiredL1 for each L2 block in the window:

  1. Steady state. Derive through several L1 blocks, including L1 blocks that carry no batch data and span batches whose blocks share a source. Catches a key choice that differs (derivation source versus the block's own L1 origin).
  2. L1 reorg. Trigger an L1 reorg with the existing ReorgTriggered DSL deep enough to unwind batches, then derive past it. Catches a truncation point that differs from the resume point.
  3. CL restart, EL kept. Restart the kona verifier's CL only. Mirrors TestNotTruncateDatabaseOnRestartWithExistingDatabase (kona-node,safedb,op-acceptance-tests: validate restart persistence #22265).
  4. CL sync resume. Mirrors TestPreserveDatabaseOnCLResync.
  5. EL resync. Stop both and wipe the EL. Mirrors TestTruncateDatabaseOnELResync. Until the resume-from-tip follow-up on kona-node: record safe heads into kona-safedb #23474 lands, kona-node clears its history here while op-node keeps and resumes from its tip, so this case documents the known difference and is expected to fail until then.
  6. Disabled. A kona-node without a safedb path answers both RPCs with the same error class op-node does.

The existing safeheaddb_elsync and safeheaddb_clsync packages keep running under DEVSTACK_L2CL_KIND=kona-node (#23479); this package is the direct comparison.

Done

Status evidence: merged #23590 and the earlier local-run report above. The deliberate wrong-key red/green result is now recorded in #23818 (CI job 5812896). EL-resync history preservation remains long-term work in #23658.

Status — 2026-10-29

24-hour soak passed on 2026-10-29: 7,454/7,454 SafeDB queries matched op-node: 7,422 consecutive queries across L1 12,014,303–12,021,724 plus 32 older queries, with zero RPC errors/mismatches and all 5,758 prior answers unchanged. The upper bound was safe on both nodes at 19:51:48 UTC, beyond the 19:50:49 UTC soak endpoint. Kona retained its original PVC and had zero container restarts after the controlled pod recreation. Final evidence.

This adds a final merged-build full-range live RPC comparison plus a controlled CL restart with the EL/PVC retained. It does not rerun the full parity package, deliberately mutate the key, trigger an L1 reorg, or wipe the EL data directory. The later native suite and mutation evidence is recorded below; canonical-tip EL-sync recovery remains deferred in #23658.

Current validation — 2026-10-30

The final implemented five-case package has passed in native CI job 5811528. Mixed-node tests run once in the op-node matrix because they explicitly start both implementations; the Kona matrix intentionally skips the duplicate topology.

A true canonical-tip EL-sync recovery policy and interval-based gap detection remain deferred in #23658/#23659. The deliberate key-mutation criterion is backed by executed negative and positive controls, rather than implementation alone.

Runtime evidence: #23818 (comment)

All six case results are retained in native CI job 5812786 artifacts. The aggregate acceptance job and the remaining required checks have also passed.

Activity

  1. self-assigned this
    on Oct 22, 2026
  2. einar-oplabs commented on Oct 22, 2026

    @einar-oplabs
    ContributorAuthor

    Scenario 5 (EL resync) is not in the first version of the package. Two blockers: kona-node clears its history when it resumes above the recorded tip (the resume-from-tip follow-up on #23474), and the devstack has no primitive to wipe an op-reth data dir between a stop and a start. Scenarios 1 to 4 and 6 pass locally against the branch; the first run against the previous kona-node failed for the expected reasons (method-not-found read as an empty database, disabled database read as empty).

  3. changed the title [-]op-acceptance-tests: op-node and kona-node safedb parity[/-] [+]kona-node,safedb,op-acceptance-tests: verify parity with op-node[/+] on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions