Skip to content

op-core: differential tests between Go and Rust on generated inputs #24250

Open

Description

@sebastianst

The OP Stack code in op-core is pinned by golden vectors: literal values that the Rust stack computes too. A golden vector covers the inputs someone thought of. This issue proposes to also compare the Go functions with their Rust equivalents on generated inputs, and records an analysis of where that pays off. Nothing here blocks the go.mod flip (#20266).

What a prototype showed

A throwaway prototype compared the fee functions and the DA footprint of raw transaction bytes between op-core/fees and rust/op-revm over a generated corpus.

  • The two sides agreed on every case.
  • Two bugs planted in the Go FastLZ code (a changed match-length constant and an off-by-one in a distance bound) passed every existing test, golden vectors included. The corpus caught both, but only after the generator was widened to large transactions with repeated blocks. What the generator produces decides what the diff finds.
  • Cost is small: the Rust side builds in well under a minute from cold, and a large corpus runs in seconds on both sides.

Design

Follow ops/scripts/test-interop-deposits-diff.sh: a Go dump command and a Rust dump binary read the same corpus and write comparable output, and a script diffs it. The corpus is generated from a fixed seed, not checked in, so the check is deterministic.

Alternatives considered:

  • A vector file generated by one side and asserted by the other. No cross-language build, and it runs in plain go test and cargo test on every PR. It is golden vectors at a larger scale and a good complement: a few hundred generated cases checked into both suites.
  • A Go fuzz target that calls Rust through cgo or a helper process. The strongest bug finder, but it puts a Rust build back into go test, which the interop-deposits script was written to avoid, and its results vary between runs, so it could not block a PR.
  • Separate property tests with a shared seed corpus. They cannot see a divergence between the two sides.

Where it pays off

Go Rust Value
Span-batch transaction fields (op-node/rollup/derive) kona batch/transactions.rs, batch/raw.rs High. The batcher controls these bytes and they are consensus-critical. docs/ai/derivation.md asks for an accept-set diff over a generated corpus, but no harness is committed.
DepositTx and PostExecTx decoding and hashes (op-core/types) op-alloy transaction/deposit.rs, post_exec.rs Yes. It compares what each side accepts, which golden vectors of valid values cannot.
L1 cost, FastLZ size, operator fee, DA footprint per transaction (op-core/fees) op-revm l1block.rs, transaction.rs Yes; the prototype covers it. These Go functions only run in tooling, so the diff protects tooling, not consensus.
Holocene and Jovian extraData (op-core/eip1559) op-alloy eip1559.rs Yes. The accept-set is small and cheap to diff.
Receipt encodings and receipts root (op-core/types) op-alloy receipt envelope, op-reth and kona root computation Moderate. The generator has to emit fork and receipt-version pairs that fit together.
Genesis block hash, chain-config JSON, RPC JSON op-reth chainspec, op-alloy RPC types Low. The inputs are configuration; per-fork golden vectors and the op-reth fixtures are enough.

What a harness has to handle

  • Inputs must stay within what both sides represent. With fee parameters drawn from the full uint256 range, a large share of cases differ, because the Rust side saturates and the Go side uses big integers. No chain reaches those values, so the generator stays within the wire types.
  • The function boundaries differ. The Go L1 cost functions rely on their callers to skip deposits; the Rust side skips them inside the executor. Each dump command has to copy its caller's dispatch.
  • Known differences need explicit rules. The two sides differ on purpose in a few places, e.g. op-alloy parses a post-exec payload when it decodes the transaction and op-core keeps it opaque, and the RPC JSON key sets differ (op-core, op-alloy: align the RPC JSON of deposit transactions and receipts with the specs #24223). An exclusion belongs on the input side of the generator, as a narrow rule with a link to its issue, never as a filter on the output. The run prints how many cases each rule excluded, so a change in that count is visible.
  • CI routing. The existing diff runs as interop-deposits-go-rust-diff in the Rust CI workflow, which a PR only triggers when it changes rust/, op-e2e/, op-core/nuts/, .circleci/ or mise.toml. A Go-only change to a compared function would skip the diff until it runs on develop. This applies to the existing interop-deposits diff too.

Steps

  1. Fees and DA footprint: a Go dump command, a Rust dump binary and a diff script, hosted in the job of the interop-deposits diff, with a generator that reaches large and repetitive inputs.
  2. Make Go changes to the compared packages trigger the job.
  3. Deposit and post-exec decoding and hashes.
  4. Span batches. This is the largest step: it needs a structured generator and a rollup config on the kona side.
  5. Optionally, check a few hundred generated cases into both suites as a vector file.

To decide

  • Should the diff block PRs? If so, should Go-path changes trigger the whole Rust CI workflow or a smaller dedicated one?
  • Start with the fee functions, which are cheap and already prototyped, or go straight to the consensus decoders?
  • A fixed seed for the PR check, plus a rotating seed on a schedule that does not block and logs its seed?
  • Which side generates checked-in vectors? Disagreements are decided from the specs either way.

🤖 Co-created with Claude Opus 5.5 and Claude Fable 5.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions