Skip to content

[G3] Cross-repo parity acceptance and contract-drift CI: build burn against relayhistory main, run the corpus parity suite, publish a real-machine parity report #183

Description

@willwashburn

Part of #160 (group 3 — depends on all of group 2 here and burn #557/#558; parallel with #184).

Problem

Once burn depends on the published ai-hist crate, relayhistory can break burn without noticing: a parser change that alters message_id derivation, timestamps, or usage dedup silently changes burn's turn_content_fingerprint and either duplicates or collapses cost rows (burn ledger/fingerprint.rs:97-115). "Tests pass here" is a return value, not an effect; the effect lives in burn's ledger.

Scope

  1. Contract-drift job in ci.yml: check out AgentWorkforce/burn (main), point its ai-hist dependency at this PR's workspace via [patch.crates-io], and run cargo test -p relayburn-sdk --features relayhistory-source --test relayhistory_parity (the suite from burn #557). Required check on PRs touching crates/ai-hist/src/** (after [G1] Merge ai-hist-core and ai-hist-engine into one published ai-hist crate with feature gates, a release pipeline and a semver policy #162 that is the whole library). A failure names the corpus case and the differing table/field.
  2. Reverse job in burn (file the mirror task on burn #562's checklist): burn CI runs its parity suite against the published crate version it pins, so a relayhistory release cannot be adopted with drift.
  3. Real-machine parity report: docs/parity/2026-XX-XX-burn-parity.md produced by running, on at least three developer machines with real Claude/Codex/OpenCode histories (and one with Cursor/Grok), both backends into separate ledgers and burn state parity --against (burn #555). Report: sessions compared, turns compared, exact-match %, every class of difference with root cause and the issue that fixes it, token totals per model and per harness for both, cost delta. Follow CLAUDE.md: confirm against effect (query both ledgers for specific message_ids), include a positive control (a deliberately altered turn shows up as a diff).
  4. Corpus completeness: every difference class found on real machines that the corpus did not cover gets a new anonymized fixture in tests/fixtures/ (scrub content; keep structure) so it cannot regress.
  5. Gate: [G3] Retire duplicated harness knowledge: docs, sibling-repo runbooks, burn collector transfer, and the "one owner" guardrail test #184 and burn #562 (cutover) may not start until this report shows 100% identity-vector match for Claude and Codex and every remaining difference is documented as intended.

Acceptance

  • The drift job is a required status check and has caught at least one injected regression in a draft PR (e.g. change Codex token_json delta → parity fails).
  • The parity report is merged with the numbers above and links to fixtures added.

Out of scope

Fixing what the report finds (goes back to the owning issue).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions