Skip to content

Payload parity harness: guard store_state payloads the way the frame fixtures guard frames #246

Description

@kevinthelago

Part of #234. This is the durable fix — every other child in the umbrella is catch-up that will rot again without it.

The gap

The frame contract is guarded: both repos hold the same tunnelProtocol.fixtures.json (committed blob 46cde9fdea0e53f20271ac8e38a756262c6e47c4 in both), and tunnelProtocol.fixtures.test.ts decodes every fixture into our typed models and deep-equals back, failing on drift in either direction.

The payload contract is not guarded at all. store_state is {domain, rev, json} where json is an opaque serialized string, so every per-domain payload shape is invisible to that test. That is how #237 and #238 accumulated silently while 351 tests stayed green.

Root cause — and why the runtime must NOT change

Our selectors are deliberately tolerant. src/lib/mirror/payload.ts says so in its header: "Selectors must never throw on a missing, extra, or mistyped field." selectSecurityView returns EMPTY_SECURITY_VIEW; selectGlance returns undefined; glanceL0Input defaults status ?? 'idle'.

That posture is correct for a phone in the field and wrong as a test oracle. Running a tolerant selector over a drifted payload produces a blank page, not a failure. The harness must supply a separate strict oracle. Tolerant at runtime, brutal in CI.

Prerequisite that blocks everything else

Mobile CI does not run the tests. .github/workflows/ci.yml has three jobs — install, typecheck, export — and no npm test anywhere. The 351 tests, including the frame parity test this design extends, are developer-local only. It also triggers on main only, so per the branch flow (feature → develop → main) it would never run on a feature PR.

Everything below is decorative until that is fixed.

Design

Artifact

  • Desktop: src/features/tunnel/lib/storePayloads.fixtures.json
  • Mobile: src/lib/tunnel/storePayloads.fixtures.json

Same directories and naming as each repo's existing frame fixtures, so one sync loop covers both files. Structure: { $comment, domains: { <domain>: <payload> }, variants: { … } }.

variants plays the role auth_no_fcm / auth_ok_pre_grant play today — it pins optionality and nullability so a field cannot quietly become required (glance_l0 with drill: null, skills_no_lessons, blueprints_no_team).

Fixture size discipline is a rule, not a nicety: 2–3 rows per collection, never more; target under 15 KB. Cap behaviour (SECURITY_AUDIT_CAP = 200, AUTOMATION_RUNS_CAP = 10) stays asserted desktop-side in storeProjections.test.ts. Beyond that size the deepEqual diffs stop being readable, which destroys the harness's main practical virtue.

Desktop side — fixtures generated BY the real builders

New: src/features/tunnel/lib/storeProjections.fixtures.ts — extract the typed inline consts that already exist in storeProjections.test.ts (const projects: ProjectLite[], const profile: AgentProfile, const fleet: FleetPlan, …) into an exported PROJECTION_INPUTS, each entry satisfies its real domain model.

This is the hinge of the whole design. A rename in ProjectLite / AgentProfile / AuditRecord becomes a compile error at the fixture file, which forces a regeneration, which fails mobile.

New: storePayloads.fixtures.test.ts (vitest) with four assertions:

  1. regenerates the committed file when UPDATE_STORE_FIXTURES=1 (guarded to never self-heal in CI)
  2. the committed fixtures equal JSON.parse(JSON.stringify(builderOutput)) — round-tripped, because JSON.stringify drops undefined-valued keys and that is what actually goes on the wire
  3. every registered STORE_DOMAIN has a payload fixture — a coverage-gap guard
  4. the variants pin the nullable fields

Plus "fixtures:store" in package.json. Desktop CI already runs typecheck and tests, so no new CI wiring is needed there.

Mobile side — two layers, both load-bearing

Shared helper: src/lib/tunnel/fixtureDecode.ts — lift str/num/bool/arr/copyOptStr/copyOptNum verbatim out of tunnelProtocol.fixtures.test.ts so both tests import one definition. (Not a .test.ts, so the npm test glob skips it.)

Layer A — strict re-encode. Per domain, a decoder copies only the fields our page model reads, then deep-equals back against the raw fixture:

Drift What fires
fixture has a field we don't know never copied, so it is an extra key in rawdeepEqual fails with the field in the diff
we require a field the fixture lacks str(o,'action') / num(r,'at') throws field "action" must be a string
field changed JS type both fire at once
desktop publishes a domain we don't model no mobile decoder for domain "security"
our vocabulary constant is stale mobile STORE_DOMAINS is out of sync

Layer B — selector smoke. Layer A has one hole, and it is exactly the hole the glance bug fell through: we read status optionally, so a faithful decoder uses copyOptStr, which does not fail on absence. Had desktop merely deleted status without adding anything, Layer A alone would pass. role, faults, name, and kind are all optional-read too.

Layer B closes it by asserting the canonical payload produces a non-degenerate view — that some project is not idle, that audit[0].action is not the em-dash fallback, that at is not null.

The invariant that gives Layer B teeth, to be stated in the fixture header and enforced: no fixture field may carry a value equal to a mobile selector's fallback. If the fixture ships a default-looking value, the smoke test cannot distinguish "read correctly" from "fell back".

Layer A tells you which field drifted. Layer B tells you the page is blank. Ship both, and say so in the test header so nobody later "simplifies" Layer B away.

Would it have caught the two known bugs?

Glance statushealth+activity: desktop-side, satisfies ProjectLite[] makes it a typecheck failure at authoring time, forcing a regeneration. Mobile-side, Layer A reports + health + reason + activity as uncopied extras, and Layer B reports "EVERY project fell back to status idle". Caught twice — but note the delete-only variant would be caught by Layer B alone, which is why it is mandatory.

Security {ts,pane,toolName,target} vs {at,action,detail,actor}: four assertions fire, in order — the STORE_DOMAINS mismatch (a live bug today, before any payload work), then arr(o,'assignments') throwing, then num(r,'at') throwing, then Layer B's em-dash check.

A specific hazard this surfaces: securityView.ts:72 reads actor with the alias list ['actor','session','pane','role']. Desktop rows carry pane — so actor would resolve and look correct. Those speculative alias lists are drift camouflage: written before the desktop shape existed, they turn a hard failure into a plausible-looking half-render. #237 must delete them, and Layer B must assert per-field, not just !empty — because !empty passes on a payload where only actor resolved.

Rejected alternatives

Alternative Why not
Generate types from a shared JSON Schema Adds codegen and a schema language to two repos that have neither, and makes the schema a third artifact to sync. Fatal flaw: additionalProperties: false gives direction 1 only — nothing forces mobile to actually read a field. We could ignore health forever and validate green.
JSON Schema validation (ajv) Same read-vs-validate flaw plus a runtime dep. The decode-and-deep-equal idiom needs zero dependencies and is already proven in this exact directory.
Publish @bsc/tunnel-contract npm package Trades file skew for version skew — we pin ^1.2.0 and drift silently with more ceremony. Needs a publish step per contract change and a private registry that does not exist. A submodule is the same plus Windows pain.
Runtime strict parsing Actively harmful — turns a cosmetic desktop addition into a blank page on a user's phone. The tolerance is correct in production.
Record real frames from a live tunnel Needs a paired desktop plus relay in CI, nondeterministic, cannot run in this repo at all.

Honest tradeoffs: the mobile decoder is a second hand-maintained copy of the mobile model — that duplication is the mechanism, and there is no way to get "fails on unknown field" from a selector whose contract is "never throw". Sync stays manual; CI detects staleness, it does not fix it. The Rust relay stores json opaquely and has no payload model to drift, so it correctly takes no part in this — the frame fixtures already carry its obligation.

CI enforcement

.gitattributes, both repos, land first:

* text=auto eol=lf
*.fixtures.json text eol=lf -diff=json

then git add --renormalize . once per side. The fixtures are already LF in git so this produces no blob change — it fixes the working tree, making the header's byte-identical claim literally true. Today mobile's Windows checkout is 5474 bytes vs the desktop's 5376 (98 CRLFs), so any working-tree hash comparison false-fails.

scripts/check-contract-sync.sh, identical in both repos, comparing by git blob hash (git hash-object locally vs the contents API's .sha) — content-addressed, so EOL- and checkout-agnostic, and no clone needed. Hard-fails, with a [contract-pending] PR-title escape hatch for the first half of a coordinated pair (mirroring the desktop's existing [ci-all] idiom). Needs a fine-grained CONTRACT_SYNC_TOKEN PAT — GITHUB_TOKEN is scoped to the current repo and cannot read a private peer.

A soft continue-on-error warning on the one PR that most needs coordination is exactly how drift accumulates. Hard-fail.

Rollout

Phase 0 — four small independent steps, all landable now:

Phase 1 — desktop authors the canonical file:

  • D2: extract PROJECTION_INPUTS into storeProjections.fixtures.ts. Pure refactor, highest-value single step — it delivers the compile-error tripwire even if nothing else ships.
  • D3: add the fixtures JSON, its test, and npm run fixtures:store. The coverage assertion will immediately flag that plan has no builder in storeProjections.ts (it is published by the planner instead). Resolve by moving/adding buildPlanPayload or committing an explicit UNPROJECTED_DOMAINS exemption with a linked issue — do not silently drop the domain.

Phase 2 — mobile consumes it:

Phase 3 — enforcement last, so it does not fire on the PRs building the harness:

  • D4 + M5: check-contract-sync.sh and the contract-sync job in both repos; create CONTRACT_SYNC_TOKEN.

Steady state: desktop PR changes a builder → fixtures:store → land with [contract-pending] → mobile PR copies the file, updates decoder and selector, lands green → the next desktop PR is green again.

Cost

Per new domain: ~15 min desktop (one typed PROJECTION_INPUTS entry plus regeneration — no hand-written JSON, and the coverage test will not let you forget), ~30–45 min mobile (one decoder mirroring the selector, one smoke test, one map entry).

Per change to an existing domain: one coordinated PR pair, which is already the norm for frame changes, plus a ~5s hash comparison.

The honest ongoing tax: the mobile decoder is a second copy of the mobile model, so when a selector legitimately starts reading a new optional field you edit two places — realistically ~2 extra edits per domain per year. Bounded by two rules: decoders cover only fields the page renders (with a per-domain IGNORED_FIELDS set that documents deliberate ignores rather than hiding them), and the fixture size cap.

Net: about one extra hour per domain, in exchange for a hard failure with a named field in the message, for the exact class of bug that currently ships 351 green tests alongside a blank page.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions