Skip to content

feat(qa): L3 answer-packet state schemas, surface-contract spec, first evidence folder#576

Merged
HomenShum merged 2 commits into
mainfrom
feat/packet-state-schemas
Jul 17, 2026
Merged

feat(qa): L3 answer-packet state schemas, surface-contract spec, first evidence folder#576
HomenShum merged 2 commits into
mainfrom
feat/packet-state-schemas

Conversation

@HomenShum

@HomenShum HomenShum commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary (corrected post-merge: this PR merged with exactly two commits)

  1. Answer-packet state schemas (L3) — the contract's answerPacket section declares, per response shape × evidence support, which packet fields MUST appear, which must NEVER leak, and which honesty markers are required. chatRuns.contract.test.ts executes every clause through the real runtime functions — 25 tests from one JSON file — and joins the runtime-smoke allowlist. Reversion-proved.
  2. SPEC.md — the one-page portable surface-contract spec for cross-repo convergence.
  3. First dated evidence folder20260717-design-nitpick/ with five schema-valid before/after pairs regenerated from pre-fix(redesign): design nitpick pass — honest dark captures, state copy, icons, row layout #574 sources.

Follow-up coverage/judge/nightly work ships separately in the next PR (its content was briefly described here by mistake while that commit was unmerged).

🤖 Generated with Claude Code

HShuM and others added 2 commits July 17, 2026 14:02
…spec

The contract gains "verifiable meaning": an answerPacket section declares,
per response shape x evidence support, which packet fields MUST appear,
which must NEVER leak (compact rendering is driven by field emptiness, so
a leak is a rendered regression), and which honesty markers are required
(Source needed prefixes, limitation risk rows). A new executor drives
every declaration through the REAL detectRequestedResponseShape +
applyDeterministicResponsePolicy — 8 shapes x supported/unsupported plus
detector agreement, 25 tests from one JSON file — and joins the
runtime-smoke allowlist.

SPEC.md distills the pattern to one portable page (three layers, schema,
verifier contract incl. independent replay, fail-closed hash binding for
served projections) so the second in-portfolio implementation can
converge on one spec instead of two bespoke ones.

Reversion-proved: flipping the bullets lineCount clause fails exactly the
two bullets tests; restored, 25/25.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…after

Five before/after pairs (desktop+mobile x dark+light + the forced
context-miss state) regenerated honestly: before = pre-#574 sources
rebuilt and captured with the FIXED spec so both sides render true dark.
Schema-valid manifest per the ui-contract folder convention.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@HomenShum
HomenShum enabled auto-merge (squash) July 17, 2026 21:02
@vercel

vercel Bot commented Jul 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
nodebench-ai Ready Ready Preview, Comment Jul 17, 2026 9:07pm

Request Review

@HomenShum
HomenShum merged commit 6cc98df into main Jul 17, 2026
15 checks passed
@HomenShum
HomenShum deleted the feat/packet-state-schemas branch July 17, 2026 21:08
@HomenShum HomenShum changed the title feat(qa): L3 answer-packet state schemas, surface-contract spec, first evidence folder feat(qa): L3 state schemas, spec, evidence folder, full coverage + nightly loop Jul 17, 2026
@HomenShum HomenShum changed the title feat(qa): L3 state schemas, spec, evidence folder, full coverage + nightly loop feat(qa): L3 answer-packet state schemas, surface-contract spec, first evidence folder Jul 17, 2026
@github-actions

Copy link
Copy Markdown

Demo: walkthrough of the surfaces this PR changed is available as a workflow artifact (pr-demo-576) at https://github.com/HomenShum/NodeBenchAI/actions/runs/29613717439

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants