feat: make chat the single NodeBench workspace#561
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
PR size advisoryThis PR adds 651 lines of substantive change. CONTRIBUTING.md defines a soft limit of ~400 LOC. If the PR is genuinely cohesive (e.g. an architecture map, a generated migration, a deletion of a dead module), no action is needed. Otherwise consider:
This is advisory — it does not block the merge. |
HomenShum
changed the title
Vercel preflight failed:
|
❌ Dogfood Visual QA Gate: FAILED
ArtifactsDownload the Generated by Dogfood QA Gate |
Tier B regression failed against previewPreview URL: https://nodebench-ratb744hk-hshum2018-gmailcoms-projects.vercel.app Likely a kit-parity DOM regression. Check the Playwright report artifact attached to the run for the failing selector + screenshot. This check would have caught all 4 A9 sessionRows iterations as a single PR — that's why it exists. |
Tier B regression failed against previewPreview URL: https://nodebench-g59cstvw7-hshum2018-gmailcoms-projects.vercel.app Likely a kit-parity DOM regression. Check the Playwright report artifact attached to the run for the failing selector + screenshot. This check would have caught all 4 A9 sessionRows iterations as a single PR — that's why it exists. |
|
Demo: walkthrough of the surfaces this PR changed is available as a workflow artifact ( |
detectRequestedResponseShape matched "return only the title" but not "return only its title": the determiner group listed articles and no possessives, so the possessive phrasing fell through to the five-section memo and silently overrode an explicit user output constraint on a paid run. The determiner set now lives in one TITLE_DETERMINER constant shared by every title pattern. It was previously duplicated across two regexes, which is how the two could disagree in the first place. Found by re-verifying the 2026-07-16 production audit against main instead of trusting it. Four of its five P1s were already fixed by #550/#561/#564 within hours of the run; only this one survived. The suite stayed green because every title case it asserted used the one phrasing that worked, so the new tests use the audit's verbatim production prompts and cover the negative cases the wider pattern could now over-match. Both test files are added to the CI runtime-smoke allowlist, which is an explicit file list rather than a full-suite run. Without that, the regression guard would never execute in CI and would repeat the same blind spot one level up. Also pins DEFAULT_TIERS to the runtime's modelForTier with a parity test. The two were hand-maintained mirrors with no test binding them; drift would make the paid preflight advertise a model the runtime never runs. The composer header comment still claimed "provider names appear only in the trace", the opposite of the disclosure contract shipped in #550. Co-authored-by: hshum <hshum@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2 participants
Outcome
/redesign/chatthe single main-site NodeBench decision workspaceVerification
npx tsc --noEmit --pretty falseRelease
Ready for the repository P0 CI-gated squash merge. Do not bypass required checks.