Skip to content

Run the working-tree snapshot test alone, after every other test file - #146

Merged
NSchatz merged 1 commit into
mainfrom
harness/1-fix-red-main-ccda-release
Oct 19, 2026
Merged

NSchatz merged 1 commit into
mainfrom
harness/1-fix-red-main-ccda-release

Conversation

@NSchatz

@NSchatz NSchatz commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Why main is red

The Release workflow's Verify step on d906bda (run 37705238602) failed in test/conformance/working-tree-is-clean.test.ts:

AssertionError: expected Set{} to deeply equal Set{ '.cosyte-usage-stdout/' }

That test compares git status of the real working tree before and after a conformance run. The .cosyte-usage-stdout/ directory belongs to a different file: the README usage case in test/docs-content.test.ts writes it and removes it in afterAll. Vitest runs test files in parallel, so the "before" snapshot saw the directory and the "after" one did not. Two more files write into the tree briefly in the same way: the tsup build that docs-content spawns (tsup.config.bundled_<hash>.mjs at the root) and the phi-scan override-gate case (it appends to the tracked phi-scan-overrides.md, then restores it). The Release runs before d906bda passed on the same test, so this is a race, not a regression.

The fix

vitest.config.ts now has two projects:

  • suite: every test file except the snapshot test, sequence.groupOrder: 0, still in parallel.
  • working-tree: only test/conformance/working-tree-is-clean.test.ts, sequence.groupOrder: 1. Vitest starts this group only after the first group has finished, so nothing else writes to the tree while the snapshot is taken.

The assertion is unchanged, and no test is skipped, deleted or loosened. The include globs move onto the first project because extends: true concatenates include arrays. Set at the root, they pulled the whole suite into the second project, and a first draft of this change caught that. documentation/agent-notes.md gets a short paragraph in the AC-9 section that points at the config.

No changeset: nothing under src/ changes, and no published surface moves.

Verification

  • Reproduction: a temporary probe test (not committed) holds a path in the repo root until the conformance run starts writing. On main it fails AC-9 with the same shape as CI. With this change the probe never overlaps the run, and AC-9 passes.
  • Full run, JSON reporter: the snapshot file starts 1.3 s after the last other file ends. 43 files, 1369 passed, 1 skipped, the same counts as before.
  • Release Verify sequence locally: pnpm typecheck, lint, test, build, attw, all green. Also green: test:coverage (all directory gates), format:check, check:no-emdash, check:no-internal-refs, check:agent-notes, phi-scan.

Harness-Task: 1

🤖 Generated with Claude Code

The release Verify step on d906bda went red in
test/conformance/working-tree-is-clean.test.ts: its "before" git status
held .cosyte-usage-stdout/ and its "after" one did not. That directory is
written and removed by the README usage case in test/docs-content.test.ts,
which Vitest ran in parallel with the snapshot. Two more files write into
the tree for a moment the same way: the tsup build docs-content spawns
(tsup.config.bundled_<hash>.mjs) and the phi-scan override-gate case
(phi-scan-overrides.md). Earlier Release runs passed on the same test, so
this was a race, not a regression.

vitest.config.ts now splits the suite into two projects. The snapshot
file has its own project with a later sequence.groupOrder, so it starts
only after the rest of the suite has finished; every other file still
runs in parallel. The assertion is unchanged. The include globs move onto
the first project because extends: true concatenates include arrays.

Measured: a probe test that holds a transient path in the repo root
until the conformance run starts reproduces the CI failure on main and
never overlaps the run with this change. In a full run the snapshot file
starts 1.3 s after the last other file ends; 43 files, 1369 passed,
1 skipped, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NSchatz
NSchatz merged commit 8d7de6f into main Oct 19, 2026
11 checks passed
@NSchatz
NSchatz deleted the harness/1-fix-red-main-ccda-release branch October 19, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant