[P0] Replace the Windows worktree artifact-routing wall-clock timeout with deterministic phase evidence

[mohanagy]

Parent: #654
Target branch: next
Depends on: #689 and #690 completed
Blocks: #654 completion, merge of #681 and #680, and start of #657
Milestone: v0.33.0-beta.1 — Trust Foundation

Outcome

Make the linked-worktree artifact-routing test prove correctness deterministically on Windows and the other supported platforms, without relying on one fixed 20-second wall-clock budget around Git repository setup, worktree creation, graph generation, assertions, and cleanup.

The issue is limited to the test/integration contract and any minimal testability seam required to observe its phases. It must not change Madar's graph semantics or artifact-routing behavior unless a separate product defect is conclusively reproduced.

Confirmed evidence

Protected matrix attempt 3 for frozen commit:

b1300f8fcc2758404abc5e6064433c4d8b2ab40b

failed in:

• workflow run: 31711572439
• run attempt: 3
• lane: windows-latest, Node 22
• job: 94523609780
• runner image: Windows Server 2025
• Node: 22.23.2
• npm: 10.9.8

Exact failure:

tests/unit/workspace.test.ts > worktree artifact routing > keeps a linked worktree graph outside the source checkout and isolated from the primary checkout
Error: Test timed out in 20000ms
Location: tests/unit/workspace.test.ts:46:3
Observed file duration: approximately 21.3 s

The guarded raw log contained:

Failed to start forks worker: 0
Timeout waiting for worker to respond: 0

The failure-only raw-log artifact is:

name: vitest-guard-logs-31711572439-3-windows-latest-node22
artifact id: 9190243054
SHA-256: 90520d018b7a76cb3e172f8b78b070d94e5fa7bd0a0c131d33a0d567def9fa08

Five other OS/Node lanes in the same matrix passed. This is therefore distinct from the absorbed forks-worker symptom fixed by #690.

Current test contract

The affected test currently places one 20-second outer timeout around all of these operations:

1. Create temporary primary and linked-worktree directories.
2. git init.
3. Configure Git username/email.
4. Create and stage a README.
5. Commit the repository.
6. git worktree add -b feature <linked>.
7. Run generateGraph(linked, { noHtml: true }).
8. Assert the output directory and graph are outside the linked source checkout.
9. Assert the primary checkout remains isolated.
10. Run git worktree remove --force <linked>.
11. Delete temporary files.

One aggregate wall-clock timeout cannot identify which phase is slow and can fail on a correct but slower Windows filesystem or Git worktree operation.

Required investigation

1. Reproduce or directly prove the fragile contract

Instrument the test by phase before changing its semantics.

Capture durations and outcomes for:

• repository initialization;
• Git configuration;
• add and commit;
• linked-worktree creation;
• graph generation;
• graph/artifact assertions;
• linked-worktree removal;
• temporary-directory cleanup.

Run the instrumented test repeatedly on:

• Windows Node 20;
• Windows Node 22;
• Ubuntu Node 22;
• macOS Node 22.

Determine which phase crossed or approached the aggregate 20-second bound.

Do not infer the phase only from the final timeout.

2. Separate correctness from latency

The correctness contract is:

• the linked worktree resolves to the correct primary Git common directory;
• Madar places graph artifacts outside the linked source checkout;
• the graph path is deterministic and usable;
• the primary checkout's artifacts remain isolated;
• cleanup leaves no linked worktree registration or temporary artifact behind.

These conditions should be observed directly.

A true latency test belongs in a separately declared performance fixture with an evidence-supported budget. This issue must not silently turn integration correctness into a performance assertion.

3. Replace the single aggregate timeout

Prefer:

• explicit phase completion;
• direct assertions after each synchronous Git command;
• deterministic graph/artifact path checks;
• a test helper that reports the exact phase on failure;
• phase-specific deadlock limits only where an operation can genuinely hang.

The final test may retain a generous outer deadlock timeout, but it must not use that timeout as the primary correctness criterion.

Do not solve the issue by changing only:

20_000 → a larger number

4. Preserve real integration coverage

Continue to exercise real:

• Git repository initialization;
• Git linked worktree creation;
• Madar graph generation;
• path resolution;
• worktree removal.

Do not replace the complete test with mocks.

Mocks or injected delay controls may supplement the real integration test to prove phase diagnostics and cleanup behavior.

5. Cleanup correctness

Cleanup must run when failure occurs during:

• worktree creation;
• graph generation;
• assertions;
• worktree removal.

After the test, verify:

git worktree list --porcelain

contains no stale registration for the linked path.

Verify both temporary directories are removed when expected.

Do not hide cleanup failures behind the original timeout error.

Explicit non-goals

• No graph identity, multigraph, artifact-v2, retrieval, Pack, extraction, MCP, installer, or release behavior changes.
• No global Vitest timeout increase.
• No retry.
• No test skip or quarantine.
• No permanent worker reduction.
• No weakening of the guarded complete-suite commands from [P0] Make ordinary Vitest commands fail on absorbed worker-start failures #690.
• No change to the next beta release scope.
• No [P0][Graph PR B] Add deterministic semantic multigraph storage and artifact v2 #657 implementation.
• No merge of feat(qualification): freeze the independent qualification contract (#655) #681 or feat(graph): PR A — plural relationship queries and explicit topology APIs (#656) #680 before [P0] Stabilize the complete Vitest suite and protected CI merge gate #654 is complete.

Required characterization and tests

At minimum add or retain proof for:

1. Primary repository setup completes and Git commands report actionable failures.
2. Linked worktree creation succeeds with spaces in the temporary path.
3. Windows path separators do not affect graph artifact routing.
4. Graph artifacts live outside the linked source checkout.
5. Primary and linked checkout artifacts remain isolated.
6. Graph generation failure identifies the graph-generation phase.
7. Worktree-add failure identifies the worktree-add phase.
8. Cleanup runs after assertion failure.
9. Cleanup runs after graph-generation failure.
10. git worktree list --porcelain has no stale linked registration afterward.
11. A deliberately slowed phase does not create a false correctness failure merely because the aggregate runtime exceeds 20 seconds.
12. A genuinely hung or failed phase fails with the phase name and retained diagnostics.
13. No global timeout, retry, skip, quarantine, or worker-count workaround is introduced.

Validation

At minimum run:

npm ci
npm run typecheck
npm run build
npx vitest run tests/unit/workspace.test.ts --maxWorkers=1
npx vitest run tests/unit/workspace.test.ts --maxWorkers=4
npm run test:run
npm run test:coverage
npm run verify:pack-parity
npm pack --dry-run
npm run registry:validate
npm run release:verify

Run the focused workspace test repeatedly on Windows Node 20 and Node 22.

Then run at least three exact-commit complete six-lane protected matrices:

Ubuntu × Node 20
Ubuntu × Node 22
macOS × Node 20
macOS × Node 22
Windows × Node 20
Windows × Node 22

For every lane and attempt report:

• job conclusion;
• workspace test result and duration;
• guarded complete-suite status;
• worker-start signature count;
• handshake-timeout signature count;
• raw-log readability/control result;
• cleanup result.

Do not rerun until green and report only the successful attempt.

Acceptance criteria

• The failing Windows phase is identified through direct evidence, or the aggregate-timeout fragility is proven with controlled phase delay.
• The test no longer relies on one fixed 20-second wall-clock budget as its correctness contract.
• No timeout-only workaround is used.
• Real linked-worktree and graph-generation integration remains covered.
• Failure diagnostics name the exact phase.
• Cleanup is guaranteed and verified after every tested failure phase.
• No stale linked worktree remains registered.
• Windows path behavior is explicitly tested.
• Focused tests pass repeatedly on Windows Node 20 and Node 22.
• Three complete exact-commit protected matrices pass.
• Every accepted lane has zero canonical worker-start signatures.
• Raw-log controls prove scanning was active.
• No unexplained timeout or nontermination remains.
• No unrelated roadmap scope enters the diff.

Relationship to #693 and #654

The fresh macOS 26 ARM64 VM qualification in #693 may finish and establish its own environment boundary independently.

However, #654 remains incomplete while this Windows timeout is unexplained. Three clean protected matrices are unavailable until this issue is resolved and rerun on one exact commit.

Do not discard or rerun the failed matrix as though it were clean evidence.

Rollback

Revert the phase-aware test/helper changes together. Do not restore the fixed 20-second aggregate timeout as the permanent gate. If the proposed design is invalid, return to investigation with the retained Windows log and phase measurements.

Agent handoff

Start from current origin/next in an isolated worktree after the live #693 diagnostics run has finished or while it continues remotely. Begin with phase instrumentation and the exact Windows failure receipt. Keep the PR focused on workspace.test.ts, directly shared workspace/Git test helpers, and minimal testability seams. Return the phase timing table, identified failure phase, files changed, cleanup proof, focused Windows results, exact protected matrices, raw-log counts, and remaining uncertainty. Stop after opening a focused PR for review.

[mohanagy]

Qualification complete — PR #696 at 9530e48b

Three sequential exact-head six-lane protected matrices, all eighteen lanes inspected from raw logs.

Matrix	Run	Lanes	Failed to start forks worker	Timeout waiting for worker to respond	[phase-run]	Guarded invocations/lane	Positive control
1	31742328001	6/6 green	0	0	0	1	PASS
2	31743043019	6/6 green	0	0	0	1	PASS
3	31743659051	6/6 green	0	0	0	1	PASS

Dispatched via workflow_dispatch against the branch head, so each executed 9530e48b itself rather than a merge commit. ci.yml sets cancel-in-progress: true, so they were necessarily sequential. Raw logs 1.45 / 1.41 / 1.47 MB.

The zeros are demonstrated, not merely absent. For each lane the scanner appends a synthetic Failed to start forks worker line to a copy of that lane's log and re-scans; the count must rise by exactly one. It did, on all eighteen.

Review

CodeRabbit returned four findings, all self-rated non-blocking. Two were accepted after independent verification against the code and are fixed in 9530e48b:

• cleanup() could throw. reserveName sat outside the try, so a duplicate cleanup name escaped the function. Callers invoke cleanup() from finally, where a throw replaces the in-flight work failure — the one outcome the rest of that function is built to prevent. Latent rather than live: no names currently collide, but remove-temp-root is reused across five call sites. Now recorded as a cleanup failure instead of thrown.
• The primary emit path was untested. emit(true) with all phases passing is the route usePhaseRun takes when a test fails — the path that produces CI diagnostics. A regression there would have removed those diagnostics without failing anything.

Two were declined: splitting the marked-deadlock scenario (already green, low value), and renaming totalMs (that string is grepped in CI, so a rename changes an output surface — a clarifying comment was added instead).

Both fixes were mutation-tested rather than accepted structurally. Inverting the usePhaseRun emit condition kills exactly the lifecycle assertion; restoring the throwing reserveName kills the duplicate-name tests.

Independent verification of the isolation fix

The order-dependency this PR's earlier head carried was real and I reproduced it before it was fixed: four of five named tests failed alone with Linked worktree fixture was not resolved. At 9530e48b all seven named tests pass alone from fresh invocations, under three shuffle seeds, and three times each at --maxWorkers=1 and --maxWorkers=4.

Worth recording as a method note: whole-file runs repeated at two worker counts cannot detect a test that depends on another test. Only isolation and reordering can. That gap is why the defect survived an otherwise thorough proof set.

Scope

Three test files. src/, .github/, package.json, and vitest.config.ts are untouched, verified by diff against next. No production behaviour changes.

A production gap identified during this work — synchronous Git discovery commands with no process policy and a bare catch that would make a timeout indistinguishable from a legitimate null — is deliberately not addressed here and is tracked in #697.

Two infrastructure observations found while qualifying this PR, neither a defect in it, are recorded on #654: coverage mode has only one lane of CI exposure, and tests/setup.ts accumulates one empty temp directory per worker process indefinitely.

Status

Reviewed and qualified. Not merged — that is the maintainer's call. Refs #695. Related parent: #654. #654 remains open.

[mohanagy]

Complete — merged to next and qualified on the merged commit

PR #696 squash-merged with exact-head protection (--match-head-commit 9530e48b9180d8628cfea2dda0c8637d24f90987).

Reviewed head : 9530e48b9180d8628cfea2dda0c8637d24f90987
Merged next   : 6b6b3d8459ace1c26cc7d05abb27dc8cf3f72c08
main          : 3371ada8425efa7f8cabdac781fa227feaea7a6a (unchanged)
Scope         : 3 test files, +950 / -64. No production file.

Acceptance criteria

#	Criterion	Evidence	Result
1	Failing Windows phase identified, or aggregate fragility proven	Phase instrumentation on run 31730399080 produced per-phase attribution; largest single Windows unit 3617 ms against the former 20 s aggregate — the budget was consumed by accumulation across phases, not one slow phase	met
2	No fixed 20 s wall-clock budget as correctness contract	Aggregate timeout removed. NON_GATING_ELAPSED_CEILING_MS = 60_000 is explicitly non-gating and cannot interrupt a synchronous body	met
3	No timeout-only workaround	No retry, skip, quarantine, worker reduction, or global timeout change. Verified by diff against next	met
4	Real linked-worktree and graph-generation integration retained	Real git worktree add and real generateGraph still executed in the fixture	met
5	Failure diagnostics name the exact phase	PhaseFailure carries phase, kind, durationMs, and a live timeline	met
6	Cleanup guaranteed and verified after every tested failure phase	Cleanup runs in finally at five sites plus the setup-failure path; failures surface via throwCleanupErrors or attachCleanupErrors + rethrow	met
7	No stale linked worktree registration remains	git worktree list --porcelain asserted at workspace.test.ts:320, :373, :402	met
8	Windows path behavior explicitly tested	sep import, replaceAll('\\','/'), .normalize('NFC'); fixture at madar worktree ünïcode covers spaces, Unicode, and a trailing space	met
9	Focused tests pass repeatedly on Windows Node 20 and Node 22	Both Windows lanes green in all six matrices (3 branch-head + 3 merged-next)	met
10	Three complete exact-commit protected matrices	Branch head: 31742328001, 31743043019, 31743659051. Merged next: 31748654971, 31749252628, 31749780064	met
11	Zero canonical worker-start signatures in every accepted lane	36 lanes total, 0 Failed to start forks worker, 0 Timeout waiting for worker to respond	met
12	Raw-log controls prove scanning was active	Synthetic signature appended to a copy of each lane log; count rose by exactly one on every lane	met
13	No unexplained timeout or nontermination remains	210/210 test files pass per lane, 2956 tests, zero Test timed out occurrences	met
14	No unrelated roadmap scope in the diff	Exactly 3 test files; src/, .github/, package.json, vitest.config.ts untouched	met

Merged-next qualification — 18 lanes on 6b6b3d84

Attempt	Run	Lanes	worker-start	handshake	[phase-run]	invocations/lane	control
1 (post-merge push)	31748654971	6/6	0	0	0	1	PASS
2	31749252628	6/6	0	0	0	1	PASS
3	31749780064	6/6	0	0	0	1	PASS

Each lane runs exactly one guarded Vitest invocation — five npm run test:run, Ubuntu Node 22 npm run test:coverage. workspace-phase-run.test.ts executed in all six lanes; phase-run.ts reports 97.4% coverage in the coverage lane.

Per-lane totals: 2953 passed | 1 expected fail | 2 skipped on Linux and macOS, 2938 | 1 | 17 on Windows, 2956 in every lane. The single expected failure is the deliberate test.fails case that exercises usePhaseRun's emit-on-failure path; it is documented in the PR body.

Test independence

The original submission carried an inter-test order dependency — four of five named tests failed alone. All shared state now lives in beforeAll. At the merged commit all seven named tests pass alone from fresh invocations, under shuffle seeds 695 / 20260813 / 314159, and three times each at --maxWorkers=1 and --maxWorkers=4.

Scope and follow-ups

No production behaviour changed. Unbounded synchronous production Git discovery — gitPath() in src/shared/workspace.ts, no timeout, bare catch — is tracked in #697 and was deliberately not addressed here.

Two infrastructure observations found during qualification, both non-blocking: #698 (coverage mode runs on Ubuntu Node 22 only) and #699 (per-worker temp home accumulation).

Related parent: #654. #654 remains open pending its own final acceptance matrix.