Parent: #654
Target branch: next
Depends on: #689 merged, #690 completed
Blocks: #654 completion, merge of #681 and #680, and start of #657
Milestone: v0.33.0-beta.1 — Trust Foundation
Outcome
Close the remaining evidence gap in #654 by proving, with controlled and reproducible experiments, whether the local forks-worker startup failures are caused by shared-host resource contention rather than a repository defect that also affects isolated protected runners.
This issue is the final qualification gate for #654. It is primarily an evidence and reliability task. It may produce a narrowly scoped diagnostics/documentation PR when required, but it must not invent a code change merely to create a pull request.
Current confirmed state
The following work is already complete or pending merge:
The remaining uncertainty is causal: the local/shared-host distinction is strongly supported, but it has not yet been demonstrated through a controlled quiet-versus-contended experiment on the same commit and environment.
Required question
Determine whether this statement is supported:
On a supported, isolated environment, Madar's authoritative guarded complete-suite commands complete without worker-start signatures; the reproducible local failures occur when unrelated concurrent Node/Vitest workloads starve Vitest's forks-worker startup handshake.
Do not broaden the claim beyond the collected evidence.
Step 1 — Freeze the qualification identity
Use one exact next commit after #689 and #690 are merged.
Record:
- commit SHA;
- OS and architecture;
- Node version;
- npm version;
- exact
package-lock.json SHA-256;
- CPU core count;
- physical/logical memory;
- free memory before each run;
- process count;
- Node/Vitest process count;
- relevant ulimit/process limits;
- Vitest worker configuration;
- command;
- environment variables affecting Vitest/Node;
- raw log path and checksum.
Do not compare runs from different commits or lockfiles as if they were one experiment.
Step 2 — Define the experiment before running it
Pre-register at least these conditions on the same supported local host:
A. Quiet baseline
- no unrelated repository agent task;
- no unrelated Node/Vitest process;
- normal foreground services only;
maxWorkers: 4 unchanged;
- guarded
test:run and test:coverage.
B. Controlled external Node contention
Create an explicit, reversible load outside the Madar process. Prefer a small repository-controlled or temporary harness that can vary:
- number of competing Node processes;
- CPU duty cycle;
- optional memory pressure within safe machine limits;
- duration.
Do not use arbitrary background applications as the only contention source.
C. Controlled worker-count diagnostic
Run the same suite under worker counts 1, 2, and 4 for diagnosis only.
Do not change the repository's supported default merely because a lower diagnostic setting succeeds.
D. Recovery condition
After removing controlled contention, rerun the exact guarded commands and prove the environment returned to the quiet result.
The minimum experiment should demonstrate an A → B → D pattern on the same commit. Repeat enough times to distinguish a causal relationship from one coincidental pass/failure.
Step 3 — Measure the worker-start boundary
For every run capture:
- child command exit code;
- guard exit code;
- worker-start signature count;
- handshake-timeout signature count;
- affected test files;
- whether affected files pass alone;
- total test files and assertions;
- test duration;
- peak RSS where practical;
- CPU/load average where practical;
- total process count;
- Node/Vitest process count;
- whether the process terminates cleanly;
- unrelated assertion failures;
- retained log path and checksum.
Vary controlled contention gradually where practical and identify the smallest observed load at which failures appear.
Do not claim a universal threshold from one machine. Report it as the observed boundary on the recorded host.
Step 4 — Prove the protected-runner distinction repeatedly
On the same exact commit, run at least three complete protected six-lane matrices:
Ubuntu × Node 20
Ubuntu × Node 22
macOS × Node 20
macOS × Node 22
Windows × Node 20
Windows × Node 22
For every lane and attempt report separately:
- job conclusion;
- guarded test command used;
- guarded coverage command used where applicable;
- worker-start signature count;
- handshake-timeout signature count;
- test and coverage exit statuses;
- log readability marker;
- injected/control-signature detection result.
A green job conclusion is insufficient without raw-log inspection.
Step 5 — Resolve the Node/npm support statement
The audited reference named npm 12.0.1, while current validated work uses npm 12.0.2 with the same lockfile.
Document the supported merge identity as:
- supported Node 20/22 patch lines that satisfy the locked toolchain;
- exact lockfile checksum;
- npm
12.0.1 audited reference;
- npm
12.0.2 accepted only when npm ci succeeds without mutating the lockfile and all protected gates pass.
Do not claim support for every npm patch version.
Update documentation only when current wording is inconsistent.
Step 6 — Re-evaluate every #654 acceptance criterion
Create a final matrix:
| #654 acceptance criterion |
Evidence |
Result |
#654 remains open until all of the above are satisfied.
In particular:
- root cause must be stated at the strength supported by the controlled experiment;
- no worker-start error may occur in the accepted quiet/reference and protected qualification runs;
- no unexplained timeout/nontermination may remain;
- the complete guarded commands must be the one authoritative suite contract;
- original and corrected logs must be retained;
- supported Node/npm/lock documentation must agree.
Permitted outcomes
Outcome A — causal local-resource distinction proven
#654 may be closed when:
- quiet A and recovery D are repeatedly clean;
- controlled B reproducibly introduces the signature;
- isolated protected matrices are repeatedly clean;
- no unexplained repository-level failure remains;
- every acceptance criterion is satisfied.
Outcome B — quiet local run still fails
Keep #654 open.
Investigate the reproducible repository or host-specific cause; do not attribute it to unrelated contention without evidence.
Outcome C — controlled contention does not affect failure rate
Keep #654 open and reject the current causal hypothesis.
Outcome D — local experiment cannot be isolated credibly
Use repeated protected matrices as the supported merge environment, document the local host as unsuitable for qualification, and obtain an additional clean dedicated local/VM runner before closing #654.
Do not lower the closure standard merely because the developer workstation cannot be isolated.
Explicit non-goals
Validation
At minimum, on the frozen commit:
npm ci
npm run typecheck
npm run build
npm run test:run
npm run test:coverage
npm run verify:pack-parity
npm pack --dry-run
npm run registry:validate
npm run release:verify
Run affected historical files independently and in the previously failing groupings.
Every full run must use the guarded commands and retain raw evidence.
Acceptance criteria
Rollback
This issue should not require a product rollback. Revert any diagnostics-only repository change that proves unreliable while retaining the evidence artifacts and issue record. Never revert the guarded commands merely because they expose a real environmental failure.
Agent handoff
Start only after #689 and #690 are merged into next. Use one isolated worktree and one exact commit. Reserve exclusive access to heavy local runs. Stop unrelated local agents and test processes before quiet qualification. Pre-register the experiment, run quiet and controlled conditions, retain all logs, inspect raw protected CI logs, and return the final #654 criterion matrix. Do not begin #657 or merge #681/#680. Stop after the #654 closure decision.
Parent: #654
Target branch:
nextDepends on: #689 merged, #690 completed
Blocks: #654 completion, merge of #681 and #680, and start of #657
Milestone:
v0.33.0-beta.1 — Trust FoundationOutcome
Close the remaining evidence gap in #654 by proving, with controlled and reproducible experiments, whether the local forks-worker startup failures are caused by shared-host resource contention rather than a repository defect that also affects isolated protected runners.
This issue is the final qualification gate for #654. It is primarily an evidence and reliability task. It may produce a narrowly scoped diagnostics/documentation PR when required, but it must not invent a code change merely to create a pull request.
Current confirmed state
The following work is already complete or pending merge:
npm run test:runandnpm run test:coveragecommands. They fail when raw Vitest logs contain:Failed to start forks workerTimeout waiting for worker to respondeven if Vitest later exits zero.
The remaining uncertainty is causal: the local/shared-host distinction is strongly supported, but it has not yet been demonstrated through a controlled quiet-versus-contended experiment on the same commit and environment.
Required question
Determine whether this statement is supported:
Do not broaden the claim beyond the collected evidence.
Step 1 — Freeze the qualification identity
Use one exact
nextcommit after #689 and #690 are merged.Record:
package-lock.jsonSHA-256;Do not compare runs from different commits or lockfiles as if they were one experiment.
Step 2 — Define the experiment before running it
Pre-register at least these conditions on the same supported local host:
A. Quiet baseline
maxWorkers: 4unchanged;test:runandtest:coverage.B. Controlled external Node contention
Create an explicit, reversible load outside the Madar process. Prefer a small repository-controlled or temporary harness that can vary:
Do not use arbitrary background applications as the only contention source.
C. Controlled worker-count diagnostic
Run the same suite under worker counts 1, 2, and 4 for diagnosis only.
Do not change the repository's supported default merely because a lower diagnostic setting succeeds.
D. Recovery condition
After removing controlled contention, rerun the exact guarded commands and prove the environment returned to the quiet result.
The minimum experiment should demonstrate an A → B → D pattern on the same commit. Repeat enough times to distinguish a causal relationship from one coincidental pass/failure.
Step 3 — Measure the worker-start boundary
For every run capture:
Vary controlled contention gradually where practical and identify the smallest observed load at which failures appear.
Do not claim a universal threshold from one machine. Report it as the observed boundary on the recorded host.
Step 4 — Prove the protected-runner distinction repeatedly
On the same exact commit, run at least three complete protected six-lane matrices:
For every lane and attempt report separately:
A green job conclusion is insufficient without raw-log inspection.
Step 5 — Resolve the Node/npm support statement
The audited reference named npm
12.0.1, while current validated work uses npm12.0.2with the same lockfile.Document the supported merge identity as:
12.0.1audited reference;12.0.2accepted only whennpm cisucceeds without mutating the lockfile and all protected gates pass.Do not claim support for every npm patch version.
Update documentation only when current wording is inconsistent.
Step 6 — Re-evaluate every #654 acceptance criterion
Create a final matrix:
#654 remains open until all of the above are satisfied.
In particular:
Permitted outcomes
Outcome A — causal local-resource distinction proven
#654 may be closed when:
Outcome B — quiet local run still fails
Keep #654 open.
Investigate the reproducible repository or host-specific cause; do not attribute it to unrelated contention without evidence.
Outcome C — controlled contention does not affect failure rate
Keep #654 open and reject the current causal hypothesis.
Outcome D — local experiment cannot be isolated credibly
Use repeated protected matrices as the supported merge environment, document the local host as unsuitable for qualification, and obtain an additional clean dedicated local/VM runner before closing #654.
Do not lower the closure standard merely because the developer workstation cannot be isolated.
Explicit non-goals
Validation
At minimum, on the frozen commit:
Run affected historical files independently and in the previously failing groupings.
Every full run must use the guarded commands and retain raw evidence.
Acceptance criteria
Rollback
This issue should not require a product rollback. Revert any diagnostics-only repository change that proves unreliable while retaining the evidence artifacts and issue record. Never revert the guarded commands merely because they expose a real environmental failure.
Agent handoff
Start only after #689 and #690 are merged into
next. Use one isolated worktree and one exact commit. Reserve exclusive access to heavy local runs. Stop unrelated local agents and test processes before quiet qualification. Pre-register the experiment, run quiet and controlled conditions, retain all logs, inspect raw protected CI logs, and return the final #654 criterion matrix. Do not begin #657 or merge #681/#680. Stop after the #654 closure decision.