Skip to content

[P0] Establish the causal local-resource boundary and complete #654 qualification #693

Description

@mohanagy

Parent: #654
Target branch: next
Depends on: #689 merged, #690 completed
Blocks: #654 completion, merge of #681 and #680, and start of #657
Milestone: v0.33.0-beta.1 — Trust Foundation

Outcome

Close the remaining evidence gap in #654 by proving, with controlled and reproducible experiments, whether the local forks-worker startup failures are caused by shared-host resource contention rather than a repository defect that also affects isolated protected runners.

This issue is the final qualification gate for #654. It is primarily an evidence and reliability task. It may produce a narrowly scoped diagnostics/documentation PR when required, but it must not invent a code change merely to create a pull request.

Current confirmed state

The following work is already complete or pending merge:

The remaining uncertainty is causal: the local/shared-host distinction is strongly supported, but it has not yet been demonstrated through a controlled quiet-versus-contended experiment on the same commit and environment.

Required question

Determine whether this statement is supported:

On a supported, isolated environment, Madar's authoritative guarded complete-suite commands complete without worker-start signatures; the reproducible local failures occur when unrelated concurrent Node/Vitest workloads starve Vitest's forks-worker startup handshake.

Do not broaden the claim beyond the collected evidence.

Step 1 — Freeze the qualification identity

Use one exact next commit after #689 and #690 are merged.

Record:

  • commit SHA;
  • OS and architecture;
  • Node version;
  • npm version;
  • exact package-lock.json SHA-256;
  • CPU core count;
  • physical/logical memory;
  • free memory before each run;
  • process count;
  • Node/Vitest process count;
  • relevant ulimit/process limits;
  • Vitest worker configuration;
  • command;
  • environment variables affecting Vitest/Node;
  • raw log path and checksum.

Do not compare runs from different commits or lockfiles as if they were one experiment.

Step 2 — Define the experiment before running it

Pre-register at least these conditions on the same supported local host:

A. Quiet baseline

  • no unrelated repository agent task;
  • no unrelated Node/Vitest process;
  • normal foreground services only;
  • maxWorkers: 4 unchanged;
  • guarded test:run and test:coverage.

B. Controlled external Node contention

Create an explicit, reversible load outside the Madar process. Prefer a small repository-controlled or temporary harness that can vary:

  • number of competing Node processes;
  • CPU duty cycle;
  • optional memory pressure within safe machine limits;
  • duration.

Do not use arbitrary background applications as the only contention source.

C. Controlled worker-count diagnostic

Run the same suite under worker counts 1, 2, and 4 for diagnosis only.

Do not change the repository's supported default merely because a lower diagnostic setting succeeds.

D. Recovery condition

After removing controlled contention, rerun the exact guarded commands and prove the environment returned to the quiet result.

The minimum experiment should demonstrate an A → B → D pattern on the same commit. Repeat enough times to distinguish a causal relationship from one coincidental pass/failure.

Step 3 — Measure the worker-start boundary

For every run capture:

  • child command exit code;
  • guard exit code;
  • worker-start signature count;
  • handshake-timeout signature count;
  • affected test files;
  • whether affected files pass alone;
  • total test files and assertions;
  • test duration;
  • peak RSS where practical;
  • CPU/load average where practical;
  • total process count;
  • Node/Vitest process count;
  • whether the process terminates cleanly;
  • unrelated assertion failures;
  • retained log path and checksum.

Vary controlled contention gradually where practical and identify the smallest observed load at which failures appear.

Do not claim a universal threshold from one machine. Report it as the observed boundary on the recorded host.

Step 4 — Prove the protected-runner distinction repeatedly

On the same exact commit, run at least three complete protected six-lane matrices:

Ubuntu × Node 20
Ubuntu × Node 22
macOS × Node 20
macOS × Node 22
Windows × Node 20
Windows × Node 22

For every lane and attempt report separately:

  • job conclusion;
  • guarded test command used;
  • guarded coverage command used where applicable;
  • worker-start signature count;
  • handshake-timeout signature count;
  • test and coverage exit statuses;
  • log readability marker;
  • injected/control-signature detection result.

A green job conclusion is insufficient without raw-log inspection.

Step 5 — Resolve the Node/npm support statement

The audited reference named npm 12.0.1, while current validated work uses npm 12.0.2 with the same lockfile.

Document the supported merge identity as:

  • supported Node 20/22 patch lines that satisfy the locked toolchain;
  • exact lockfile checksum;
  • npm 12.0.1 audited reference;
  • npm 12.0.2 accepted only when npm ci succeeds without mutating the lockfile and all protected gates pass.

Do not claim support for every npm patch version.

Update documentation only when current wording is inconsistent.

Step 6 — Re-evaluate every #654 acceptance criterion

Create a final matrix:

#654 acceptance criterion Evidence Result

#654 remains open until all of the above are satisfied.

In particular:

  • root cause must be stated at the strength supported by the controlled experiment;
  • no worker-start error may occur in the accepted quiet/reference and protected qualification runs;
  • no unexplained timeout/nontermination may remain;
  • the complete guarded commands must be the one authoritative suite contract;
  • original and corrected logs must be retained;
  • supported Node/npm/lock documentation must agree.

Permitted outcomes

Outcome A — causal local-resource distinction proven

#654 may be closed when:

  • quiet A and recovery D are repeatedly clean;
  • controlled B reproducibly introduces the signature;
  • isolated protected matrices are repeatedly clean;
  • no unexplained repository-level failure remains;
  • every acceptance criterion is satisfied.

Outcome B — quiet local run still fails

Keep #654 open.

Investigate the reproducible repository or host-specific cause; do not attribute it to unrelated contention without evidence.

Outcome C — controlled contention does not affect failure rate

Keep #654 open and reject the current causal hypothesis.

Outcome D — local experiment cannot be isolated credibly

Use repeated protected matrices as the supported merge environment, document the local host as unsuitable for qualification, and obtain an additional clean dedicated local/VM runner before closing #654.

Do not lower the closure standard merely because the developer workstation cannot be isolated.

Explicit non-goals

Validation

At minimum, on the frozen commit:

npm ci
npm run typecheck
npm run build
npm run test:run
npm run test:coverage
npm run verify:pack-parity
npm pack --dry-run
npm run registry:validate
npm run release:verify

Run affected historical files independently and in the previously failing groupings.

Every full run must use the guarded commands and retain raw evidence.

Acceptance criteria

  • One exact commit and lockfile identity is used for the final qualification.
  • Quiet-host and controlled-contention conditions are pre-registered and reproducible.
  • The same host demonstrates a clean → failing-under-contention → clean-after-contention pattern, or an additional dedicated local/VM runner supplies the missing clean-host evidence.
  • The causal claim is limited to what the experiment proves.
  • Three complete protected six-lane matrices pass on the exact commit.
  • Raw logs show zero known worker-start signatures in every accepted protected lane.
  • Control samples prove raw-log scanning was active in every attempt.
  • No unexplained timeout, nontermination, or unrelated assertion failure remains in accepted qualification runs.
  • Historical affected files pass alone and in representative groupings.
  • Node/npm/lock documentation matches validated reality.
  • One authoritative guarded complete-suite command contract is documented and used by protected CI.
  • Original failure logs and final evidence receipts are retained and linked.
  • Every [P0] Stabilize the complete Vitest suite and protected CI merge gate #654 acceptance criterion is mapped to evidence and satisfied.
  • [P0] Stabilize the complete Vitest suite and protected CI merge gate #654 is closed only after the complete matrix is reviewed.
  • No unrelated roadmap scope enters the work.

Rollback

This issue should not require a product rollback. Revert any diagnostics-only repository change that proves unreliable while retaining the evidence artifacts and issue record. Never revert the guarded commands merely because they expose a real environmental failure.

Agent handoff

Start only after #689 and #690 are merged into next. Use one isolated worktree and one exact commit. Reserve exclusive access to heavy local runs. Stop unrelated local agents and test processes before quiet qualification. Pre-register the experiment, run quiet and controlled conditions, retain all logs, inspect raw protected CI logs, and return the final #654 criterion matrix. Do not begin #657 or merge #681/#680. Stop after the #654 closure decision.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions