The new cross-platform E2E harness reports failed commands as PASS and is not wired into CI

[alpCaner]

Found while reviewing

PR #352, feat/v06-batch at 1cab240df85d5896c5390697c069720ff2bb24d3.

A real run of the newly committed scripts/e2e-full.sh ended with:

=== Linux: 23 passed, 0 failed ===

but at least two named assertions did not prove what they claimed.

False-green flash leg

jrun marks a command PASS when stdout is parseable JSON and stderr is empty. It never checks RC, ok, or exitCode.

The script creates/builds blinky-e2e, then invokes:

tan flash --dry-run --format json

from the parent directory, without --project blinky-e2e. The command actually returned:

{
  "command": "flash",
  "ok": false,
  "exitCode": 1,
  "project": {"root": ".../proj", "boardYaml": null},
  "issues": [{"code": "flash.manifest-not-found", "severity": "error"}]
}

The harness printed PASS flash: one envelope, 0-byte stderr (exit 1) and counted it toward 23/0.

False-green root equality leg

The section named doctor and bootstrap resolve the SAME root only tests that both strings are nonempty. The observed run printed different values:

doctor=.../proj/alp-sdk
bootstrap=.../proj/alp-workspace/alp-sdk
PASS #322: both resolve an SDK

Isolation and CI problems

• scripts/e2e-linux-freeze.sh hardcodes /home/caner/tan-cli, fetches/checks out the branch there, and therefore tests a different checkout when invoked from an isolated worktree.
• Both scripts commit real /home/caner and /Users/Caner paths. The repository's own leak gate fails on six occurrences.
• The full E2E scripts are not referenced by any workflow; native Windows/macOS jobs run unit/freeze smoke only, not this bootstrap/build E2E.
• The SDK clone is an unpinned shallow clone of the current default branch, so a result cannot be reproduced later from the tan SHA alone.

Acceptance criteria

• jrun accepts an explicit expected exit/ok verdict and fails on any mismatch.
• Flash targets the project that was built and asserts the expected plan/verdict.
• The root-equality assertion compares the intended values rather than non-emptiness.
• No developer-specific paths; the helper operates on its current checkout/arguments.
• The SDK revision is explicit and recorded.
• Full E2E is wired on the native platforms claimed, or the claim is scoped to the platforms actually run.
• test_no_leaked_host_paths.py passes.

Related: PR #352.

[alpCaner]

Fixed on dev in ac79d4c (PR #352). dev IS this repo's default branch, so Closes does fire here -- but this issue was filed AFTER the PR body's closing-keyword list was written, so it was never in it. Closed by hand; a merged PR body cannot close retroactively.

Verified on the merged-dev freeze in a pristine ubuntu:24.04 carrying only ca-certificates, git, python3.

jrun now takes an expected exit code (any where the host legitimately decides) and on EVERY call enforces envelope.exitCode == process exit code and ok == (exit 0). Measured across seven commands in the container -- the invariant holds on all of them.

#322 asserts equality, not non-emptiness; flash targets the project that was built and asserts project.root/project.boardYaml; the SDK clone is pinned via ALP_SDK_REF with its SHA recorded (sha=8af7f28 in the run above). scripts/e2e-container.sh runs the same harness bind-mounted.

NOT claimed by this close: wiring the full e2e into CI on the platforms the harness covers. That half of this issue is still open work.