Skip to content

Record wave-3 wiring status and audit notes #45

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
im-tyler merged 1 commit into from
Oct 4, 2026
Merged

Record wave-3 wiring status and audit notes #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions AUDIT_OPEN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,37 @@
# Open audit items

## 2026-10-04 — wave 3 wired behind flags (S01/S04, S05/S06, S09, S11, S16, S18, S23, S25)

All off by default or observe-only; see the programme's wave-3 table for flags.
Items an operator or reviewer should know:

- **Default-on behaviour that is new:** `SHIP_SAFE_FETCH` defaults to shadow, so
each forge API call does one extra background address lookup and may log a
one-time warning per host. The request is never changed. Enforcement needs
`SHIP_SAFE_FETCH=on` and, for a private forge, `SHIP_SAFE_FETCH_ALLOW`.
- **Kernel fix (PR #38) changes behaviour where it was broken:** a cell blocked
past its timeout is now interrupted instead of the kernel being killed.
POSIX-only. The original `TS-019` flake was not reproduced, so the cause is
unproven; keep watching for it.
- **`SHIP_BUDGET_RESERVATION=on` is single-process only.** It is refused off the
file runtime, but two file-runtime processes on one host cannot be detected.
- **`SHIP_MODEL_ROUTING=on` cannot switch mid-run** until `durable.ts` feeds the
tool-call journal; side effects are treated as uncertain.
- **`SHIP_POLICY_SHADOW` records disagreements only**, so no rate is available;
a missing record file means unknown, not zero.
- **S16 overlap** scans only the 200 most recent runs and does not count a task
proposed but not yet launched.
- **S05 digest** is recorded but not compared; readiness staleness is unchanged.
- **S09** forge thread resolution is read but not used; GitHub's REST API does
not expose it.
- **S18** has no worker wiring: nothing provisions the trees or runs the check.
- Earlier "unwired" notes for S09, S11, S16, S25, S01/S04 above now read as
"wired behind a flag, observe-only or off by default".
- The three scripts-suite failures several agents reported (grader matrix,
deploy-recovery reference, report totals) did **not** reproduce in CI on
any wave-3 PR; they appeared only in agent sandboxes. Parallel agents sharing
the fixed grading port 8901 is the leading suspect, unproven.

## 2026-10-04 — grader sensitivity and UI audit (S02, S10)

- **Eight product-journey graders were too weak and are tightened (PR #34).**
Expand Down
17 changes: 17 additions & 0 deletions docs/SHIP_RELEASE_PROGRAMME_2026-09-21.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -521,6 +521,23 @@ This section supersedes the earlier table where rows overlap (S13, S28, S05, S06
| S10 | #33 | `scripts/ui-audit.mjs` (47 routes x 5 viewports, optional axe-core, JSON and markdown report) and five defect-class fixes (input labels, empty table headers, underlined in-text links, keyboard-reachable scrolling regions, narrow-viewport overflow); report in `docs/UI_AUDIT_2026-10-03.md` | 14 helper tests with negative controls, 3 web source guards, one full audit run (230 page loads; no error-level finding or axe violation on audited pages apart from the framework 404) | No human observation of target users; 79 targets under 24px kept as compact controls (design decision); framework plain-text 404 (upstream-report candidate, not filed); keyboard activation, offline and error states, other roles, screen readers, `/connect*`, `/oidc/*`, `/hooks/*`, `/api/*` not audited |
| S28 | #30 | `timing` in `audit --format json` (not CSV), `timingSummary` with an n>=20 gate | 8 tests, CSV byte-pinned, mutation control; CLI wiring type-checked only | No real-store run; **the n>=20 threshold was an agent's choice and needs confirmation**; capacity table not recomputed |

#### Wave 3 (2026-10-04): wiring behind flags

Wave 3 wires wave-2 modules into the product. **Every change below is off by default or observe-only**, proven by tests that compare default behaviour with and without the change. "Checked" still means automated tests only; no row has a verified outcome and nothing was run against a live system or spent. Turning any flag on is an operator decision.

| Package | PR | Flag (default) | Implemented | Checked | Still open (specific) |
| --- | --- | --- | --- | --- | --- |
| S18 | #36 | `SHIP_INTEGRATION_CHECK=on` plus a per-coordination declaration (off) | Executed two-repository check (`src/integration-check.ts`): runs a declared command against the exact producer and consumer revisions, refuses to run on a wrong or dirty tree; completion gate in `coordination.ts`; static compatibility never satisfies it; an upstream move makes evidence stale | 10 tests running the real `verify-pair.mjs` through `LocalExecutor` on real git checkouts, 3 negative controls | Nothing in the worker provisions the trees or runs it; no route or UI; no sandbox run; merge ordering and version windows; evidence uncapped |
| S11 | #37 | `SHIP_BUDGET_RESERVATION=off|shadow|on` (off) | Ledger wired at intake, daily-budget check and completion settle; unpriced spend stays unknown | 11 tests on the real paths, negative controls | Single-process file ledger (`on` refused off the file runtime; two processes on one host undetectable); settle step untested; child and retry scopes and lease fencing unwired; no counters endpoint |
| Hygiene | #38 | none | Kernel cancel now sends a real interrupt (the old call did not wake a blocked cell, so every timeout killed the kernel); `@grpc/grpc-js` 1.14.5 (clears GHSA-m9gg-hp2v-232j, dev-only); steer-route refusal tests | Negative controls for the kernel and the steer route | Original `TS-019` flake not reproduced under load, so the cause is unproven; kernel interrupt is POSIX-only |
| S16 | #39 | `SHIP_SCHEDULE_OVERLAP=on` (off) | Worker sweep passes `isRunning`: overlap `skip` and `queue` enforced; unreadable run log blocks every schedule (fail closed) | 8 tests on the real sweep path, negative controls | Only the 200 most recent runs are scanned; a task proposed but not launched is not counted; the worker interval loop has no harness |
| S05, S06 | #40 | `SHIP_STACK_DETECT=on` (off) | Read-only stack proposals through a bounded executor read; `project detect` CLI; setup-page suggestions with no apply button; proposals-to-recipe adapter; additive digest recorded beside the config id | 35 tests over six synthetic repo trees through a real executor, 8 negative controls (two needed new tests) | Digest recorded but not compared, so staleness is unchanged; no real-repo corpus; private repos CLI only; setup page not rendered; recipe not executed anywhere |
| S23 | #41 | `SHIP_MODEL_ROUTING=shadow|on` (off) | Policy loader, failure classifier, fallback outside retry, insert-only `ship_model_segments` table; shadow records and never changes the model | 29 tests, 8 negative controls | Tool-call journal not fed by `durable.ts`, so `on` refuses every mid-run switch; records per process; no real outage or gateway; fixed-versus-routed comparison not built |
| S09 | #42 | `SHIP_FINDING_CONTINUITY=on` (off) | Advisory continuity classification in the existing scan step (additive fields), run-page block; opt-in forge inline comments | 12 root and 3 web tests, real three-revision git path, off-equivalence | Thresholds unvalidated; forge resolution read but not used; diff from a shallow checkout; no merge gate by design |
| S25 | #43 | `SHIP_POLICY_SHADOW=on` (off) | Policy resolved beside nine existing decision points; disagreements recorded to JSONL; `policy shadow-report` | 20 tests, union-merge control, 3 mutations, default-off byte-identical | No enforcement; agreements not counted (no rate); worker intake budget not hooked; models, tools, retention have no decision point; **service-account role undecided** |
| S01, S04 | #44 | `SHIP_SAFE_FETCH` unset = **shadow**, `on`, `off`; `SHIP_SAFE_FETCH_ALLOW` | Pinned fetch is the default for forge API calls in `git.ts` and `forge-state.ts`; shadow warns once per host without changing the request | 17 tests including a real loopback server and a rebinding resolver | Setup route, ladder-steps, worker and deploy entry points; `git clone` and `git fetch` subprocesses unguarded; shadow log must be reviewed on a real deployment before `on`; shadow adds one address lookup per forge call (unmeasured) |


Also merged this wave: #9 plan and inventory, #10 devalue 5.9.3 override. S10 and S02 grader sensitivity landed after this table was first written and are the first two rows after S13/S06.

### Evidence that was not available
Expand Down
Loading