diff --git a/devlog/_plan/260906_a_runtime_stack/000_plan.md b/devlog/_plan/260906_a_runtime_stack/000_plan.md index 521e47c38a..c724b27854 100644 --- a/devlog/_plan/260906_a_runtime_stack/000_plan.md +++ b/devlog/_plan/260906_a_runtime_stack/000_plan.md @@ -17,13 +17,14 @@ | Cycle | Artifact | Consumes | Delivers | |---|---|---|---| -| roadmap | 000 + 010..060 | live dev and public contributor changes | audited full integration plan; docs only | +| roadmap | 000 + 010..080 | live dev and public contributor changes | audited full integration plan; docs only | | sse | 010_sse.md | existing SSE relay boundary | failure notification independent of tee cancellation | | ws | 020_ws.md | prior transport baseline | WS outbound policy and pool identity | | recovery | 030_recovery.md | validated transport stack | MESSAGE recovery + reparse/cache semantics | | affinity | 040_affinity.md | recovery/reparse fields | stable Command Code conversation identity | | capabilities | 050_capabilities.md | final effective dispatch behavior | policy selection congruent with dispatch | -| landing | 060_landing.md | independently verified stack layers | current dev inclusion and closeout | +| windows-fixtures | 070_windows_fixtures.md | current Windows failure evidence | deterministic verifier repair below A stack | +| landing | 080_landing.md | independently verified stack layers and verifier repair | current dev inclusion and closeout | The owner explicitly requested a stack. Independent transport fixes are retained as separate cumulative layers to expose interaction at each head; this publication order is not a claim of a hard dependency between SSE and WS. The actual code dependency is recovery before affinity. Each layer has its own PR diff, regression proof and CI. Bottom-up merge only; retarget before deleting parent branches. Keep stacks short by landing verified lower layers while subsequent cycles continue when possible. diff --git a/devlog/_plan/260906_a_runtime_stack/003_audit_resolution.md b/devlog/_plan/260906_a_runtime_stack/003_audit_resolution.md index e772aeb294..f871eeb920 100644 --- a/devlog/_plan/260906_a_runtime_stack/003_audit_resolution.md +++ b/devlog/_plan/260906_a_runtime_stack/003_audit_resolution.md @@ -2,7 +2,7 @@ Independent reviewer returned GO-WITH-FIXES (2). Both findings accepted and folded before B: -1. Implementation-cycle D previously implied full CI/dev landing, inconsistent with prepared stack layers. 010..050 now explicitly distinguish exact-head remote focused/type verified draft preparation from 060 full-gate landing. Final objective and full-CI-before-merge criteria remain unchanged. +1. Implementation-cycle D previously implied full CI/dev landing, inconsistent with prepared stack layers. 010..050 now explicitly distinguish exact-head remote focused/type verified draft preparation from 080 full-gate landing. Final objective and full-CI-before-merge criteria remain unchanged. 2. Affinity reparse tests required a cohort option the shared post helper did not accept. 040 now names tests/helpers/agent-task-recovery.ts option extension, internal handler forwarding, and true/false/undefined observation in real initial/cache-only adapter calls. Private remote host/user paths were replaced with placeholders; exact machine setup remains ignored scratch. No product edits or local suites in roadmap cycle. diff --git a/devlog/_plan/260906_a_runtime_stack/004_windows_amendment.md b/devlog/_plan/260906_a_runtime_stack/004_windows_amendment.md new file mode 100644 index 0000000000..53b8712654 --- /dev/null +++ b/devlog/_plan/260906_a_runtime_stack/004_windows_amendment.md @@ -0,0 +1,3 @@ +# Windows verifier amendment + +Full Windows CI for SSE head failed two unchanged shutdown-spill fixtures. Logs and causal analysis are retained in ignored ci-triage/report.md. C confirms no concurrent ownership of responses-state.test.ts. Add a separate windows-fixtures PABCD after capabilities and before final landing. It repairs test-only clocks/fallback isolation, independently validates on Windows, publishes a small foundation PR and inserts its verified change beneath the source stack. Refresh descendants bottom-up while preserving contributor commits and required current-head checks. No production ACL/budget change, no test skip, no unexamined rerun. The final landing document moves to080; no existing completion criterion is weakened. Owner explicitly authorized admin merge. diff --git a/devlog/_plan/260906_a_runtime_stack/005_amendment_audit.md b/devlog/_plan/260906_a_runtime_stack/005_amendment_audit.md new file mode 100644 index 0000000000..f871b8eb37 --- /dev/null +++ b/devlog/_plan/260906_a_runtime_stack/005_amendment_audit.md @@ -0,0 +1,3 @@ +# Capability and verifier amendment audit + +Independent reviewer: capability plan PASS; roadmap GO-WITH-FIXES one prerequisite finding. Accepted. Added a new windows-fixtures prerequisite to landing while retaining its existing capabilities edge. No task/criterion completion states or existing prerequisite edges were removed. The durable dependency graph now prevents final landing from being selected before Windows verifier completion. Replaced stale060 landing references with080. Temporary Windows verification workflow still requires concrete security review before push. diff --git a/devlog/_plan/260906_a_runtime_stack/010_sse.md b/devlog/_plan/260906_a_runtime_stack/010_sse.md index e70fc2c223..27aee24e97 100644 --- a/devlog/_plan/260906_a_runtime_stack/010_sse.md +++ b/devlog/_plan/260906_a_runtime_stack/010_sse.md @@ -4,7 +4,7 @@ Status: candidate plan, docs-only; implementation class C3 (stream lifecycle). E ## Implementation-cycle completion versus landing -This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `060_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. +This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `080_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. ## Source, authorship and drift diff --git a/devlog/_plan/260906_a_runtime_stack/020_ws.md b/devlog/_plan/260906_a_runtime_stack/020_ws.md index 766c662986..2db3f2d07b 100644 --- a/devlog/_plan/260906_a_runtime_stack/020_ws.md +++ b/devlog/_plan/260906_a_runtime_stack/020_ws.md @@ -4,7 +4,7 @@ Status: candidate plan after layer 010, docs-only; implementation class C4 for t ## Implementation-cycle completion versus landing -This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `060_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. +This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `080_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. ## Source, authorship and drift diff --git a/devlog/_plan/260906_a_runtime_stack/030_recovery.md b/devlog/_plan/260906_a_runtime_stack/030_recovery.md index 99ea201b44..92accfca44 100644 --- a/devlog/_plan/260906_a_runtime_stack/030_recovery.md +++ b/devlog/_plan/260906_a_runtime_stack/030_recovery.md @@ -6,7 +6,7 @@ land; no implementation or verification pass is claimed here. ## Implementation-cycle completion versus landing -This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `060_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. +This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `080_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. ## Loop specification and scope @@ -23,7 +23,7 @@ This decade cycle ends with a reviewed prepared draft PR, exact-carried-head foc - Verifier: exact-layer remote focused regressions, full Cross-platform CI, privacy and type gates, and independent recovery-boundary review. Commands below are planned for remote execution only; none ran during planning. -- Stop condition: reviewed prepared draft and exact-head remote focused/type evidence; full CI/dev inclusion are required by 060 before feature completion. Partial #3661 stays open. +- Stop condition: reviewed prepared draft and exact-head remote focused/type evidence; full CI/dev inclusion are required by 080 before feature completion. Partial #3661 stays open. - Memory artifact: this file and main-owned `000` roadmap/evidence ledger. - Outcomes: DONE only with the evidence above; NOOP only if current dev already contains equivalent behavior and regressions; BLOCKED for external CI/review diff --git a/devlog/_plan/260906_a_runtime_stack/040_affinity.md b/devlog/_plan/260906_a_runtime_stack/040_affinity.md index c35e7c5db2..8c1380a72e 100644 --- a/devlog/_plan/260906_a_runtime_stack/040_affinity.md +++ b/devlog/_plan/260906_a_runtime_stack/040_affinity.md @@ -6,7 +6,7 @@ This first-cycle artifact is docs only; re-read current source at this layer's P ## Implementation-cycle completion versus landing -This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `060_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. +This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `080_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. ## Loop specification and scope @@ -21,7 +21,7 @@ This decade cycle ends with a reviewed prepared draft PR, exact-carried-head foc default trust for unclassified cache keys, or extra OAuth cache-key forwarding. - Verifier: remote identity/forwarding/reparse regressions, full current-head CI, privacy/type gates and independent boundary review. No local verifier runs. -- Stop: reviewed prepared draft atop recovery, with exact-head remote focused/type evidence. Full current-head gates and dev ancestry remain required in 060. +- Stop: reviewed prepared draft atop recovery, with exact-head remote focused/type evidence. Full current-head gates and dev ancestry remain required in 080. - Memory artifact: this file plus main-owned roadmap/ledger. Main alone owns FSM, goal, implementation, Git and stack integration. - Resources: existing gh credentials and later own-branch writes only. Inherited diff --git a/devlog/_plan/260906_a_runtime_stack/050_capabilities.md b/devlog/_plan/260906_a_runtime_stack/050_capabilities.md index b545f21bae..51514b6215 100644 --- a/devlog/_plan/260906_a_runtime_stack/050_capabilities.md +++ b/devlog/_plan/260906_a_runtime_stack/050_capabilities.md @@ -2,7 +2,7 @@ ## Implementation-cycle completion versus landing -This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `060_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. +This decade cycle ends with a reviewed prepared draft PR, exact-carried-head focused remote activation evidence and remote typecheck, with full CI dispatched. That cycle D does not claim the bug shipped, full CI passed, or an issue resolved. `080_landing.md` retains the mandatory full current-head cross-platform/type/privacy/docs evidence, review, dev ancestry and immediate source-PR/fully-resolved-issue closure gates. Later P consumes the verified prepared stack parent; it need not have landed yet. Only final landing yields feature DONE. ## Candidate implementation contract @@ -14,7 +14,7 @@ Status: candidate planning, not implementation or merge approval. Revalidate at - Goal: runtime selection and ordinary management dry-run agree on effective transport capabilities and exclude unresolved, missing, or disabled providers before scoring. - Non-goals: new provider metadata, registry precedence redesign, catalog UI, OAuth refresh, request transport changes, Lab activation changes, release operations, or changing caller-supplied synthetic dry-run evidence semantics. - Verifier: remote focused routing/API regressions plus exact-head full Cross-platform CI and a remote documentation build. No local tests, typecheck, builds, or verifier execution in this planning assignment. -- Stop: independently working reviewed draft with original authorship and exact-head remote focused/type evidence. Full current-head gates/dev ancestry remain required by 060. +- Stop: independently working reviewed draft with original authorship and exact-head remote focused/type evidence. Full current-head gates/dev ancestry remain required by 080. - Memory artifact: this document and the main-owned roadmap/evidence ledger. - Outcomes: DONE only after verified dev integration; NOOP only if current dev independently contains all behavior and regressions; BLOCKED for unavailable external CI/credentials; NEEDS_HUMAN/UNSAFE for a policy decision outside authorization; a resource checkpoint is reassessment, never fabricated completion. - Delegation: inherited parallel read-only reviewers are authorized. Main reclaims a packet after two distinct failed workers; further write delegation requires a P amendment with exact ownership. diff --git a/devlog/_plan/260906_a_runtime_stack/051_capability_refresh.md b/devlog/_plan/260906_a_runtime_stack/051_capability_refresh.md new file mode 100644 index 0000000000..190122e007 --- /dev/null +++ b/devlog/_plan/260906_a_runtime_stack/051_capability_refresh.md @@ -0,0 +1,5 @@ +# Effective-capability layer P refresh + +Consume050 on prepared affinity6b00fa8d6. Original #3671 remains7b1beb9c5, with two Hako commits. Carry both; fix the remaining public review by initializing routeResolutionFailed to !provider || provider.disabled===true before resolving enabled candidates. Preserve synthetic caller-supplied dry-run evidence semantics and the core/Lab slot boundary. Explicitly test missing and disabled candidates under allow/penalize/exclude, with healthy sibling and with none, at both runtime/evaluator and ordinary management dry-run. + +Regression worker owns only tests/routing/routing-capability-model-matching.test.ts and tests/routing/routing-profile.test.ts. Main owns production, four existing guide locales and runtime SOT. No local tests/typecheck/build; pinned remote Bun1.4 --isolate focused checks, actual API assertions, privacy/docs/fullCI before landing. Windows verifier repair is registered as an additional required cycle and does not weaken any earlier gate. diff --git a/devlog/_plan/260906_a_runtime_stack/070_windows_fixtures.md b/devlog/_plan/260906_a_runtime_stack/070_windows_fixtures.md new file mode 100644 index 0000000000..daab52ed55 --- /dev/null +++ b/devlog/_plan/260906_a_runtime_stack/070_windows_fixtures.md @@ -0,0 +1,123 @@ +# 070 — Deterministic Windows shutdown-spill fixtures + +Status: P amendment; documentation only. Future implementation is a separate C2 test-harness cycle after 050, before 080. C confirmed no ownership collision. Main owns the FSM, implementation, remote execution and insertion of this foundation beneath the runtime stack. + +## Evidence and boundary + +[Windows job 101339545421](https://github.com/lidge-jun/opencodex/actions/runs/33978547130/job/101339545421), head `4b34cbb8d3f308cd2b01e8d87784c65afb50a40f`, Bun 1.4.0: 3048 pass, 39 skip, 2 fail, 1 unhandled error. The two failures are in `tests/responses/responses-state.test.ts`: + +- Stable-tail (1297): the 500 ms drain timer selected synchronous fallback; its unmocked ACL runner failed with EICACLS. The actual async-delay cause is unmeasured. Global ACL call 7 is also an unreliable publication marker: snapshot and directory hardening share this runner. +- Reserved budget (1438): only the ACL clock is synthetic. `spill-store.ts:232` charged real serialization/filesystem elapsed time and exhausted the deadline before temp-file hardening. The expensive operation is not identified. + +Local evidence inputs: `.tmp/a-runtime-stack/ci-triage/report.md`, `sse-windows5.log:4038–4217`, and `prior-101262480176.log:3894–3897` in that same scratch directory. The earlier job passed the two cases; its overall run was not green. Unchanged source on the sampled dev is not an independently reproduced current-dev failure. Do not describe this as an SSE regression, a proven harmless transient, or a green Windows gate. + +Future edit set: **only `tests/responses/responses-state.test.ts`**. No production, workflow, manifest, shared fixture or budget changes. Reuse `forceWindowsAclLane`, `isSpillAclTarget`, `ICACLS_OK`, existing clock/runner setters, and spill event recording. Keep existing deadline, fallback, exhaustion and watchdog tests. No sleeps for synchronization, timeout increases, skips or relaxed assertions. + +Read-only owners inspected: + +| Owner | Contract retained | +|---|---| +| `src/responses/state.ts:595` | Drain races the observed tail against a real timer; `Date.now` alone cannot freeze that timer. | +| `src/responses/state.ts:771` and `:813` | Separate fallback reserve, remaining-budget forwarding, and repeated observation until the publication tail is stable. | +| `src/responses/spill-store.ts:100`, `:153`, `:225` | Existing I/O events and injectable spill clock; each harden gets min(per-call cap, remaining whole-write budget). | +| `src/lib/windows-secret-acl.ts:360`, `:410`, `:589` | Async runner timer; injected ACL clock; grant/inheritance/remove calls consume one harden deadline. | +| `tests/responses/ws-upstream.test.ts:725` | Existing Bun `jest.useFakeTimers` / `advanceTimersByTime` / `useRealTimers` convention. | + +## Hunk 1 — Stable-tail ordering, not elapsed disk time + +At the test import, add `jest`. Retain 1000/500 budgets. Use fake timers **only within this test**, with `Date.now` fixed to a captured real epoch and ACL/spill clocks fixed consistently. Capture native `setImmediate` before enabling fake timers for an event-loop checkpoint; this drains runnable promise work without a sleep or timer advance. No new shared helper. + +Replace the global `aclCalls === 1/7` runner with gates on the first two distinct spill temp paths at `/grant:r`: + +```ts +const gatedTemps = new Set(); +setAsyncIcaclsRunnerForTests(async args => { + const target = args[0] ?? ""; + if (!isSpillAclTarget(args) || !target.endsWith(".tmp") || args[1] !== "/grant:r") { + return ICACLS_OK; // includes snapshot, directory and later ACL steps + } + if (!gatedTemps.has(target)) { + gatedTemps.add(target); + if (gatedTemps.size === 1) { firstEntered(); await firstGate; } + if (gatedTemps.size === 2) { secondEntered(); await secondGate; } + } + return ICACLS_OK; +}); +let syncSpillCalls = 0; +setIcaclsRunnerForTests(args => { + if (isSpillAclTarget(args)) syncSpillCalls++; + return ICACLS_OK; +}); +``` + +Both principal resolvers remain synthetic through `forceWindowsAclLane`. Both ACL runners cover **every** target; filtering controls gating/counting, never whether a real subprocess is used. A fallback must fail the ordering oracle (`syncSpillCalls === 0`), rather than being hidden by the successful mock. + +Replace the current orchestration and 25 ms sleep with this exact ordering: + +1. Enter `try/finally` before the first enqueue/await. Enable fake timers and fixed epoch clock; install both clock setters. Enqueue first response and await its temp gate. +2. Start `flushResponseState`, immediately attach both settlement handlers, recording `flushed` and any error in a resolved outcome object. This avoids an unhandled rejection if an earlier assertion fails. +3. Enqueue second response **after** starting flush, then release first. Await second temp gate. Await a native `setImmediate` checkpoint, advance fake timers by 25 ms, then another native checkpoint. The drain timer stays below 500 ms; no real elapsed filesystem time can fire it. +4. Assert flush is still pending, exactly two distinct temp paths were gated, and no synchronous spill ACL calls occurred. Record `setSpillIoForTest({ record })` events and assert exactly one `stub-swap` so the first publication actually installed while the second is gated. +5. Release second, await the handled flush outcome and rethrow any captured error. Retain `{ residentCount: 0, spillStubCount: 2 }`; add pending `{ count: 0, bytes: 0 }`, two `stub-swap` events, and zero synchronous spill calls. Both stored response IDs must still expand to their distinct payloads. +6. `finally`: release **both** gates, await any started flush outcome and `flushPendingResponseSpillsForTests()` while mocks/clocks remain installed, then restore the Date spy and real timers in a nested `finally`. Existing `afterEach` restores setters. Never restore mocks while a gated async operation still owns work. + +Use a discriminated outcome (`{ ok: true } | { ok: false; error: unknown }`) rather than an undefined-error sentinel. Keep cleanup valid when either startup await/assertion fails. Fake-timer compatibility and the native checkpoint are remote Windows acceptance items, not assumed proof. Do not solve a failed fixture by globally suppressing timers or adding a production seam. + +## Hunk 2 — One logical fallback budget, actual drain timer + +At 1438, preserve `totalMs = 500`, `fallbackReserveMs = 300` and the pending async spill gate. Add the missing spill clock; scope a Date spy to the flush so outer fallback accounting and nested ACL accounting advance together. Keep native timers in this test: the unchanged 200 ms drain timer must expire while the async gate remains held. + +```diff + let aclClock = 0; + setNowForTests(() => aclClock); ++setResponseSpillNowForTests(() => aclClock); +``` + +Record `{ target, timeoutMs, spentBefore }` for **spill** synchronous ACL calls. Snapshot ACL calls return `ICACLS_OK` without charging the spill clock. For each spill call, record before incrementing `aclClock += 20`; preserve successful command results. + +```ts +const epoch = Date.now(); +const nowSpy = spyOn(Date, "now").mockImplementation(() => epoch + aclClock); +// Start only after the async spill gate announces entry. +try { + await flushResponseState(); // native 200 ms drain timer selects sync fallback +} finally { + release(); + try { await flushPendingResponseSpillsForTests(); } + finally { nowSpy.mockRestore(); } +} +``` + +An enclosing `try/finally` must also cover enqueue and `await started`, releasing the gate on early failure. Preserve all three original assertions: at least six spill commands, maximum deadline <= 150, and `200 + aclClock <= 500`. Add: + +- Every timeout is positive and <= `300 - spentBefore` (independent literal budget oracle). +- Within each target's grant/inheritance/remove sequence, each next timeout is exactly 20 ms smaller; do **not** assert global monotonicity across targets because a new harden has its own per-call cap. +- The async gate has not been released when synchronous spill work begins; fallback actually ran, pending count/bytes become zero, one spill stub remains, and replay contains the original payload. + +The Date spy prevents unmeasured real disk latency from consuming this logical-budget fixture. It does not disable the native drain timer. Real-time termination coverage remains in the unchanged cap-expiry test (1339) and `shutdown fallback budget exhaustion is contained by a child watchdog` (1613), using `tests/helpers/responses-state-shutdown-budget-child.ts`. Do not claim this test measures OS elapsed latency. + +## Windows red, control and proof + +Main executes these later on real Windows with the repository-pinned Bun, in isolated remote checkouts. Nothing below authorizes local tests in this documentation task. + +1. Preserve the failed root-head job/logs above. Run the original two tests on the pinned pre-fix baseline; record actual results, including a pass. Do not require random failure or accept retries as a fix. +2. In remote scratch only, force the old stable-tail drain to expire by holding the second gate until a recorded fallback entry. Use a counted synchronous sentinel that reports EICACLS instead of invoking native ACL tools. Confirm rejection and the fallback call; never infer the missing-mock path from elapsed time alone. This is a controlled mechanism probe, not proof that the same delay happened in CI. +3. In remote scratch only, use the existing spill `record("write")` event to advance a separate wall clock by 301 ms once synchronous fallback has begun. On the original reserved-budget fixture, spill uses that clock and fails before temp hardening; with the proposed shared logical spill clock, the same wall-clock perturbation cannot consume the ACL budget. Record entry and clock values. Keep this probe separate from production and from the committed passing fixture. +4. Prove oracle sensitivity with isolated remote mutations: (a) stop drain after its first observed tail, expecting the revised stable-tail pending/zero-fallback oracle to fail; (b) reset the ACL deadline for each command, expecting per-target 20 ms decrease assertions to fail. Separately advance the **injected spill clock** beyond 300 at the write event and require ETIMEDOUT, proving deadline enforcement remains active. Restore every mutation before green verification; retain diff and failing assertion for each probe. +5. Run the unchanged named cap-expiry and child-watchdog controls, then the whole focused file on the new exact head: + +```sh +# Remote Windows only; these commands are a future verifier recipe. +bun test --isolate --timeout 60000 tests/responses/responses-state.test.ts +bun run typecheck +``` + +6. Dispatch the actual Windows full-suite workflow on that exact head, including `bun test --isolate --timeout 60000 tests --shard=5/6` and every other required shard. Inspect job execution, not aggregate success with skipped tests. Record head SHA, Bun version, commands, job URLs, counts and absence of unhandled errors. Run current-head Linux/macOS gates and required scans as well. + +Implementation D means an independently reviewed prepared foundation draft with exact-head focused Windows evidence and remote typecheck; it is **not landing**. Main inserts the verified foundation beneath the stack, refreshes descendants bottom-up with original attribution intact, obtains required current-head gates, then admin-merges in dependency order. Verify each landed SHA is an ancestor of freshly fetched dev before closing a superseded PR or fully resolved issue. Partial issues retain their residual scope. See `080_landing.md`. + +Documentation acceptance: this file names both failed fixtures, all clock/timer boundaries, complete runner/cleanup coverage, executable negative controls, one-file implementation scope and separate landing gates. No test execution or implementation success is claimed here. + +## Remote execution fallback amendment + +The existing direct Windows SSH endpoint is unavailable; the reachable auxiliary host is Linux without Windows interop. Use GitHub Actions for actual Windows proof. If the existing full-suite workflow cannot execute focused causal probes, a separate owner-only `codex/a-verify-windows` branch may hold a temporary verification workflow triggered only by pushes to that exact branch. This workflow is never included in a product PR or merged to dev. It uses `windows-latest`, read-only contents permission, pinned checkout with `persist-credentials: false`, the existing pinned-Bun setup, fixed repository test commands and the exact carried fixture commit. No secrets, untrusted command inputs, self-hosted runner access or release permissions. It may execute the narrowly specified scratch mutations with guaranteed source restoration and upload logs. Independent security audit of the concrete workflow is required before pushing it. Standard per-head full CI remains the final gate; the temporary verifier cannot mark those checks green. diff --git a/devlog/_plan/260906_a_runtime_stack/060_landing.md b/devlog/_plan/260906_a_runtime_stack/080_landing.md similarity index 100% rename from devlog/_plan/260906_a_runtime_stack/060_landing.md rename to devlog/_plan/260906_a_runtime_stack/080_landing.md diff --git a/docs-site/src/content/docs/fr/guides/routing-profile-editor.md b/docs-site/src/content/docs/fr/guides/routing-profile-editor.md index b437c28b3d..18575f21c8 100644 --- a/docs-site/src/content/docs/fr/guides/routing-profile-editor.md +++ b/docs-site/src/content/docs/fr/guides/routing-profile-editor.md @@ -37,6 +37,14 @@ résultat du plafond. ## Simuler un profil enregistré +Les capacités des candidats utilisent la configuration effective du fournisseur, +après application du registre. Les exigences de localité (`localOnly` et +`remoteAllowed`) utilisent donc l’adresse amont effective. Si elle ne peut pas être +classée, `unknownEvidence.capability` détermine l’admissibilité du candidat. +Une configuration de fournisseur invalide qui ne peut pas être résolue est toujours +exclue avec `route-unavailable`, même si les capacités inconnues sont autorisées. +Les fournisseurs absents ou désactivés sont également exclus avec `route-unavailable` avant le calcul des scores. + Sélectionnez un profil enregistré et utilisez **Évaluation à sec** pour ajouter des éléments propres à la requête, tels que la taille de la fenêtre de contexte, l’utilisation d’outils, l’entrée d’images ou la sortie structurée. La simulation évalue l’admissibilité et la notation, mais n’envoie jamais de requête à un modèle en amont. Les modifications non enregistrées ne sont pas prises en compte par la simulation. Enregistrez d’abord le profil afin que la révision et l’évaluation affichées correspondent à la même configuration. diff --git a/docs-site/src/content/docs/guides/routing-profile-editor.md b/docs-site/src/content/docs/guides/routing-profile-editor.md index 5cf5fc6d71..d53e0d3616 100644 --- a/docs-site/src/content/docs/guides/routing-profile-editor.md +++ b/docs-site/src/content/docs/guides/routing-profile-editor.md @@ -38,6 +38,14 @@ cap outcome. ## Dry-run a saved profile +Candidate capabilities use the effective provider configuration after registry +overrides are applied. Locality requirements (`localOnly` and `remoteAllowed`) +therefore use the effective upstream address. If that address cannot be classified, +the profile's `unknownEvidence.capability` setting decides eligibility. +An invalid provider configuration that cannot be resolved is always excluded with +`route-unavailable`, even when unknown capabilities are allowed. +Missing or disabled providers are also excluded with `route-unavailable` before scoring. + Select a saved profile and use **Dry-run evaluation** to add request evidence such as context-window size, tool use, image input, or structured output. Dry-run evaluates eligibility and scoring but never sends an upstream model request. Unsaved edits are not used by dry-run. Save the profile first so the displayed revision and evaluation refer to the same configuration. diff --git a/docs-site/src/content/docs/tr/guides/routing-profile-editor.md b/docs-site/src/content/docs/tr/guides/routing-profile-editor.md index dd7aa50d72..74ab8a7bdc 100644 --- a/docs-site/src/content/docs/tr/guides/routing-profile-editor.md +++ b/docs-site/src/content/docs/tr/guides/routing-profile-editor.md @@ -52,6 +52,14 @@ ayrıdır. ## Kaydedilmiş bir profilde deneme çalıştırması (dry-run) yapma +Aday yetenekleri, kayıt defteri kuralları uygulandıktan sonraki etkin sağlayıcı +yapılandırmasını kullanır. Yerellik gereksinimleri (`localOnly` ve `remoteAllowed`) +bu nedenle etkin üst sunucu adresine göre değerlendirilir. Adres sınıflandırılamıyorsa, +adayın uygunluğunu profilin `unknownEvidence.capability` ayarı belirler. +Çözümlenemeyen geçersiz sağlayıcı yapılandırmaları, bilinmeyen yeteneklere izin +verilse bile `route-unavailable` ile her zaman dışlanır. +Eksik veya devre dışı sağlayıcılar da puanlama öncesinde `route-unavailable` ile dışlanır. + Kaydedilmiş bir profili seçin ve bağlam penceresi boyutu, araç kullanımı, görsel girişi veya yapılandırılmış çıktı gibi istek kanıtları eklemek için **Deneme çalıştırması değerlendirmesi (Dry-run evaluation)**'ı kullanın. Deneme @@ -99,5 +107,3 @@ Düzenleyici şu uç noktaları kullanır: } } ``` - - diff --git a/docs-site/src/content/docs/zh-tw/guides/routing-profile-editor.md b/docs-site/src/content/docs/zh-tw/guides/routing-profile-editor.md index e6ae93a76e..4aa10fcaa7 100644 --- a/docs-site/src/content/docs/zh-tw/guides/routing-profile-editor.md +++ b/docs-site/src/content/docs/zh-tw/guides/routing-profile-editor.md @@ -32,6 +32,10 @@ OpenCodex 儀表板中的 **Models → Routing** 分頁可以直接管理 `confi ## 試跑已儲存的設定檔 +候選能力使用套用 registry 覆寫後的有效供應商設定。因此，本地性需求（`localOnly` 與 `remoteAllowed`）會依據實際上游位址判定。若無法分類該位址，則由設定檔的 `unknownEvidence.capability` 決定候選是否合格。 +無法解析的無效供應商設定一律以 `route-unavailable` 排除，即使原則允許未知能力也是如此。 +缺少或停用的供應商也會在評分前以 `route-unavailable` 排除。 + 選取一個已儲存的設定檔，使用 **Dry-run evaluation** 加入請求證據，例如 context-window 大小、工具使用、圖片輸入或結構化輸出。試跑會評估資格與評分，但永遠不會送出上游模型請求。 未儲存的編輯不會被試跑使用。請先儲存設定檔，讓顯示的 revision 與評估參照同一份設定。 diff --git a/src/routing/capability.ts b/src/routing/capability.ts index 8495951a0f..7f26e8bbdf 100644 --- a/src/routing/capability.ts +++ b/src/routing/capability.ts @@ -10,7 +10,7 @@ * how that affects eligibility. */ -import { modelInList, type OcxConfig } from "../types"; +import { modelInList, type OcxConfig, type OcxProviderConfig } from "../types"; import { isCanonicalOpenAiForwardProvider, OPENAI_CODEX_PROVIDER_ID } from "../providers/openai-tiers"; import { serviceTierSupportForModel } from "../providers/service-tier"; import { PROVIDER_REGISTRY } from "../providers/registry"; @@ -149,14 +149,20 @@ function localRemoteEvidence(baseUrl: string | undefined): Pick entry.id === providerName); + const provider = resolvedProvider ?? config.providers[providerName]; + const registryEntry = resolvedProvider === undefined + ? PROVIDER_REGISTRY.find(entry => entry.id === providerName) + : undefined; const catalogRow = cachedCatalogModels().find(model => model.provider === providerName && model.id === modelId); const isNative = providerName === OPENAI_CODEX_PROVIDER_ID && !modelId.includes("/"); @@ -224,6 +230,7 @@ export function candidateCapabilityEvidence( ? [] : modelRecordValue(provider?.modelReasoningEfforts, modelId) ?? modelRecordValue(registryEntry?.modelReasoningEfforts, modelId) + ?? provider?.reasoningEfforts ?? (isNative ? nativeReasoningEfforts(modelId) : undefined); const tierSupport = provider diff --git a/src/routing/compatibility/assemble.ts b/src/routing/compatibility/assemble.ts index 1d543690a4..5bcef26fc8 100644 --- a/src/routing/compatibility/assemble.ts +++ b/src/routing/compatibility/assemble.ts @@ -52,11 +52,26 @@ export function assemblePolicyCandidateEvidence( return profile.candidates.map(candidate => { const key = `${candidate.provider}/${candidate.model}`; const compatibility = compatibilityByCandidate?.get(key); + const provider = config.providers[candidate.provider]; + let routed: OcxProviderConfig | undefined; + let routeResolutionFailed = !provider || provider.disabled === true; + if (provider && provider.disabled !== true) { + try { + routed = options.routedProviderConfig(candidate.provider, provider); + } catch { + // This is known unavailability, not unknown capability evidence. Keep + // the failure separate so permissive unknown policies cannot select it. + routeResolutionFailed = true; + } + } return { provider: candidate.provider, model: candidate.model, - capability: candidateCapabilityEvidence(config, candidate.provider, candidate.model), + ...(routeResolutionFailed ? { routeResolutionFailed: true } : {}), + capability: routed + ? candidateCapabilityEvidence(config, candidate.provider, candidate.model, routed) + : undefined, health: policyCandidateHealthEvidence(config, candidate, now), quota: quotaEvidenceForCandidate({ provider: candidate.provider, diff --git a/src/routing/evaluator.ts b/src/routing/evaluator.ts index a07b833063..7cf801bfe9 100644 --- a/src/routing/evaluator.ts +++ b/src/routing/evaluator.ts @@ -54,6 +54,8 @@ export interface PolicyCandidateEvidence { accountRef?: string; /** Codex pool account id (provider "openai"); used to derive account-scoped quota evidence. */ codexAccountId?: string; + /** A failed effective-transport resolution excludes the candidate under every unknown policy. */ + routeResolutionFailed?: boolean; capability?: RouteCapabilityEvidence; health?: RouteHealthEvidence; quota?: RouteQuotaEvidence; @@ -278,6 +280,8 @@ export function evaluatePolicyProfile( ...requestRequirementFor(requestEvidence, evidence.capability), ]; const exclusions: RouteExclusionReason[] = []; + const routeUnavailable = evidence.routeResolutionFailed === true; + if (routeUnavailable) exclusions.push({ code: "route-unavailable" }); const bad = unsatisfiedOrUnknown(requirements); for (const requirement of bad) { if (requirement.outcome === "unsatisfied") { @@ -310,7 +314,7 @@ export function evaluatePolicyProfile( if (unknownCostBlocked) { exclusions.push({ code: "cost-limit-unknown", detail: "maxEstimatedCostUsd" }); } - let eligible = !unsatisfied && !excludedByUnknown && !overCostLimit && !unknownCostBlocked; + let eligible = !routeUnavailable && !unsatisfied && !excludedByUnknown && !overCostLimit && !unknownCostBlocked; // Trace/dry-run copy only: report the profile cap that was applied and the // operator-visible outcome. Do not feed this copy into costScore() — that diff --git a/structure/01_runtime.md b/structure/01_runtime.md index 26f501419a..7fb1c00997 100644 --- a/structure/01_runtime.md +++ b/structure/01_runtime.md @@ -15,7 +15,7 @@ | `src/config/process-state.ts` | Owns `ocx.pid`, `runtime-port.json`, cheap liveness, full command-line identity verification, and snapshot-guarded cleanup. | | `src/server/ports.ts` | Owns bind availability and ephemeral-port selection. Temporary probes dispose accepted peers and wait for listener close before reporting success. | | `src/cli/status.ts` / `src/cli/status-probes.ts` | Status snapshot assembly and the shared read-only health/stale-process probes used by status and doctor. Probe evidence keeps recorded-port choice, before/after snapshots and per-call timer cleanup together. | -| `src/router.ts` | Provider/model selection before adapter dispatch. | +| `src/router.ts` | Provider/model selection before adapter dispatch. Policy execution and ordinary management dry-run share effective-provider capability evidence; unresolved, missing, and disabled providers are excluded before scoring. | | `src/types.ts` | Shared config, parsed request, adapter, and event types. | | `src/reasoning-effort.ts` | Codex reasoning-level definitions (`low`/`medium`/`high`/`xhigh`), per-model effort mapping, and catalog effort sanitization. | | `src/codex/shim.ts` | Codex autostart shim: replaces the `codex` binary with a wrapper that auto-starts the proxy on demand. It skips startup for management subcommands even when value-taking global flags precede the subcommand, and transactionally restores complete, stable external launcher replacements without a watcher or PATH rediscovery. | diff --git a/tests/routing/routing-capability-model-matching.test.ts b/tests/routing/routing-capability-model-matching.test.ts index bb956c2d8d..df5a6b22c2 100644 --- a/tests/routing/routing-capability-model-matching.test.ts +++ b/tests/routing/routing-capability-model-matching.test.ts @@ -1,10 +1,19 @@ -import { describe, expect, test } from "bun:test"; +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { validateConfigCandidate } from "../../src/config"; +import { NoEligiblePolicyCandidateError, routeModel, routedProviderConfig } from "../../src/router"; import { candidateCapabilityEvidence } from "../../src/routing/capability"; +import { assemblePolicyCandidateEvidence } from "../../src/routing/compatibility/assemble"; import { evaluatePolicyProfile } from "../../src/routing/evaluator"; +import { closeRequestHistoryIndex } from "../../src/routing/history/indexer"; +import { getRoutingProfile } from "../../src/routing/profile"; import { PROVIDER_REGISTRY } from "../../src/providers/registry"; import { modelRecordValue } from "../../src/reasoning-effort"; import { isModelTextOnly } from "../../src/vision"; -import type { OcxConfig, OcxProviderConfig } from "../../src/types"; +import type { OcxConfig, OcxProviderConfig, OcxRoutingProfileConfig } from "../../src/types"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; /** * `candidateCapabilityEvidence` describes what the resolver will do with a candidate, @@ -35,6 +44,278 @@ function configFor(provider: OcxProviderConfig): OcxConfig { return { providers: { custom: provider } } as unknown as OcxConfig; } +describe("policy capability evidence uses the effective provider", () => { + let testDir: string; + let previousHome: string | undefined; + + beforeEach(() => { + previousHome = process.env.OPENCODEX_HOME; + testDir = mkdtempSync(join(tmpdir(), "ocx-effective-capability-")); + process.env.OPENCODEX_HOME = testDir; + }); + + afterEach(() => { + closeRequestHistoryIndex(); + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + removeTreeWithRetry(testDir); + }); + + function policyConfig( + name: string, + provider: OcxProviderConfig, + model: string, + require: OcxRoutingProfileConfig["require"], + ): OcxConfig { + const result = validateConfigCandidate({ + port: 10100, + defaultProvider: name, + providers: { [name]: provider }, + routingProfiles: { guarded: { candidates: [{ provider: name, model }], require } }, + }); + if (!result.ok) throw new Error(result.error); + return result.config; + } + + const localOnly = { localOnly: true, remoteAllowed: false }; + const loopback = "http://127.0.0.1:11434/v1"; + + test("a loopback URL discarded by registry routing cannot satisfy a local-only policy", () => { + const config = policyConfig("deepseek", { + adapter: "openai-chat", baseUrl: loopback, allowPrivateNetwork: true, + }, "deepseek-v4-flash", localOnly); + const before = structuredClone(config); + + expect(routeModel(config, "deepseek/deepseek-v4-flash").provider.baseUrl) + .toBe("https://api.deepseek.com"); + expect(() => routeModel(config, "policy/guarded")).toThrow(NoEligiblePolicyCandidateError); + expect(config).toEqual(before); + }); + + test.each(["custom-local", "ollama"])("a genuine local %s endpoint remains eligible", name => { + const config = policyConfig(name, { + adapter: "openai-chat", baseUrl: loopback, allowPrivateNetwork: true, + }, "local-model", localOnly); + const before = structuredClone(config); + + const route = routeModel(config, "policy/guarded"); + expect(route.providerName).toBe(name); + expect(route.provider.baseUrl).toBe(loopback); + expect(route.routeDecision?.requirements).toEqual([ + { id: "local-only", expected: true, actual: true, outcome: "satisfied" }, + { id: "remote-allowed", expected: false, actual: false, outcome: "satisfied" }, + ]); + expect(config).toEqual(before); + }); + + test("an explicitly public endpoint remains ineligible for a local-only policy", () => { + const config = policyConfig("deepseek", { + adapter: "openai-chat", baseUrl: "https://api.deepseek.com", + }, "deepseek-v4-flash", localOnly); + expect(() => routeModel(config, "policy/guarded")).toThrow(NoEligiblePolicyCandidateError); + }); + + test("a local candidate is selected after excluding a registry-pinned remote candidate", () => { + const config = policyConfig("deepseek", { + adapter: "openai-chat", baseUrl: loopback, allowPrivateNetwork: true, + }, "deepseek-v4-flash", localOnly); + config.providers.local = { adapter: "openai-chat", baseUrl: loopback, allowPrivateNetwork: true }; + config.routingProfiles!.guarded!.candidates.push({ provider: "local", model: "local-model" }); + + const route = routeModel(config, "policy/guarded"); + expect(route.providerName).toBe("local"); + expect(route.provider.baseUrl).toBe(loopback); + expect(route.routeDecision?.candidates.map(candidate => candidate.eligible)).toEqual([false, true]); + }); + + test("registry no-vision defaults participate before policy image requirements", () => { + const config = policyConfig("deepseek", { + adapter: "openai-chat", baseUrl: "https://api.deepseek.com", + modelInputModalities: { "deepseek-v4-flash": ["text", "image"] }, + }, "deepseek-v4-flash", { imageInput: true }); + const routed = routeModel(config, "deepseek/deepseek-v4-flash"); + expect(isModelTextOnly(routed.provider, routed.modelId)).toBe(true); + expect(() => routeModel(config, "policy/guarded")).toThrow(NoEligiblePolicyCandidateError); + }); + + test("the effective model context ceiling gates a policy requirement", () => { + const config = policyConfig("openai-apikey", { + adapter: "openai-responses", baseUrl: "https://api.openai.com/v1", + modelContextWindows: { "gpt-6-astra": 2_000_000 }, + }, "gpt-6-astra", { minContextWindow: 1_500_000 }); + const routed = routeModel(config, "openai-apikey/gpt-6-astra"); + expect(routed.provider.modelContextWindows?.["gpt-6-astra"]).toBe(1_050_000); + expect(() => routeModel(config, "policy/guarded")).toThrow(NoEligiblePolicyCandidateError); + }); + + test("canonical forward auth filled by routing satisfies the encrypted-task requirement", () => { + const config = policyConfig("openai", { + adapter: "openai-responses", baseUrl: "https://chatgpt.com/backend-api/codex", + }, "gpt-5.5", { encryptedCodexTasks: true }); + + const route = routeModel(config, "policy/guarded"); + expect(route.provider.authMode).toBe("forward"); + expect(route.routeDecision?.candidates[0]?.capability?.encryptedCodexTasks).toBe(true); + expect(config.providers.openai!.authMode).toBeUndefined(); + }); + + test("the effective provider-wide reasoning ladder participates in policy selection", () => { + const config = policyConfig("xiaomi-mimo", { + adapter: "openai-chat", baseUrl: "https://api.xiaomimimo.com/v1", + }, "mimo-v2.5", { reasoningEffort: "high" }); + + const route = routeModel(config, "policy/guarded"); + expect(route.provider.reasoningEfforts).toEqual(["low", "medium", "high"]); + expect(route.routeDecision?.candidates[0]?.capability?.reasoningEfforts) + .toEqual(["low", "medium", "high"]); + expect(config.providers["xiaomi-mimo"]!.reasoningEfforts).toBeUndefined(); + }); + + test("a same-named custom transport does not inherit an unrelated registry model map", () => { + const config = policyConfig("meta-model", { + adapter: "openai-responses", baseUrl: "https://custom.example/v1", + }, "muse-spark-1.3", { reasoningEffort: "high" }); + const routed = routeModel(config, "meta-model/muse-spark-1.3"); + expect(routed.provider.baseUrl).toBe("https://custom.example/v1"); + expect(routed.provider.modelReasoningEfforts).toBeUndefined(); + expect(() => routeModel(config, "policy/guarded")).toThrow(NoEligiblePolicyCandidateError); + }); + + test("an invalid unselected transport cannot prevent a healthy sibling from routing", () => { + const config = policyConfig("local", { + adapter: "openai-chat", baseUrl: loopback, allowPrivateNetwork: true, + }, "local-model", {}); + config.providers.ollama = { adapter: "openai-chat", baseUrl: " " }; + config.routingProfiles!.guarded!.candidates.push({ provider: "ollama", model: "local-model" }); + + const route = routeModel(config, "policy/guarded"); + expect(route.providerName).toBe("local"); + expect(route.provider.baseUrl).toBe(loopback); + expect(route.routeDecision?.candidates[1]?.capability).toBeUndefined(); + }); + + test("an unresolved transport contributes no positive capability evidence", () => { + const config = policyConfig("ollama", { + adapter: "openai-chat", baseUrl: loopback, + modelInputModalities: { "local-model": ["text", "image"] }, + }, "local-model", { imageInput: true }); + config.providers.ollama!.baseUrl = " "; + + const evidence = assemblePolicyCandidateEvidence(config, getRoutingProfile(config, "guarded")!, Date.now(), { + routedProviderConfig, + }); + expect(evidence[0]?.capability).toBeUndefined(); + expect(() => routeModel(config, "policy/guarded")).toThrow(NoEligiblePolicyCandidateError); + }); + + test("missing and disabled providers are not resolved for capability evidence", () => { + const config = policyConfig("local", { + adapter: "openai-chat", baseUrl: loopback, allowPrivateNetwork: true, + }, "local-model", { tools: true }); + config.providers.disabled = { ...config.providers.local!, disabled: true }; + config.routingProfiles!.guarded!.candidates.push( + { provider: "missing", model: "model" }, + { provider: "disabled", model: "model" }, + ); + const resolved: string[] = []; + const evidence = assemblePolicyCandidateEvidence(config, getRoutingProfile(config, "guarded")!, Date.now(), { + routedProviderConfig: (name, provider) => { + resolved.push(name); + return routedProviderConfig(name, provider); + }, + }); + + expect(resolved).toEqual(["local"]); + expect(evidence[0]?.capability?.tools).toBe(true); + expect(evidence[1]?.capability).toBeUndefined(); + expect(evidence[2]?.capability).toBeUndefined(); + }); + + for (const unavailable of ["missing", "disabled"] as const) { + test.each(["allow", "penalize", "exclude"] as const)( + `${unavailable} first candidate is excluded under %s unknown policy`, + capability => { + // Empty requirements prevent another capability guard from masking availability. + const config = policyConfig("local", { + adapter: "openai-chat", baseUrl: loopback, allowPrivateNetwork: true, + }, "local-model", {}); + if (unavailable === "disabled") { + config.providers.disabled = { ...config.providers.local!, disabled: true }; + } + const profile = config.routingProfiles!.guarded!; + profile.candidates.unshift({ provider: unavailable, model: "local-model" }); + profile.unknownEvidence = { ...profile.unknownEvidence, capability }; + + for (const withSibling of [true, false]) { + if (!withSibling) profile.candidates.pop(); + const resolved: string[] = []; + const evidence = assemblePolicyCandidateEvidence( + config, getRoutingProfile(config, "guarded")!, Date.now(), { + routedProviderConfig: (name, provider) => { + resolved.push(name); + return routedProviderConfig(name, provider); + }, + }, + ); + expect(resolved).toEqual(withSibling ? ["local"] : []); + expect(evidence).toHaveLength(withSibling ? 2 : 1); + expect(evidence[0]?.routeResolutionFailed).toBe(true); + expect(evidence[0]?.capability).toBeUndefined(); + const evaluation = evaluatePolicyProfile(config, "guarded", {}, evidence); + expect(evaluation.selectedIndex).toBe(withSibling ? 1 : null); + expect(evaluation.candidates[0]).toMatchObject({ + provider: unavailable, + eligible: false, + requirements: [], + exclusions: [{ code: "route-unavailable" }], + }); + if (withSibling) { + expect(evidence[1]?.routeResolutionFailed).toBeUndefined(); + expect(evidence[1]?.capability?.tools).toBe(true); + expect(evaluation.candidates[1]?.eligible).toBe(true); + const route = routeModel(config, "policy/guarded"); + expect(route.providerName).toBe("local"); + expect(route.routeDecision?.candidates.map(candidate => candidate.eligible)).toEqual([false, true]); + expect(route.routeDecision?.candidates[0]?.exclusions).toEqual([{ code: "route-unavailable" }]); + } else { + expect(() => routeModel(config, "policy/guarded")).toThrow(NoEligiblePolicyCandidateError); + } + } + }, + ); + } + + test.each(["allow", "penalize", "exclude"] as const)( + "an unresolved first candidate is excluded when unknown capabilities are %s", + capability => { + const config = policyConfig("ollama", { + adapter: "openai-chat", baseUrl: loopback, + }, "local-model", {}); + config.providers.ollama!.baseUrl = " "; + config.providers.local = { adapter: "openai-chat", baseUrl: loopback, allowPrivateNetwork: true }; + const profile = config.routingProfiles!.guarded!; + profile.candidates.push({ provider: "local", model: "local-model" }); + profile.unknownEvidence = { ...profile.unknownEvidence, capability }; + + const route = routeModel(config, "policy/guarded"); + expect(route.providerName).toBe("local"); + expect(route.routeDecision?.candidates.map(candidate => candidate.eligible)).toEqual([false, true]); + expect(route.routeDecision?.candidates[0]?.exclusions).toContainEqual({ code: "route-unavailable" }); + expect(JSON.stringify(route.routeDecision)).not.toContain("Invalid baseUrl"); + }, + ); + + test("all unresolved candidates produce a policy exclusion while explicit routing keeps validation", () => { + const config = policyConfig("ollama", { + adapter: "openai-chat", baseUrl: loopback, + }, "local-model", {}); + config.providers.ollama!.baseUrl = " "; + + expect(() => routeModel(config, "policy/guarded")).toThrow(NoEligiblePolicyCandidateError); + expect(() => routeModel(config, "ollama/local-model")).toThrow('Invalid baseUrl for provider "ollama"'); + }); +}); + describe("candidateCapabilityEvidence model matching", () => { test("a family entry covers its tagged siblings, as the resolver does", () => { const provider = providerWithFamilyEntries(); diff --git a/tests/routing/routing-profile.test.ts b/tests/routing/routing-profile.test.ts index aafca468dd..0736ec96fb 100644 --- a/tests/routing/routing-profile.test.ts +++ b/tests/routing/routing-profile.test.ts @@ -476,6 +476,67 @@ describe("routing profiles (RI-04)", () => { expect(body.candidates?.[1]).toMatchObject({ provider: "b", eligible: false }); }); + for (const unavailable of ["missing", "disabled"] as const) { + test.each(["allow", "penalize", "exclude"] as const)( + `API dry-run excludes ${unavailable} provider under %s unknown policy`, + async capability => { + const config = baseConfig({ + providers: { + local: { adapter: "openai-chat", baseUrl: "http://127.0.0.1:11434/v1", allowPrivateNetwork: true }, + }, + defaultProvider: "local", + routingProfiles: { + guarded: { + candidates: [ + { provider: unavailable, model: "local-model" }, + { provider: "local", model: "local-model" }, + ], + require: {}, + unknownEvidence: { capability }, + }, + }, + }); + if (unavailable === "disabled") { + config.providers.disabled = { ...config.providers.local!, disabled: true }; + } + for (const withSibling of [true, false]) { + if (!withSibling) config.routingProfiles!.guarded!.candidates.pop(); + const req = new ManagementRequest("http://localhost/api/routing-profiles/dry-run", { + method: "POST", + headers: { "content-type": "application/json" }, + // No synthetic candidates: exercise the same assembly as runtime routing. + body: JSON.stringify({ profile: "guarded", evidence: {} }), + }); + const response = await handleManagementAPI(req, new URL(req.url), config, { + refreshCodexCatalog: async () => {}, + }); + expect(response).not.toBeNull(); + expect(response!.status).toBe(200); + const body = await response!.json() as { + selectedIndex: number | null; + candidates: Array<{ + provider: string; + eligible: boolean; + requirements: unknown[]; + exclusions: Array<{ code: string }>; + }>; + }; + expect(body.selectedIndex).toBe(withSibling ? 1 : null); + expect(body.candidates).toHaveLength(withSibling ? 2 : 1); + expect(body.candidates[0]).toMatchObject({ + provider: unavailable, + eligible: false, + requirements: [], + exclusions: [{ code: "route-unavailable" }], + }); + if (withSibling) { + expect(body.candidates[1]).toMatchObject({ provider: "local", eligible: true }); + } + } + }, + ); + } + test("API dry-run mirrors live codex cooldown for openai candidates", async () => { const { clearCodexUpstreamHealth, recordCodexUpstreamOutcome } = await import("../../src/codex/routing"); clearCodexUpstreamHealth();