diff --git a/devlog/_plan/260912_accounts/000_plan.md b/devlog/_plan/260912_accounts/000_plan.md new file mode 100644 index 0000000000..df54929c99 --- /dev/null +++ b/devlog/_plan/260912_accounts/000_plan.md @@ -0,0 +1,48 @@ +# Accounts work is delivered as independent policy and lifecycle changes + +Readers: the integration maintainer deciding which PRs can land and which issue acceptance remains open. OAuth callback retirement is independent of pool scheduling; history precedes capacity estimation; dedicated native-main reauthorization precedes its dashboard control. Existing reset activation, reset-credit operation identity, and canonical Fake-IP transport are preserved and verified rather than reimplemented. + +## Execution contract + +Satisfy-spec HOTL, triggered by the accounts-lane delegation on 2026-09-12. Scope: PRs #4280/#4080 and issues #3375/#3376/#4211/#3781/#3898. Goal: reviewed, attributed implementation PRs and final-tip hosted CI evidence with truthful remaining acceptance. Non-goals: merges, releases, service/account/config/network changes, native GitHub stacks, local product test execution of any size, heavy local build/typecheck/install. Existing credential/tool scope only; no user-imposed token/time/agent-count cap. + +Verification: source inspections and `git diff --check` are text checks only; regression test sources run on GitHub-hosted CI at each final cumulative tip. Intermediate cycle C records source review and deferred remote evidence, never local test passes. Stop: all implementation/disposition and final CI criteria met, or actual inaccessible field acceptance distinctly recorded. Outcomes: DONE for demonstrated delivered scope; PARTIAL/NEEDS_HUMAN for authenticated field or maintainer security acceptance still missing; actual tool rejections retained without bypass. Main owns implementation; inherited-model read-only agents advise on design, reflect the concrete plan, and independently audit A. No native architect role is exposed, so none is claimed. No extra setup is required by that limitation. + +Memory artifacts: this numbered unit, session-bound goalplan/ledger, and task-local `.tmp/accounts-20260912/000_handoff.md`. Security analysis stays only in scratch. Escalation: actual tool permission denial or new out-of-scope action; main reclaims failed read-only work after two distinct failed dispatches, retaining any independence gap. + +## Delivery map + +| Cycle | Outcome | Dependency | Branch relationship | +| --- | --- | --- | --- | +| roadmap | Lock these documents, no product edit | none | local docs checkpoint | +| callback | Carry latest #4280 with author credit | roadmap | independent dev PR | +| eligibility | Automatic pool selection honors excluded plans; explicit route preserved | roadmap | independent dev PR | +| reset | Carry #4080 reset-first ordering | roadmap | independent dev PR | +| generic-family | Family headroom and cooldown context | roadmap | independent dev PR | +| lifecycle | Generic affinity and classified recovery | generic-family | child of generic-family | +| generic-health | Selection reasons and health presentation | lifecycle | child of lifecycle | +| warmup | Durable one-shot zero-usage activation | roadmap | independent dev PR | +| history | Bounded raw quota observations, generation-safe retention | roadmap | independent dev PR | +| capacity | Estimated capacity with evidence/sample count | history | child of history | +| tun | Safe probe failure classification and consumer projection | roadmap | independent dev PR | +| reauth-api | Dedicated native-main device grant persistence and CLI | roadmap | independent dev PR | +| reauth-ui | Main-card start/poll/cancel | reauth-api | child of reauth-api | +| final | Repair hosted final-tip CI, collect reviews and disposition | all implementation | no merge | + +Per-phase decade documents carry before/after contracts and conditional acceptance. Every later P revalidates source anchors. Ordinary manual chains express only real dependencies. `.github/workflows/ci.yml` runs pull_request without a base filter; no workflow modification or cancellation is authorized. Source ownership comes from `structure/manifest.json` and `structure/INDEX.md`; update all owners when their area changes, preserving relevant facts with cross-links. + +## Current evidence and limitations + +Baseline `origin/dev`: 69e3dcda755a52feb1327edad6c8ea6cefd6e871. PR #4280 live head: 1f826d92c7205f31ce174bbd987c04b2b08f7da4; its follow-up includes 404 closure and all three OAuth structure owners. PR #4080 live head: ecf6b4e48a4c2992c296fada2caf6a8132313eaa. Both remain open. Fresh source/issue snapshots are in scratch; historic CI claims in PR bodies are contributor evidence only. + +`cxc map src/codex --limit 18` is unavailable in the installed plugin (requires a source checkout); use source ownership and bounded text searches instead. Session is bound to this managed worktree and host goal exists; hooksVerified=false does not prove Stop continuation. Local tests/build/typecheck/install: NOT RUN. Authenticated TUN field acceptance cannot be inferred from injected-DNS tests. + +## Source reconciliation decisions + +#4238 already implements the excludedPlans selector; this unit completes reasons and removes automatic all-excluded fallback, preserving explicit routing and native-main exemption. #2562 latest maintainer comment chooses generic pooling, so both Google-specific routers remain design inputs. Generic work is split into family context → lifecycle → health presentation; a separate warmup cycle covers one-shot zero-usage scheduling. These units are registered in the same goalplan. #3588 reset activation and manual reset operation-id are already implemented. + +Two design follow-ups encountered inherited-model capacity errors; one same-handle retry was requested, no model/settings were changed. Independent A audit remains required. + +## Roadmap cycle outcome + +Independent design reflection and A re-audit passed with the source restrictions in 001_roadmap_audit.md. B freezes the contracts as documentation only. C checks document paths/numbering and git whitespace; local product suites NOT RUN. D next direction: execute 010_callback.md independently, then the remaining dependency-ordered cycles. Runtime behavior has not improved yet; the rejected hypotheses were native history identity by sentinel alone, attempt timing inferred from untimed attempts, and one-shot implying one physical request through a retrying primitive. diff --git a/devlog/_plan/260912_accounts/001_roadmap_audit.md b/devlog/_plan/260912_accounts/001_roadmap_audit.md new file mode 100644 index 0000000000..794bd2144f --- /dev/null +++ b/devlog/_plan/260912_accounts/001_roadmap_audit.md @@ -0,0 +1,11 @@ +# Roadmap audit locks implementation boundaries + +The source audit separates landed work from remaining acceptance. Pool design reflection (Pauli), eligibility/native-main reflection (Singer), and TUN reflection (Faraday) all aligned after concrete amendments. These are inherited-model independent reads; no native architect role or runtime execution is claimed. + +Independent A reviewer Leibniz found three blockers: native history identity across offline login replacement, capacity attempt timing and truncated ledger attribution, and a warmup primitive that retries despite a one-attempt promise. Main accepted all three. Native history is not persisted in this slice, stored-pool history binds generation, capacity uses whole contained request intervals and rejects incomplete evidence, and scheduled warmup explicitly disables model fallback. Focused re-audit returned VERDICT: PASS on 2026-09-12, against 69e3dcda755a52feb1327edad6c8ea6cefd6e871. + +Reader result: separate PRs deliver callback transport, policy reasons/selection, reset ordering, generic lifecycle, quota history/capacity, diagnostic classification, and native-main reauth. Source-backed findings justify each slice; next step is the independent callback implementation cycle. No implementation or remote verification exists yet. Local tests/build/typecheck/install: NOT RUN by user instruction. Source and doc checks do not establish runtime behavior. + +Remaining acceptance constraints: authenticated TUN field evidence; native-main cross-restart history/token capacity intentionally omitted; generic recovery requires positive provider/post-refresh evidence (permanent refresh rejection and sidecar auth without such evidence stay terminal); one-shot warmup supports stored pool only. All remain visible in final issue dispositions and are not silently marked complete. + +C correction: first git diff --cached --check rejected spaces in blank lines of the quoted public patch, so the chained commit did not run. Those documentation-only spaces were removed before retry. The B-to-C narrative mentioned a commit prematurely; the actual commit and receipt follow this correction. diff --git a/devlog/_plan/260912_accounts/010_callback.md b/devlog/_plan/260912_accounts/010_callback.md new file mode 100644 index 0000000000..57cb4130c8 --- /dev/null +++ b/devlog/_plan/260912_accounts/010_callback.md @@ -0,0 +1,239 @@ +# Retire every OAuth callback response connection + +Cycle callback; C4 auth transport. Independent of account pool features. Existing public #4280 is the change source; source read: `src/oauth/callback-server.ts:177`, `tests/oauth/oauth-callback-server.test.ts:1`. No-op leaves pooled connections reaching retired handlers; reuse the contributor patch rather than introduce a second listener implementation. + +MODIFY `src/oauth/callback-server.ts`: add private `closingResponse(body, status, contentType = "text/html")`; both 404 and callback success/error return it. Before: ordinary Response headers contain only Content-Type, 404 has no explicit headers. After: each path includes `Connection: close`; state validation and graceful listener shutdown stay intact. +MODIFY `tests/oauth/oauth-callback-server.test.ts`: sequential fixed-port login and held-exchange favicon scenarios from #4280, with deterministic flow-publication barriers and cancellation cleanup rather than new polling sleeps. +MODIFY `structure/runtime.md`, `structure/transports/inventory.md`, `structure/providers/xai-grok.md`: carry the contributor's invariant and owner links. Public troubleshooting docs describe repeat login connection retirement if needed. + +The exact reviewed public diff is reproduced below as the implementation contract. Credit: Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>. Retain latest source author commits in provenance; no original PR edits/closure. + +```diff +diff --git a/src/oauth/callback-server.ts b/src/oauth/callback-server.ts +index dc49d5fcd2..2727f8362c 100644 +--- a/src/oauth/callback-server.ts ++++ b/src/oauth/callback-server.ts +@@ -37,6 +37,27 @@ function errorHtml(message: string): string { + + export type CallbackResult = { code: string; state: string }; + ++/** ++ * Every response this listener sends ends its connection. ++ * ++ * The preferred callback port is FIXED per provider, so a later login listens on the same ++ * number — but a keep-alive socket stays bound to the flow that served it, and stopping that ++ * listener does not close an already-established connection. A client reusing the socket would ++ * hand the NEXT login's callback to the RETIRED flow, which rejects the unknown state as a CSRF ++ * mismatch while the live flow waits for a callback it can no longer receive. ++ * ++ * This is not limited to the callback itself: a browser that fetches `/favicon.ico` after the ++ * success page pools the socket on the 404, which is why the policy belongs to EVERY response ++ * rather than the callback path. Nothing here benefits from reuse — exactly one callback is ++ * expected per flow — so route every response through this helper. ++ */ ++function closingResponse(body: string, status: number, contentType = "text/html"): Response { ++ return new Response(body, { ++ status, ++ headers: { "Content-Type": contentType, "Connection": "close" }, ++ }); ++} ++ + /** + * The redirect URI advertised to providers must stay `localhost` (it is what the OAuth + * apps have registered), but Windows commonly resolves `localhost` to `::1` first while +@@ -177,7 +198,7 @@ export abstract class OAuthCallbackFlow { + #handleCallback(req: Request, expectedState: string): Response { + const url = new URL(req.url); + if (url.pathname !== this.callbackPath) { +- return new Response("Not Found", { status: 404 }); ++ return closingResponse("Not Found", 404, "text/plain"); + } + + const code = url.searchParams.get("code"); +@@ -214,10 +235,7 @@ export abstract class OAuthCallbackFlow { + }); + } + +- return new Response(ok ? SUCCESS_HTML : errorHtml(errMessage), { +- status: ok ? 200 : consumeFlow ? 500 : 400, +- headers: { "Content-Type": "text/html" }, +- }); ++ return closingResponse(ok ? SUCCESS_HTML : errorHtml(errMessage), ok ? 200 : consumeFlow ? 500 : 400); + } + + #waitForCallback(expectedState: string): Promise { +diff --git a/structure/providers/xai-grok.md b/structure/providers/xai-grok.md +index 765b85a763..16d1322412 100644 +--- a/structure/providers/xai-grok.md ++++ b/structure/providers/xai-grok.md +@@ -15,6 +15,10 @@ Grounded in the open-sourced official client (xai-org/grok-build); unit + eviden + `~/.grok/auth.json` (read-only) before any refresh and adopt a newer usable generation with + zero IdP calls (`shouldAdoptGrokGeneration`, later-expiresAt authority); an IdP refresh + detaches the credential to `source:"oauth"`. ++- **Browser login callback:** Grok's browser login uses the shared `OAuthCallbackFlow` listener ++ on a per-provider FIXED loopback port, so every response it sends closes its connection. A ++ retired flow that kept a pooled socket would capture the NEXT login's callback and reject it ++ as a state mismatch; see `src/oauth/callback-server.ts`. + - **Two-lock refresh transaction:** per-provider+account intent lock held across the IdP + exchange plus a short global store-write lock + async mutation funnel around every + `auth.json` load-merge-persist (`src/oauth/store.ts`); generation-guarded persist +diff --git a/structure/runtime.md b/structure/runtime.md +index 3099c13bfd..f31080646d 100644 +--- a/structure/runtime.md ++++ b/structure/runtime.md +@@ -139,7 +139,7 @@ The server exposes `POST /api/stop` which restores native Codex config, stops an + | --- | --- | + | `src/providers/registry.ts` | Canonical provider presets for CLI, dashboard, OAuth, key providers, and metadata. | + | `src/providers/derive.ts` | Enrichment from provider presets into user config. | +-| `src/oauth/` | OAuth providers, token storage, refresh, and auth-token resolution. | ++| `src/oauth/` | OAuth providers, token storage, refresh, and auth-token resolution. The login callback listener binds a per-provider FIXED loopback port, so consecutive logins reuse the same number; every response it sends ends its connection (`Connection: close`, including non-callback paths such as a stray `/favicon.ico` 404). Stopping the listener does not close an established socket, so without that a pooled client would deliver the next login's callback to the retired flow, which rejects the unknown state as a CSRF mismatch while the live flow waits. | + | `src/adapters/openai-responses.ts` | Native OpenAI/ChatGPT Responses passthrough. | + | `src/adapters/openai-chat.ts` | OpenAI-compatible Chat Completions bridge. | + | `src/adapters/anthropic.ts` | Anthropic Messages bridge. | +diff --git a/structure/transports/inventory.md b/structure/transports/inventory.md +index dc9af564d6..11cb227b01 100644 +--- a/structure/transports/inventory.md ++++ b/structure/transports/inventory.md +@@ -20,6 +20,7 @@ surface is listed here so a maintainer can find the owner without grepping: + | GitHub Copilot | `src/providers/xai-transport.ts` (`resolveProviderTransport`), `src/providers/github-copilot-transport.ts` | `resolveProviderTransport` selects the Copilot transport when the routed provider name is `github-copilot`; the Copilot module then resolves its headers and base URL, and the registry seeds the provider row and model fallback. | + | API-key pools | `src/providers/api-key-selection.ts`, `src/providers/key-failover.ts` | A 429 rotates the active key and records a cooldown; `provider.apiKey` keeps mirroring the active entry so routing stays single-key. | + | OAuth account failover | `src/oauth/generic-account-failover.ts`, `src/oauth/anthropic-routing.ts` | Reactive pre-output 429 recovery is presence-driven with 2+ eligible accounts. Pool and `oauthAccountFailover` flags govern proactive routing, not the reactive retry: a disabled Anthropic pool recovers through quota ordering rather than its dormant strategy, and a per-provider `enabled` beats the global default in either direction. | ++| OAuth login callback (inbound) | `src/oauth/callback-server.ts` | The only inbound transport this area owns: a short-lived loopback listener on a per-provider FIXED port. Exactly one callback is expected per flow, so EVERY response closes its connection — a retired flow must never keep a pooled socket that would capture the next login's callback. | + | Alibaba regions | `src/providers/alibaba-region-backup.ts`, `src/providers/alibaba-region-migration.ts`, `src/providers/alibaba-region-startup.ts` | Region migration backs up before rewriting and is idempotent across restarts. | + | Discovery and quota | `src/providers/model-discovery.ts`, `src/providers/quota.ts` | Discovery rejects a response over 4 MiB or past 2,000 raw rows before caching it. | + +diff --git a/tests/oauth/oauth-callback-server.test.ts b/tests/oauth/oauth-callback-server.test.ts +index a327e1df0d..a46a2a04f3 100644 +--- a/tests/oauth/oauth-callback-server.test.ts ++++ b/tests/oauth/oauth-callback-server.test.ts +@@ -29,6 +29,16 @@ class ManualFallbackFlow extends OAuthCallbackFlow { + + const ctrl: OAuthController = {}; + ++/** Keeps the listener alive across the token exchange so stray requests can reach it. */ ++class SlowExchangeFlow extends ManualFallbackFlow { ++ holdExchange?: Promise; ++ ++ override async exchangeToken(code: string, state: string, redirectUri: string): Promise { ++ await this.holdExchange; ++ return super.exchangeToken(code, state, redirectUri); ++ } ++} ++ + describe("OAuth callback server defaults", () => { + test("binds callback listeners to numeric loopback by default", () => { + const flow = new TestFlow(ctrl, 54545, "/callback"); +@@ -116,4 +126,103 @@ describe("OAuth callback server defaults", () => { + blocker.stop(true); + } + }); ++ ++ test("a retired flow cannot serve the next login on the same callback port", async () => { ++ // The preferred callback port is fixed per provider, so consecutive logins listen on the ++ // same number. Stopping a listener does not close a connection that is already open, so a ++ // client that pools the socket would deliver the SECOND login's callback to the FIRST ++ // flow, which rejects the unknown state as a CSRF mismatch while the live flow waits. ++ const port = await freeLoopbackPort(); ++ const options = { ++ preferredPort: port, ++ callbackPath: "/callback", ++ callbackHostname: "127.0.0.1", ++ callbackBindHostname: "127.0.0.1", ++ }; ++ const deliver = async (state: string): Promise => { ++ const url = new URL(`http://127.0.0.1:${port}/callback`); ++ url.searchParams.set("code", "authorization-code"); ++ url.searchParams.set("state", state); ++ const res = await fetch(url); ++ await res.text(); ++ return res.status; ++ }; ++ ++ const first = new ManualFallbackFlow(ctrl, options); ++ const firstLogin = first.login(); ++ await waitForState(() => first.generated?.state); ++ const firstState = first.generated!.state; ++ expect(await deliver(firstState)).toBe(200); ++ await firstLogin; ++ ++ const second = new ManualFallbackFlow(ctrl, options); ++ const secondLogin = second.login(); ++ await waitForState(() => second.generated?.state); ++ const secondState = second.generated!.state; ++ expect(secondState).not.toBe(firstState); ++ // Served by the LIVE flow, so the retired state is now an unknown one. ++ expect(await deliver(firstState)).toBe(400); ++ expect(await deliver(secondState)).toBe(200); ++ await secondLogin; ++ expect(second.exchanged?.state).toBe(secondState); ++ }); ++ ++ test("a non-callback request cannot pin the socket to the retiring flow", async () => { ++ // A browser that asks for /favicon.ico after the success page would pool the socket on the ++ // 404 while exchangeToken() is still running, which re-pins it to the flow that is about to ++ // retire. The close policy therefore belongs to EVERY response, not just the callback path. ++ const port = await freeLoopbackPort(); ++ const options = { ++ preferredPort: port, ++ callbackPath: "/callback", ++ callbackHostname: "127.0.0.1", ++ callbackBindHostname: "127.0.0.1", ++ }; ++ const deliver = async (state: string): Promise => { ++ const url = new URL(`http://127.0.0.1:${port}/callback`); ++ url.searchParams.set("code", "authorization-code"); ++ url.searchParams.set("state", state); ++ const res = await fetch(url); ++ await res.text(); ++ return res.status; ++ }; ++ ++ // The exchange is held open so the listener is still up for the stray request, which is ++ // exactly the window the reproduction describes. ++ const exchanging = Promise.withResolvers(); ++ const first = new SlowExchangeFlow(ctrl, options); ++ first.holdExchange = exchanging.promise; ++ const firstLogin = first.login(); ++ await waitForState(() => first.generated?.state); ++ expect(await deliver(first.generated!.state)).toBe(200); ++ const favicon = await fetch(`http://127.0.0.1:${port}/favicon.ico`); ++ await favicon.text(); ++ expect(favicon.status).toBe(404); ++ exchanging.resolve(); ++ await firstLogin; ++ ++ const second = new ManualFallbackFlow(ctrl, options); ++ const secondLogin = second.login(); ++ await waitForState(() => second.generated?.state); ++ // Without the close policy on the 404 this is answered by the retired flow and returns 400. ++ expect(await deliver(second.generated!.state)).toBe(200); ++ await secondLogin; ++ expect(second.exchanged?.state).toBe(second.generated!.state); ++ }); + }); ++ ++/** A port that is free right now; the flows bind it themselves, so it must not stay held. */ ++async function freeLoopbackPort(): Promise { ++ const probe = Bun.serve({ hostname: "127.0.0.1", port: 0, reusePort: false, fetch: () => new Response("probe") }); ++ const { port } = probe; ++ probe.stop(true); ++ return port; ++} ++ ++async function waitForState(read: () => string | undefined, timeoutMs = 5_000): Promise { ++ const deadline = Date.now() + timeoutMs; ++ while (read() === undefined) { ++ if (Date.now() >= deadline) throw new Error("timed out waiting for the login flow to publish its state"); ++ await Bun.sleep(5); ++ } ++} + +``` + +Acceptance: first login succeeds, retired state is rejected by live listener (400), live state succeeds (200); favicon during held token exchange returns 404 without trapping the next flow. Failure/malformed callback paths close their connection too. Regression source is mandatory, local runtime execution NOT RUN. Hosted final-tip CI must cover oauth callback/bind and OrcaRouter provider suites. Security review checks unchanged state/PKCE, loopback destinations, no credential disclosure. Public code already describes the issue; additional security analysis goes to scratch only. + +P stale check: whole contributor patch fails only at inventory table context because the API-key row changed. Selected source/test/runtime/xai hunks pass `git apply --check`. During B retain current inventory rows and append the new callback row after OAuth failover manually; do not overwrite current transport contracts. + +Callback P revalidation after roadmap D: next direction is independent callback carry. Source hunks still apply; #4280 remains open at the same 1f826d92c head. Replace contributed polling helper with onAuth Promise.withResolvers readiness, AbortController deadline and finally cleanup settling held exchanges and login promises. Keep runtime Connection: close unconditional for both paths; no forced fetch header masks the defect. Shorten helper comment while preserving retirement rationale. Native role unavailable; inherited-model consultation is explicitly authorized. Product tests NOT RUN; source audit then remote CI. + +Callback design ALIGNED (Lagrange) and independent A PASS (Leibniz): readiness resolves in onAuth queueMicrotask, login rejection rejects readiness; deadlines armed after handler registration; finally resolves held exchange, aborts flows, clears timers and settles login promises. Add response-only close-header checks alongside behavioral oracles. diff --git a/devlog/_plan/260912_accounts/011_callback_delivery.md b/devlog/_plan/260912_accounts/011_callback_delivery.md new file mode 100644 index 0000000000..23fa061b4a --- /dev/null +++ b/devlog/_plan/260912_accounts/011_callback_delivery.md @@ -0,0 +1,9 @@ +# Callback retirement implementation and evidence + +Carried #4280 at 1f826d92c7205f31ce174bbd987c04b2b08f7da4 by luvs01. Every callback-listener response uses a closing response helper; state validation, HTML escaping and graceful shutdown stay unchanged. Runtime, xAI and transport inventory ownership docs are updated. Tests retain two fixed-port flow scenarios and add malformed/provider-error response checks; onAuth microtask readiness and per-flow abort/finally cleanup replace polling waits. + +Necessity/source search: closingResponse and loopbackBindHostnames in callback-server.ts; no existing response-closing owner found. Reuse existing OAuthCallbackFlow and ManualFallbackFlow. The only new production helper is private and joins two response sites. Local tests/build/typecheck/install: NOT RUN. git diff --check is a whitespace check only. Independent implementation/security read and hosted CI follow publication; no bug-fixed claim until execution evidence exists. + +This first delivery includes the accounts roadmap documentation checkpoint; product delta is callback-only. Other features remain unimplemented and their docs describe pending work. Original #4280 stays open for coordinator disposition after integration. No merge is performed here. + +Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> diff --git a/devlog/_plan/260912_accounts/020_eligibility.md b/devlog/_plan/260912_accounts/020_eligibility.md new file mode 100644 index 0000000000..94719a4f58 --- /dev/null +++ b/devlog/_plan/260912_accounts/020_eligibility.md @@ -0,0 +1,19 @@ +# Finish automatic plan policy and visible exclusion reasons + +Cycle eligibility; C3 selection policy. Depends only on roadmap, independent dev PR. #4238 already added excludedPlans; do not reimplement its selector. Source: routing.ts:1044-1090 and 1326; explicit fixedAccountId path auth-context.ts:826/915. + +MODIFY `src/codex/routing.ts`: export the existing normalized policy predicate (or move the pure plan calculation into `src/codex/plan.ts` and reuse it). Add the predicate to BOTH configured-account fallback guards at preview :2152 and detailed resolve :2391. Before, an all-excluded pool returns its excluded active row; after, ordinary selection returns null/none. Explicit fixed routes retain existing auth, pause, entitlement checks. Native __main__ remains exempt, avoiding physical auth reads on selection-only paths. + +```diff +- && !isCodexAccountPaused(config, active) ++ && !isCodexAccountPaused(config, active) ++ && !isCodexAccountPlanExcluded(config, active) +``` + +MODIFY `src/codex/auth-api.ts`: poolAccountDto adds optional `selectionExcludedReason: "plan_excluded"`, derived from the SAME predicate and config, never from credential health; include current plan already in DTO. MODIFY `src/cli/account-api.ts` AccountRow/CodexAccountDto mapping and `src/cli/account.ts` statusText to show `not-auto-selected(plan=)`. MODIFY `gui/src/components/codex-account-pool-types.ts`, pool-card badge in `codex-account-pool-cards.tsx`, and all locale catalogs: separate localized reason; do not mutate paused/needsReauth and do not disable explicit routing. Unknown plan and empty policy remain eligible; reauth renewal clears the reason dynamically. + +Field chain: existing excludedPlans config create/save/load → same normalized predicate → account DTO JSON → CLI/GUI optional union → status and badge. No new config field or minimumPlan ordering. Enforcing tier: runtime automatic selection only; explicit fixed account intentionally bypasses this selection rule, not auth; residual unknown-plan and native-main exemptions documented, no hard account-block claim. + +MODIFY existing `tests/codex-integration/codex-pool-plan-exclusion.test.ts`: replace last-account soft fallback test with none/preview none; test normalized plan update and explicit fixed route. Extend account API/CLI and card tests for reason and renewal clearing. Sync ownership docs and providers configuration pages that describe the old soft exception. Retain source attribution of #4238; no recarry of already-landed commits. Local tests/build/typecheck NOT RUN. Hosted CI plus rendered artifact from final tip supplies execution proof. + +Exclusion reason derives from the routing config plan, not a display-only freshly observed plan if persistence failed. This preserves truth between selection and explanation. diff --git a/devlog/_plan/260912_accounts/030_reset.md b/devlog/_plan/260912_accounts/030_reset.md new file mode 100644 index 0000000000..875a6dccc5 --- /dev/null +++ b/devlog/_plan/260912_accounts/030_reset.md @@ -0,0 +1,16 @@ +# Add Codex reset-first through the canonical pool settings API + +Cycle reset; C3 scheduling. Independent of quota history and eligibility. Carry #4080 at ecf6b4e48a4c2992c296fada2caf6a8132313eaa, credited to Terry Tan . Its public diff is a design input, with mandatory canonical-contract adaptation below. Do not enable reset-first for Anthropic or generic OAuth pools. + +MODIFY source paths in #4080: `src/codex/routing.ts`, `src/codex/pool-rotation.ts`, `src/codex/auth-api.ts`, `src/types/config.ts`, `src/cli/account-extended.ts`, `src/cli/account.ts`; retain existing priority, eligibility, threshold, and healthy affinity. New Codex strategy sorts earliest FUTURE short/weekly reset after filtering, then usage and stable order. Unknown/elapsed reset is not preferred. Threshold zero disables usage filtering while retaining ordering; exhausted-account behavior remains existing safe fallback. + +Additional MODIFY `src/oauth/pool-settings-capability.ts` and `src/server/management/oauth-account-routes.ts`: use a Codex-specific parser that accepts reset-first; canonical PUT /api/pool/settings and GET normalization must preserve it. Generic/Anthropic parsers keep rejecting reset-first. Update `gui/src/account-pool-strategy.ts`, strategy controls/settings, `gui/src/pool-settings.ts` types, locale translations and config docs from #4080 for the canonical route. + +```diff +- strategy: normalizeAccountPoolStrategy(config.accountPoolStrategy) ++ strategy: normalizeCodexAccountPoolStrategy(config.accountPoolStrategy) +``` + +Field chain: CLI/GUI strategy creation → canonical PUT parser → config.accountPoolStrategy write → config load + canonical GET parser → pool rotation/preview/failover, CLI and GUI display. Audit every existing strategy comparison/default, not just the union. No schema migration or new dependency. Exact contributor diff remains `.tmp/accounts-20260912/pr4080.diff` during planning; changes are adapted to current callers before B. + +Extend regression sources for canonical PUT/GET/save/reload, legacy endpoint, non-Codex rejection, tied/missing/elapsed resets, threshold zero, priorities, affinity and failover. Existing #4080 test cases are retained/adapted. Update all source ownership docs; screenshot of final rendered strategy control is included with PR. Local suites/build/typecheck/install NOT RUN; final head hosted CI supplies proof. #3376 remains partial until history/capacity; monthly/Anthropic/latest-first scope is reported separately. diff --git a/devlog/_plan/260912_accounts/040_generic_family.md b/devlog/_plan/260912_accounts/040_generic_family.md new file mode 100644 index 0000000000..24ee77cfc8 --- /dev/null +++ b/devlog/_plan/260912_accounts/040_generic_family.md @@ -0,0 +1,16 @@ +# Scope generic quota evidence and cooldowns by model family + +Cycle generic-family; C3, foundation for lifecycle. Extend existing generic pool; do not stack Google-specific #2562/#3283. Current `src/oauth/generic-account-failover.ts:200` already activates kernel strategies; quota threshold remains unused. #4299 narrow head 4583f9793295f75d4bf69d0bfb0900a550bc05bd supplies family-ranking input; adapt with Co-authored-by: chilung when reused. + +MODIFY `src/oauth/account-quota-rank.ts`: optional requested-model context chooses matching Antigravity Gem/Cla windows only, preserving no-model existing behavior and conservative unknown evidence. MODIFY generic-account-failover.ts health key/eligibility/ranking/fill-first: quota cooldown key is provider+account+known family, auth failures remain global. Thread model context into both kernel fill-first headroom and quota branch. `autoSwitchThreshold` is consumed by quota selection; zero disables proactive usage threshold, not upstream exhaustion. Reactive quorum activation stays unchanged; provider/global flags affect proactive preference only. + +```ts +type GenericQuotaScope = "account" | "gemini" | "claude"; +type GenericSelectionContext = { modelId?: string; sessionKey?: string; now: number }; +``` + +MODIFY `src/server/responses/core.ts` initial preference and every generic retry site to pass actual routed modelId; snapshot admission remains guarded and uses account-matched routing metadata. No extra Lab import. Field chain: route.modelId creation → in-memory context only → no disk serialization → headroom and health scope consumers. No provider error body's arbitrary string is allowed as a family identifier; family mapping is bounded known model semantics. + +Extend existing generic failover and account quota rank tests, plus a real handleResponses regression with opposing Gem/Cla windows. Assert Claude quota cooldown leaves Gemini usable, global auth exclusion blocks both, unknown model remains conservative, threshold zero semantics, kernel fill-first consumes same family. Sync `src/oauth/`, `src/server/` ownership docs and operating config docs. Local runtime suites NOT RUN; hosted final cumulative lifecycle tip verifies the foundation. + +Reflection REF-01 accepted. GenericQuotaScope is account|gemini|claude. GenericSelectionContext carries modelId?, sessionKey?, now. Thread it through headroom/exhaustion/ranking/eligibility/fill-first/initial-preference/rotation/Retry-After. Global cooldown blocks every family; known-family cooldown blocks that family; unknown context considers all relevant cooldowns. Clear/reconcile removes every scope. Core anchors: initial 4350; rotation 5542,6480,6853,7587,7998; refresh 5420,7377; admission 4054,4149,4413. Revalidate anchors at each P. Sidecar uses its actual routed model identity; absence uses account-conservative quota scope. diff --git a/devlog/_plan/260912_accounts/041_generic_lifecycle.md b/devlog/_plan/260912_accounts/041_generic_lifecycle.md new file mode 100644 index 0000000000..322a2fcaea --- /dev/null +++ b/devlog/_plan/260912_accounts/041_generic_lifecycle.md @@ -0,0 +1,25 @@ +# Bind admitted generic accounts and classify recoverable failures + +Cycle lifecycle depends on generic-family. C4 credential/retry. Existing owners: `generic-account-failover.ts`, `oauth/store.ts`, `server/responses/core.ts:4054/4149/5537`, `oauth/anthropic-routing.ts:759` provides a commit-after-resolution precedent. + +MODIFY generic-account-failover.ts: bounded process-local conversation affinity keyed by provider/session identity, with explicit idle TTL and entry cap; no conversation id means no affinity. Look up only live non-reauth accounts, release on expiry/removal/credential-generation change or classified failure, bind admitted account after guarded snapshot application rather than proposal. Extend note-success/selection context to carry actual account generation. Preserve explicit account selectors and manual active selection semantics; never persist conversation bodies. + +MODIFY existing generic recovery branches in core.ts: retain one same-request retry budget and existing no-output replay boundary. Handle only post-refresh account authentication rejection and provider-classified quota/account 403; ordinary permission/region/policy 403 stays terminal. Classifier uses existing provider error/code owners, not arbitrary text heuristics. Auth failure marks only rejected credential generation unhealthy, quota failure records family cooldown. Retry snapshot token/project/routing metadata must all describe the chosen account. + +```ts +// The exact evidence-discriminated GenericOAuthFailure union is specified below. +``` + +No stored broad rotateOn flag is introduced until its every consumer is grounded; defaults express only classified safe recovery. Field chain: adapter response classifier → internal failure object → existing recovery dispatch (not persisted credentials) → health/affinity invalidation and request attempt recovery marker. Add a bounded explicit recovery-kind union only if required, update log normalizer/GUI label/serialization together. + +Tests: multi-turn affinity holds; removal/reauth/expiry releases; proposed stale credential never binds; post-refresh 401 rotates one account; quota-classified 403 rotates; unrelated 403 does not; no rotation after downstream output; budget exhaustion terminates; initial and continuation paths match. New test files join both layout registries. Credential threat model and draft details stay in `.tmp/`; public unit contains safe design only. All source ownership docs updated. Local tests NOT RUN; hosted final cumulative tip plus independent security review required. + +Reflection REF-01 overrides the earlier broad failure type and family-affinity key. Affinity is provider+session, not family: a model change retains account if eligible for that family, otherwise releases it. Cap 2048 entries, idle TTL 30 minutes, prune on write and deletion/generation change. Context now is passed from one request clock. The only classified failure union is: +```ts +type GenericOAuthFailure = + | {kind:"terminal"} + | {kind:"auth"; status:401; evidence:"post-refresh-401"; generation:string} + | {kind:"auth"; status:403; evidence:"provider-account-credential"; generation:string} + | {kind:"quota"; status:403|429; scope:"account"|"gemini"|"claude"; retryAfter?:string}; +``` +First 401 uses existing refresh. Refresh transport error never marks unhealthy. Provider-account 403 requires existing closed code classification; when absent it is terminal. One request-wide recovery budget and unconditional client-output committed marker prohibits new dispatch after any output, including continuation and sidecar. Preserve sidecar quota-only callback unless its existing error contract can carry authenticated post-refresh evidence; record terminal auth limitation rather than invent evidence. Snapshot generation type is revalidated against OAuthAccessSnapshot before implementation. diff --git a/devlog/_plan/260912_accounts/042_generic_health.md b/devlog/_plan/260912_accounts/042_generic_health.md new file mode 100644 index 0000000000..464e6ea17e --- /dev/null +++ b/devlog/_plan/260912_accounts/042_generic_health.md @@ -0,0 +1,13 @@ +# Show the existing pool's selection and health state + +Cycle generic-health depends on lifecycle. Existing pseudonymous account attribution at `src/providers/label.ts:41`, usage log serialization at `src/usage/log.ts:554`, summaries at `src/usage/summary.ts:1237` are reused, not recreated. + +MODIFY existing OAuth health DTO/projector and `src/server/management/oauth-account-routes.ts` to expose bounded selection reason and health/cooldown scope alongside per-account quota. MODIFY `gui/src/hooks/useProviderAccountPools.ts` typed account projection, shared current/all-account card renderer and locale catalogs for reason. CLI account status uses same closed reason. Aggregate pool counts by healthy/cooling/reauth and known/unknown quota; do not sum unlike family/window percentages into fictitious capacity. + +```ts +type GenericSelectionReason = "affinity" | "manual" | "quota" | "round-robin" | "fill-first" | "auth-failover" | "quota-failover"; +``` + +Creation: admitted generic selector; serialization: authenticated account DTO and required per-attempt usage history; deserialization: typed optional client fields; consumers: status/account card/aggregate counts. No raw user/account/credential identifiers added to logs. Tests cover missing legacy fields, successful recovery clearing error, removed accounts, family-specific cooldown display and stale response merge. Source/structure/user docs align. Local suites/build NOT RUN; hosted final cumulative tip and rendered account card required. + +Reflection REF-02 accepted: request-history reason is REQUIRED. Add optional accountSelectionReason/accountQuotaScope to PersistedUsageAttempt in src/usage/log.ts and normalize/serialize closed unions. Stamp after admission per attempt in src/server/request-log.ts and core, preserving prior attempts. Update request-history API/client detail renderer and all localized labels. Legacy rows omit safely. Regression source covers persistence/reload, multi-account retries, and no overwrite of earlier reason. diff --git a/devlog/_plan/260912_accounts/045_warmup.md b/devlog/_plan/260912_accounts/045_warmup.md new file mode 100644 index 0000000000..bd8d304f47 --- /dev/null +++ b/devlog/_plan/260912_accounts/045_warmup.md @@ -0,0 +1,21 @@ +# Schedule one zero-usage account activation durably + +Cycle warmup independent of generic lifecycle. Existing `src/codex/quota-auto-refresh.ts` persists reset-boundary activation in codexQuotaAutoRefresh (#3588); preserve it. `src/quota/reset-seen-store.ts` owns reset-observer baselines and deduplication only. `src/codex/warmup.ts` remains invocation owner. Stable reset-credit operation IDs already exist and need no replacement. + +Extend the existing activation scheduler/store with an explicit one-shot target timestamp for a selected zero-usage account, using codexQuotaAutoRefresh as specified below. Creation must be authenticated CLI/API with account identity, dueAt and stable operation handle; persist pending/running/completed state before dispatch. The exact schema and scheduler boundary are specified below. No live account warmup is executed in this task. + +Before: scheduler acts only on observed reset boundaries. After: a persisted one-shot request can activate a confirmed zero-usage eligible account at dueAt once, survives restart, and is cancelled/invalidated on account deletion or credential replacement. Never spend reset credits or infer user consent from login presence. Integration tests use injected clock/transport; assert duplicate submissions, restart, removal, failure/cancel, non-zero usage, and one dispatch at due time. All local suites NOT RUN. API/CLI contract and scheduler source owners updated; exact due-time semantics remain subject to source-grounded P revalidation. + +## Concrete scheduler contract + +Use `src/codex/quota-auto-refresh.ts:272` minute sweep and its existing `warmAccount` owner. MODIFY `src/types/config.ts:786` and strict `src/config.ts:931` entry schema with optional `oneShot: { operationId: string; dueAt: number; credentialGeneration: number; status: "pending" | "claimed" | "completed" | "uncertain" | "cancelled" | "failed" }`. This slice supports stored pool accounts only; native-main requires its separate ownership flow and is excluded. dueAt is finite milliseconds, future and within 30 days; operationId validated UUID. No new timer/store/service. Config persistence is the current scheduler authority, so claim synchronously with mutatePersistedConfig before dispatch; failure to persist causes no warmup. A claimed row after restart becomes uncertain and is not automatically retried. Completed/failed/cancelled state stays as one bounded row until explicit replacement; same operationId retries return that state. + +NEW dedicated strict handler `src/codex/warmup-schedule-api.ts` for PUT/GET/DELETE `/api/codex-auth/warmup-schedule` (account id request/query required), registered next to existing account routes. PUT validates current stored generation, non-paused/non-reauth/non-validation-pending pool membership and a fresh measured zero usage snapshot before writing. GET returns only operationId/dueAt/status; DELETE changes pending to cancelled and rejects claimed. CLI `ocx account warmup --at --operation-id ` is dispatched through existing `src/cli/account.ts` and `src/cli/account-auth.ts`; capability/help maps updated. + +At each due sweep, refresh stale quota first, then require all measured gating windows zero with no exhausted/unknown primary reading; recheck membership, current credential generation, plan eligibility and spending intent immediately before claim. Nonzero/mismatched/deleted accounts settle failed/cancelled without dispatch. One-shot does not enable recurring fiveHour/weekly booleans. At most one upstream attempt per operation: dispatch outcome settles completed/failed; crash after claim becomes uncertain for explicit operator reconciliation, never exactly-once success claimed. Recovery DTO/copy explains that claimed is not verified success. This avoids the impossible guarantee of atomically committing local config and remote spending. + +Field chain: strict API/CLI input→mutatePersistedConfig→strict config load→existing minute sweep→status read. New status values update every schema/consumer/default switch; deletion reconciliation removes account-owned schedule. Tests register both layout maps and cover API idempotence, nonzero/unknown quota, stale generation, restart pending versus claimed, failed persistence, cancellation and exactly one attempted dispatch with injected clock. + +Reflection REF-03: activation persistence is codexQuotaAutoRefresh, not reset-seen-store (observer only). Extend existing config-routes authenticated settings handling for schedule fields where possible; dedicated schedule handler delegates same validated mutation owner. Final oneShot status vocabulary is pending|claimed|completed|uncertain|cancelled|failed. Claimed on hydrate becomes uncertain; never automatic resend. Failed completion persistence retries the marker only, not upstream work. Concurrent recurring/one-shot due work shares one invocation under same eligible generation and uses same completion result. Tests add crash-after-claim/send, failed completion write, simultaneous due and cancellation during async metadata. This final vocabulary supersedes the earlier shorter type. + +A3 accepted: scheduled one-shot uses an explicit single-attempt option `allowModelFallback?: boolean` on CodexWarmupOptions in src/codex/warmup.ts. warmCodexAccount defaults remain unchanged; when false, propagate the first result and never enter FALLBACK_MODELS. Existing warmAccount passes false for a claimed one-shot (including shared recurring work); ordinary manual/recurring defaults retain existing bounded fallback. Test physical fetch call count on 400/404 and partial completion, not only warmAccount invocation count. diff --git a/devlog/_plan/260912_accounts/050_history.md b/devlog/_plan/260912_accounts/050_history.md new file mode 100644 index 0000000000..fab9e603be --- /dev/null +++ b/devlog/_plan/260912_accounts/050_history.md @@ -0,0 +1,23 @@ +# Retain bounded raw quota observations + +Cycle history; C3 persistence. Independent of reset-first strategy. Source: `src/codex/quota.ts:265` commits merged snapshots, `:678` persists latest-only, `:735` clears; `src/codex/quota-types.ts:1` defines quota windows. New history attaches only after writer-generation and native-main identity guards. No new dependency or optional subsystem import on the core path. + +MODIFY `src/codex/quota.ts`: extend version-1 quota cache with optional bounded per-account history; store fresh raw observation fields (not carried windows) alongside updatedAt, and preserve explicit window reset identity. Credits-only writes do not append samples. Hydrate only validated bounded numeric rows, ignore malformed input, and deep-copy returned arrays. Bound both per-account samples (200) and retained age (30 days). Clear/reconcile removes matching history; unknown legacy files yield empty history. A stale main writer cannot append; native identity change clears old main observations before accepting new ones. + +Before: +```ts +type QuotaDiskFile = { version: 1; quotas: Record; mainPolicyQuota?: MainPolicyQuota }; +``` +After: +```ts +type QuotaDiskFile = { version: 1; quotas: Record; mainPolicyQuota?: MainPolicyQuota; history?: Record }; +export function getAccountQuotaHistory(accountId: string): StoredAccountQuota[]; +``` + +MODIFY `src/codex/auth-api.ts` account quota DTO to expose requested bounded history through a protected read route, preserving existing DTO compatibility. MODIFY CLI account quota read path to support history display/JSON with existing management transport. No secret/claim/tag is recorded; account key is the same local cache key, never an upstream bearer. Add tests in the existing quota cache test owner (or register a new domain test in both layout maps), plus protected API/CLI contract cases. Sync all `src/codex/` ownership docs using relevant statement or a cross-link; configuration docs explain retention and that snapshots alone do not establish token capacity. + +Field chain: creation is guarded quota commit; serialization is existing atomic quota-cache writer; deserialization is bounded validated hydrate; consumers are copied history getter, authenticated API/CLI, then capacity in the next cycle. Acceptance: old cache compatibility; 201 observations retain 200; credits-only and stale generations append none; different reset windows stay distinguishable; main identity change and removal discard old rows; corrupt/unbounded disk input is ignored/bounded. Local runtime checks NOT RUN; hosted quota/API/CLI regression suite at final history/capacity tip. + +Reflection REF-04: fixed aggregate bounds: 64 account identities, 4096 rows, 2 MiB serialized history payload and 4 MiB whole cache read bound. During append/hydrate evict oldest observed rows, tie-break account key; prune accounts absent from authoritative roster. Never include dynamic raw account identities in logs. History retains actual per-window provenance (response-header or WHAM where available), reset boundary and window family; partial inherited values do not count. Overlarge/malformed cache read fails to empty history without blocking newest quota. Tests include many-account overflow, byte overflow, deterministic ties and remove/restart. + +A1 accepted: native main history is deliberately NOT hydrated from disk in this slice. It can be sampled in-process only after identity observation and cleared on identity change; persistence omits __main__. Pool history envelopes bind stable configured account identity and stored credential generation, pruning mismatches on hydrate. This avoids attributing offline identity replacements to an old main label. Acceptance explicitly covers main replacement while stopped and account-id reuse. Main cross-restart history remains a documented limitation; bounded durable history is provided for stored pool accounts. diff --git a/devlog/_plan/260912_accounts/060_capacity.md b/devlog/_plan/260912_accounts/060_capacity.md new file mode 100644 index 0000000000..931e0b6646 --- /dev/null +++ b/devlog/_plan/260912_accounts/060_capacity.md @@ -0,0 +1,18 @@ +# Estimate observed effective capacity without claiming an upstream limit + +Cycle capacity depends on history. Source: `src/usage/log.ts` already persists accountLogLabel, timestamp, reported/estimated usage and per-attempt attribution; `src/codex/account-label.ts` owns safe labels. Use those existing records instead of storing credentials or duplicating request attribution. + +NEW `src/codex/quota-capacity.ts`: a pure estimator receives copied raw history and account-attributed reported usage observations. For each short/weekly/monthly window, pair adjacent fresh percentage observations only when reset identity matches, time increases and percentage delta is positive. Sum reported token usage in that interval, count per-attempt records once, exclude estimated/local/unattributed usage and reset/refund crossings. Estimate tokens per full window as observedTokens * 100 / percentageDelta; aggregate defensible intervals with median and report sampleCount plus observed-token lower-bound caveat. No valid interval returns null, never zero or a fabricated capacity. Bounded scan is invoked on management request, never routing; estimation is informational and does not overrule live quota. + +```ts +export type CodexCapacityEstimate = { + window: "short" | "weekly" | "monthly"; + estimatedTokens: number; + sampleCount: number; + confidence: "observed-lower-bound"; +}; +``` + +MODIFY history read API/CLI projection to attach per-window estimates with sample count and caveat; expose an existing account-card detail surface only if it can be honestly rendered and verified. Field chain: pure estimator creation; API JSON serialization; existing typed CLI/client deserialization; explicit informational display consumers. No persisted estimate schema needed. Tests feed independently hand-calculated intervals, 0% delta, reset rollover, missing timestamps/identity, cross-account records, retries, estimated usage, and extreme numeric input. Sync quota/usage ownership docs and user configuration guidance. Full closure of #3376 requires both history and meaningful capacity; reset-first alone stays partial. Local suites NOT RUN; hosted final cumulative tip is the verifier. + +A2 accepted: use readUsageSnapshotForManagement; if truncatedPrefixBytes>0, entriesTruncated, entriesDropped>0, missing revision, or invalid timing then return insufficient-evidence with no estimate. Treat each request as interval [timestamp, timestamp+durationMs] (request-log.ts:1039/1072); include only requests wholly contained in a quota-observation interval. Boundary-spanning requests contribute nothing. For included requests count reported physical attempts matching the exact pool label once; do not count both request total and attempts. Without attempts accept request-level reported usage only with matching label and no recovery ambiguity. Native main is excluded from token capacity because its historical label cannot establish identity after replacement. Current pool logLabel must be unique; legacy fallback labels/id reuse require insufficient evidence unless continuity is proven by history generation. Same-reset positive deltas only. Hand-worked boundary-spanning, truncation, missing identity and retry rows are mandatory regression fixtures. diff --git a/devlog/_plan/260912_accounts/070_tun.md b/devlog/_plan/260912_accounts/070_tun.md new file mode 100644 index 0000000000..d37cd09632 --- /dev/null +++ b/devlog/_plan/260912_accounts/070_tun.md @@ -0,0 +1,23 @@ +# Preserve canonical transport and classify failed quota reads + +Cycle tun; C3 account diagnostic, independent dev branch. #3799 and #3872 are ancestors of baseline; no re-carry. Source: `src/providers/quota.ts:2795`, quota cache :1570/:2093/:2162; `src/server/management/oauth-account-routes.ts:325`; shared account view `ProviderAccountQuota.tsx:8`. + +MODIFY `src/providers/quota-types.ts` dependency-free contract: +```ts +export const QUOTA_FAILURE_CODES = ["account_unavailable", "access_denied", "rate_limited", "upstream_error", "redirect_blocked", "destination_blocked", "dns_failed", "timeout", "transport_error", "response_unusable"] as const; +export type QuotaFailureCode = typeof QUOTA_FAILURE_CODES[number]; +export function parseQuotaFailureCode(value: unknown): QuotaFailureCode | undefined { + return QUOTA_FAILURE_CODES.find(code => code === value); +} +// AccountQuotaFields gains quotaFailure?: QuotaFailureCode. +``` + +MODIFY `src/providers/quota.ts`: private classified Antigravity probe returns available quota+source or unavailable failure+legacy null/throw disposition. Public fetchAntigravityUsageQuota retains existing null/rejection behavior. Summary redirect/401/403 terminates without fallback; any other failure tries existing models fallback; final attempt determines category, successful fallback clears failure. Classify ProviderOutboundPolicyError→destination_blocked; DestinationDnsResolutionError→dns_failed; PinnedHttpError timeout codes→timeout, output_byte_limit→response_unusable; DOMException TimeoutError→timeout; remaining errors→transport_error. Never use message regexes. JSON/body failures stay response_unusable since readQuotaJson cannot distinguish timeout from malformed data. + +Before: cache failure stores `{ts, quota: lastGood, unavailable: true}`. After: adds only closed `quotaFailure`, no error object/message/body/URL. Credential/project preparation failures are account_unavailable, not reauth verdicts. Cache generation/inflight/TTL guards remain. Success constructs fresh entry without failure. Persist only existing quota projection; diagnostic codes remain transient. Extend account result and API projection only when unavailable; stale identity/config projection omits category. + +MODIFY GUI `components/provider-workspace/types.ts`, `hooks/useProviderAccountPools.ts`: parse incoming code; enriched success/pending clears it, roster-only refresh preserves it only for same id/mode, late merge explicitly copies it, local API failure clears old upstream diagnosis. `ProviderAuthPanel.tsx` forwards to `ProviderAccountQuota.tsx` for all accounts; current account whole-row pass-through stays intact. Add localized pws.quotaFailure keys to all locale files. CLI AccountRow/raw DTO/projector and quotaText show safe category, preserving generic fallback for unknown values. + +Field chain: private probe→transient cache→account results→authenticated API JSON→enum-normalized client/CLI→current/all-account quota text. Ranking/health/history do not consume it. Tests: each enum trigger, summary failure/fallback success, final-attempt precedence, stale bars, recovery, cross-account isolation, stale-config, late response, unknown wire code, and secret-free projection. Existing provider account quota fixtures supply transport injection; new files require both layout entries. Fix inventory's stale IPv6 proxy-only sentence and update every touched area owner. Local suites/build NOT RUN; hosted backend/GUI checks and rendered final-tip artifact. Authenticated TUN observation remains unmet until an authorized operator supplies exact SHA, proxy/TUN mode and sanitized successful refresh; no network/account changes here. + +Reflection TUN-R01/R02 accepted. HTTP 300–399→redirect_blocked, 401/403→access_denied, 429→rate_limited, other non-2xx→upstream_error; success with unusable quota→response_unusable. Keep providerRedirectError cancellation and discard its message. Neither status establishes plan or reauth. fetchAntigravityQuota may reuse the private probe preserving null/rejection and success source; ProviderQuota/ProviderQuotaReport gain no diagnostic field, report-only views remain generic. getCachedProviderAccountQuota returns last-good quota only. diff --git a/devlog/_plan/260912_accounts/080_reauth_api.md b/devlog/_plan/260912_accounts/080_reauth_api.md new file mode 100644 index 0000000000..10370f304f --- /dev/null +++ b/devlog/_plan/260912_accounts/080_reauth_api.md @@ -0,0 +1,29 @@ +# Reauthenticate the existing native main identity with device code + +Cycle reauth-api; C4, independent dev branch. Preserve /api/codex-auth/login rejection of __main__. Allow existing native-main credentials in every runtime role, with no codex binary/keyring requirement. Same-identity reauth only; account switching remains the native profile workflow. Main device service does not call startLoginFlow(chatgpt), whose completion persists into the OAuth store. + +MODIFY `src/oauth/chatgpt-device.ts`: factor the private grant exchange to retain raw validated token payload for a new native-only result. Existing loginChatGPTDevice still projects OAuthCredentials and returns no id_token. New loginChatGPTNativeDevice returns `{credential: OAuthCredentials, idToken: string}` only in process; reject missing access/refresh/id token or mismatched token account identity. Device callback remains human URL/code, opaque device_auth_id private. + +MODIFY `src/codex/main-account.ts`: new beginNativeMainReauth captures existing MainAuthJsonCredential snapshot into a closure without exposing it to callers; returned commit accepts complete native device tokens. At commit acquire existing withNativeMainExclusiveClaim after authorization, verify startup/recovery and in-process admission fence, assert original path/hash/inode before atomic rename, require same chatgpt account identity. Write access_token, refresh_token, id_token, account_id together, preserving allowed root metadata. Check cancellation/current-flow before entering commit and before rename. Advance mutation epoch and reconcile same-account runtime/quota/reauth state explicitly. Never retain the old identity token beside new credentials. No claim held during human polling. + +NEW `src/codex/main-device-reauth.ts`: one process-owned active flow, opaque UUID, AbortController and bounded terminal retention; injectable login/commit dependencies for tests. Start/status/cancel return only flowId, status, verificationUrl, deviceCode and closed safe failure code. Superseded/cancelled completion may not publish. Terminal data clears URL/code when no longer useful. No tokens/emails/raw account IDs in DTO/log/error. + +NEW `src/codex/main-device-reauth-api.ts`: dedicated handler for POST/GET/DELETE `/api/codex-auth/main/reauth-device`, exact opaque flow query for status/cancel, strict request keys and safe 400/404/409 errors. Register at existing management registry/auth handler boundary (read latest dispatch before B); existing management auth/origin/session controls remain authoritative. No CLI direct account file write. + +MODIFY `src/cli/account-main.ts`: `reauth --device [--no-wait]`, `reauth status --flow `, `reauth cancel --flow ` via same management API; reject extra args before start. Register capability/help and regenerate skill surface using source-only tooling if needed. Blocking wait bounded by service flow expiry; --no-wait returns handle/code and follow-up commands. + +Field chain: device token creation→native private commit only, never API serialization; flow DTO created by service→management JSON→CLI/GUI typed parsing→human code/status. Tests: same-account success without codex/keyring, wrong identity, missing token fields, cancelled late result, concurrent file replace/refresh/profile switch, pending recovery, atomic write failure, same-account quarantine clearing, no pool-row mutation, secret-free all routes, unauthorized endpoints. Security draft stays scratch; implementation and regression diff may be published. Sync all src/codex/src/oauth/src/cli/src/server ownership docs and public headless recovery instructions. Local suites/build/typecheck/install NOT RUN; hosted final API/UI tip and independent security review required. + +Reflection native publication contract: pin NativeProfileContext once. Use a short owner/shared operation for preparation; capture original bytes and dev/ino from the SAME opened descriptor using an additive snapshot variant of native-profile-store.ts readBounded, preserving no-follow/regular-file/size bounds and wrapper compatibility. Closure retains this original snapshot throughout login. Explicitly assertNativeMainOwner at preparation and commit; withNativeMainOwnerOperation tracks work but does not replace this assertion. Acquire exclusive claim after human authorization, then recheck recovery, flow, cancellation and ownership before rename. Missing owner/claim fails safely, no NativeProfileManager/keyring enrollment. Tests include same-byte replacement, capture-time replacement, in-place edits, deletion/nonregular/symlink, cancellation waiting for claim, missing owner and unsupported claim. + +```ts +type MainDeviceReauthStatus = + | {flowId: string; status: "pending"; verificationUrl: string; deviceCode: string} + | {flowId: string; status: "committing"} + | {flowId: string; status: "succeeded"; credentialUpdated: true} + | {flowId: string; status: "cancelled"} + | {flowId: string; status: "failed"; credentialUpdated?: true; code: "identity_mismatch" | "credential_changed" | "native_main_unavailable" | "device_authorization_failed" | "publication_failed" | "reconciliation_failed"}; +``` +Cancellation after publication returns succeeded, never cancelled. Post-publication reconciliation failure reports credentialUpdated=true/reconciliation_failed, no rollback claim or automatic retry. Tokens/snapshots stay private. Start waits for human-code publication or terminal result so pending always has URL/code. One active flow rejects overlapping start (409), terminal retention 5 minutes, grant deadline 15 minutes inherited from device owner. + +Reflection residual accepted: reconciliation_failed is a distinct DTO union member requiring credentialUpdated:true. Cancellation after publication preserves either succeeded or reconciliation_failed, never overwrites reconciliation failure and never reports cancelled. diff --git a/devlog/_plan/260912_accounts/090_reauth_ui.md b/devlog/_plan/260912_accounts/090_reauth_ui.md new file mode 100644 index 0000000000..9e32ba6c1b --- /dev/null +++ b/devlog/_plan/260912_accounts/090_reauth_ui.md @@ -0,0 +1,9 @@ +# Put native-main device reauth on the main card + +Cycle reauth-ui depends on reauth-api. C4 auth UI. Existing main-card uses only expired-token text at `gui/src/components/codex-account-pool-main-card.tsx:184`. Preserve pool Add/Re-login and native profile picker. + +NEW `gui/src/components/use-main-device-reauth.ts`: dedicated hook with start/poll/cancel methods using native-only namespace, flowId ownership and abort/unmount cleanup. Normalize closed status/error payloads; never accept arbitrary verification URLs (only known device verification destination from backend contract), no token/account-id fields. Poll only matching active flow and stop on terminal status; late responses from replaced flow ignored. + +MODIFY main-card component: button Re-login with device code; after start show known verification URL, human code/copy and polite pending status, cancel action; success refreshes main account state. Keep layout consistent with current card. Do not reuse AddCodexAccountModal or reauthAccountId=__main__. Add exact locale keys for all shipped languages, update prop owners/types and backend error copy. Terminal failure is actionable and safe; do not automatically retry login or switch identity. + +Field chain: dedicated API DTO→hook validated state→main-card only; no persistence of device code in browser storage. Existing parent refresh callback re-fetches main status on completion. Regression source verifies correct route, code display, cancel ownership, stale poll, success refresh, no pool Add invocation, keyboard and error states. Hosted rendered screenshots required for PR; obtain built artifacts from final hosted CI instead of local product build. Sync GUI owners and headless dashboard docs. Local tests/build NOT RUN. API→UI ordinary manual chain, merge reserved to coordinator. diff --git a/devlog/_plan/260912_accounts/100_final.md b/devlog/_plan/260912_accounts/100_final.md new file mode 100644 index 0000000000..709dc240d7 --- /dev/null +++ b/devlog/_plan/260912_accounts/100_final.md @@ -0,0 +1,7 @@ +# Verify final branch tips and hand off integration evidence + +Cycle final consumes every delivered PR. No product change unless a concrete hosted CI/reviewer finding justifies a new repair cycle. Refresh PR head/base/native membership, CI run head SHA, all jobs and outstanding review threads. GitHub-hosted final tips are the user's execution verifier; intermediate runs may exist but are not claimed as tested by this task. Do not cancel workflows or modify protection. + +MODIFY this unit's numbered evidence/closure record and task-local handoff: one row per original issue/PR with LIVE/PARTIAL/SUPERSEDED/NOOP and exact remaining acceptance; one row per new PR with URL/base/head SHA/commits/coauthor/manual-chain order; final hosted run IDs/URLs/conclusions and unresolved security/review/field acceptance. Local suites/typecheck/build/install NOT RUN. No merge or original issue closure. + +Conditional repair: download exact failing job log, identify cause, amend owning phase plan, implement smallest correction in a fresh PABCD cycle, push --no-verify and verify new final head. Source-only checks are labeled text checks, not suite evidence. After unchanged final tip's checks pass, stop retesting and collect final handoff. Do not mark goal complete while required implementation is absent. Field acceptance has separate evidence status and cannot be replaced by mocks. diff --git a/src/oauth/callback-server.ts b/src/oauth/callback-server.ts index dc49d5fcd2..a9d4b7f003 100644 --- a/src/oauth/callback-server.ts +++ b/src/oauth/callback-server.ts @@ -37,6 +37,14 @@ function errorHtml(message: string): string { export type CallbackResult = { code: string; state: string }; +/** Close every response so pooled sockets cannot send the next login to a retired flow. */ +function closingResponse(body: string, status: number, contentType = "text/html"): Response { + return new Response(body, { + status, + headers: { "Content-Type": contentType, "Connection": "close" }, + }); +} + /** * The redirect URI advertised to providers must stay `localhost` (it is what the OAuth * apps have registered), but Windows commonly resolves `localhost` to `::1` first while @@ -177,7 +185,7 @@ export abstract class OAuthCallbackFlow { #handleCallback(req: Request, expectedState: string): Response { const url = new URL(req.url); if (url.pathname !== this.callbackPath) { - return new Response("Not Found", { status: 404 }); + return closingResponse("Not Found", 404, "text/plain"); } const code = url.searchParams.get("code"); @@ -214,10 +222,7 @@ export abstract class OAuthCallbackFlow { }); } - return new Response(ok ? SUCCESS_HTML : errorHtml(errMessage), { - status: ok ? 200 : consumeFlow ? 500 : 400, - headers: { "Content-Type": "text/html" }, - }); + return closingResponse(ok ? SUCCESS_HTML : errorHtml(errMessage), ok ? 200 : consumeFlow ? 500 : 400); } #waitForCallback(expectedState: string): Promise { diff --git a/structure/providers/xai-grok.md b/structure/providers/xai-grok.md index d9155185fc..5442bc756e 100644 --- a/structure/providers/xai-grok.md +++ b/structure/providers/xai-grok.md @@ -15,6 +15,10 @@ Grounded in the open-sourced official client (xai-org/grok-build); unit + eviden `~/.grok/auth.json` (read-only) before any refresh and adopt a newer usable generation with zero IdP calls (`shouldAdoptGrokGeneration`, later-expiresAt authority); an IdP refresh detaches the credential to `source:"oauth"`. +- **Browser login callback:** Grok's browser login uses the shared `OAuthCallbackFlow` listener + on a per-provider FIXED loopback port, so every response it sends closes its connection. A + retired flow that kept a pooled socket would capture the NEXT login's callback and reject it + as a state mismatch; see `src/oauth/callback-server.ts`. - **Two-lock refresh transaction:** per-provider+account intent lock held across the IdP exchange plus a short global store-write lock + async mutation funnel around every `auth.json` load-merge-persist (`src/oauth/store.ts`); generation-guarded persist diff --git a/structure/runtime.md b/structure/runtime.md index 49a5fb6483..3db52c92d3 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -139,7 +139,7 @@ The server exposes `POST /api/stop` which restores native Codex config, stops an | --- | --- | | `src/providers/registry.ts` | Canonical provider presets for CLI, dashboard, OAuth, key providers, and metadata. | | `src/providers/derive.ts` | Enrichment from provider presets into user config. | -| `src/oauth/` | OAuth providers, token storage, refresh, and auth-token resolution. | +| `src/oauth/` | OAuth providers, token storage, refresh, and auth-token resolution. The login callback listener binds a per-provider FIXED loopback port, so consecutive logins reuse the same number; every response it sends ends its connection (`Connection: close`, including non-callback paths such as a stray `/favicon.ico` 404). Stopping the listener does not close an established socket, so without that a pooled client would deliver the next login's callback to the retired flow, which rejects the unknown state as a CSRF mismatch while the live flow waits. | | `src/adapters/openai-responses.ts` | Native OpenAI/ChatGPT Responses passthrough. | | `src/adapters/openai-chat.ts` | OpenAI-compatible Chat Completions bridge. | | `src/adapters/anthropic.ts` | Anthropic Messages bridge. | diff --git a/structure/transports/inventory.md b/structure/transports/inventory.md index 5acafbf63b..b4036699d6 100644 --- a/structure/transports/inventory.md +++ b/structure/transports/inventory.md @@ -20,6 +20,7 @@ surface is listed here so a maintainer can find the owner without grepping: | GitHub Copilot | `src/providers/xai-transport.ts` (`resolveProviderTransport`), `src/providers/github-copilot-transport.ts` | `resolveProviderTransport` selects the Copilot transport when the routed provider name is `github-copilot`; the Copilot module then resolves its headers and base URL, and the registry seeds the provider row and model fallback. | | API-key pools | `src/providers/api-key-selection.ts`, `src/providers/key-failover.ts` | A configured `apiKeyPoolStrategy` plus a cooling committed key rotates before the first send (`selectProactiveApiKeyTransport`); a 429 still rotates after the send and records a cooldown. `provider.apiKey` keeps mirroring the active entry so routing stays single-key. The pick is inert without a strategy or while the committed key is healthy. | | OAuth account failover | `src/oauth/generic-account-failover.ts`, `src/oauth/anthropic-routing.ts` | Reactive pre-output 429 recovery is presence-driven with 2+ eligible accounts. Pool and `oauthAccountFailover` flags govern proactive routing, not the reactive retry: a disabled Anthropic pool recovers through quota ordering rather than its dormant strategy, and a per-provider `enabled` beats the global default in either direction. | +| OAuth login callback (inbound) | `src/oauth/callback-server.ts` | Every response, including non-callback 404s, closes its connection so a pooled socket cannot deliver a later login to a retired flow on the same callback port. | | Alibaba regions | `src/providers/alibaba-region-backup.ts`, `src/providers/alibaba-region-migration.ts`, `src/providers/alibaba-region-startup.ts` | Region migration backs up before rewriting and is idempotent across restarts. | | Discovery and quota | `src/providers/model-discovery.ts`, `src/providers/quota.ts` | Discovery rejects a response over 4 MiB or past 2,000 raw rows before caching it. | diff --git a/tests/oauth/oauth-callback-server.test.ts b/tests/oauth/oauth-callback-server.test.ts index a327e1df0d..cd7a353a7f 100644 --- a/tests/oauth/oauth-callback-server.test.ts +++ b/tests/oauth/oauth-callback-server.test.ts @@ -1,5 +1,5 @@ import { describe, expect, test } from "bun:test"; -import { OAuthCallbackFlow } from "../../src/oauth/callback-server"; +import { OAuthCallbackFlow, type OAuthCallbackFlowOptions } from "../../src/oauth/callback-server"; import type { OAuthController, OAuthCredentials } from "../../src/oauth/types"; class TestFlow extends OAuthCallbackFlow { @@ -117,3 +117,108 @@ describe("OAuth callback server defaults", () => { } }); }); + + +/** A barrier-owned login: readiness follows callback registration, and cleanup always settles it. */ +function startCallbackLogin(options: OAuthCallbackFlowOptions, holdExchange?: Promise) { + const controller = new AbortController(); + const ready = Promise.withResolvers(); + let deadline: ReturnType | undefined; + class HeldFlow extends ManualFallbackFlow { + override async exchangeToken(code: string, state: string, redirectUri: string): Promise { + await holdExchange; + return super.exchangeToken(code, state, redirectUri); + } + } + const flow = new HeldFlow({ + signal: controller.signal, + onAuth: () => queueMicrotask(() => { + deadline = setTimeout(() => controller.abort(new Error("callback test deadline")), 5_000); + ready.resolve(); + }), + }, options); + const login = flow.login(); + void login.catch(error => ready.reject(error)); + return { + flow, login, ready: ready.promise, signal: controller.signal, + stop() { + if (deadline !== undefined) clearTimeout(deadline); + controller.abort(new Error("callback test cleanup")); + }, + }; +} + +async function availableCallbackPort(): Promise { + const server = Bun.serve({ hostname: "127.0.0.1", port: 0, reusePort: false, fetch: () => new Response("probe") }); + const port = server.port; + await server.stop(true); + return port; +} + +async function callbackResponse(port: number, path: string, signal: AbortSignal): Promise { + const response = await fetch(`http://127.0.0.1:${port}${path}`, { signal }); + await response.text(); + return response; +} + +for (const favicon of [false, true]) { + test(favicon + ? "a non-callback request cannot pin the socket to the retiring flow" + : "a retired flow cannot serve the next login on the same callback port", async () => { + const port = await availableCallbackPort(); + const options = { preferredPort: port, callbackPath: "/callback", callbackHostname: "127.0.0.1", callbackBindHostname: "127.0.0.1" }; + const exchange = Promise.withResolvers(); + const started: ReturnType[] = []; + try { + const first = startCallbackLogin(options, exchange.promise); + started.push(first); + await first.ready; + const firstState = first.flow.generated!.state; + const success = await callbackResponse(port, `/callback?code=first-code&state=${firstState}`, first.signal); + expect(success.status).toBe(200); + expect(success.headers.get("connection")).toBe("close"); + if (favicon) { + const stray = await callbackResponse(port, "/favicon.ico", first.signal); + expect(stray.status).toBe(404); + expect(stray.headers.get("connection")).toBe("close"); + } + exchange.resolve(); + await first.login; + const second = startCallbackLogin(options); + started.push(second); + await second.ready; + const secondState = second.flow.generated!.state; + expect(secondState).not.toBe(firstState); + const retired = await callbackResponse(port, `/callback?code=old-code&state=${firstState}`, second.signal); + expect(retired.status).toBe(400); + expect(retired.headers.get("connection")).toBe("close"); + const malformed = await callbackResponse(port, "/callback", second.signal); + expect(malformed.status).toBe(400); + expect(malformed.headers.get("connection")).toBe("close"); + const live = await callbackResponse(port, `/callback?code=second-code&state=${secondState}`, second.signal); + expect(live.status).toBe(200); + expect(live.headers.get("connection")).toBe("close"); + await second.login; + expect(second.flow.exchanged?.state).toBe(secondState); + } finally { + exchange.resolve(); + for (const flow of started) flow.stop(); + await Promise.allSettled(started.map(flow => flow.login)); + } + }, 15_000); +} + +test("provider errors close the callback response without retaining the socket", async () => { + const port = await availableCallbackPort(); + const flow = startCallbackLogin({ preferredPort: port, callbackHostname: "127.0.0.1", callbackBindHostname: "127.0.0.1" }); + try { + await flow.ready; + const response = await callbackResponse(port, `/callback?error=access_denied&state=${flow.flow.generated!.state}`, flow.signal); + expect(response.status).toBe(500); + expect(response.headers.get("connection")).toBe("close"); + await expect(flow.login).rejects.toThrow("Authorization failed"); + } finally { + flow.stop(); + await Promise.allSettled([flow.login]); + } +});