Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions devlog/_plan/260907_axis1_bugfixes/000_plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Axis 1: measured bug fixes and failure diagnostics

Archetype: satisfy existing contracts. Trigger: owner assigned axis 1 (#3809, #3464, #3661). Goal: deliver reviewable fixes through a manual PR chain and merge the verified scope. Non-goals: new account/retry policy, auth defaults, multipart recovery, releases, native stacks, sibling edits. Stop: merged feasible scope plus explicit unresolved dispositions. Escalation: defer a policy-dependent or unreproducible slice; reclaim a worker slice after two failed packets. Evidence: this unit plus ignored `.tmp/axis1/` and `.codexclaw` receipts. Resources: task-owned worktree/branches and GitHub repository access; Astra high leaves within host capacity; no caller-specified token or wall-clock budget.

Baseline: origin/dev 137d6a727; source PR #3809 at 4a1012359a522ddd6d7ff77203c9e5f3632d605c. Assigned 5cc8 checkout has pre-existing changes and remains untouched. Code lives in /tmp/ocx-axis1-20260907.

## Cycle map
1. wp0: docs-only scope, source audit and dependency roadmap; no runtime changes.
2. wp1: bounded quota, version-guidance and recovery-diagnostic changes; independent source/security review and structural checks. Runtime verification deferred explicitly to wp2.
3. wp2: publish ordinary PR chain, run final cumulative hosted CI, resolve findings, admin merge bottom-up and verify dev ancestry. Lower CI only if final CI fails.

## Delivery contract
The owner explicitly requests a manual delivery chain even where units are independent: quota -> CLI guidance -> recovery reasons, with each layer carrying its own tests and credit. This order is an integration order, not a fabricated runtime dependency. No native registration. Lower commits carry [skip ci] to defer duplicate workflow runs; final head does not. Skipped lower runs are never called passing. No local tests/typecheck/build suites and no hook-triggered suites; task pushes use --no-verify. Hosted ci.yml on the final head must cover all changed runtime/tests; lower-level runs are diagnostic only after final failure. Merge with --admin under the explicit owner exception; preserve original commits/trailers with merge commits, retarget each child to dev, and check integration trees against final evidence. Concurrent dev changes require fresh combined verification.

## Work boundaries
- Quota: src/providers/quota.ts, src/oauth/anthropic-routing.ts, src/oauth/health.ts, src/server/responses/core.ts, src/images/loop.ts, src/web-search/loop.ts, focused quota tests/layout, provider documentation.
- CLI: src/cli/version-skew.ts and relevant status/doctor consumers, tests/cli/cli-version-skew.test.ts, troubleshooting documentation. No service restart or repair behavior changes.
- Recovery: src/server/responses/agent-task-recovery.ts, agent-task-recovery-cache.ts, src/lib/bounded-body.ts and existing focused tests, Responses error projection if needed, recovery documentation. No expanded admission/retry.
- Main owns shared core.ts integration and test-layout files. Workers must not touch each other's paths or git index.

## Verification and acceptance
No local suite commands are executed. Source mapping, git diff --check and documentation structural checks are local evidence only. Hosted Cross-platform CI at final head provides runtime/typecheck/privacy and affected platform proof; inspect jobs for skipped coverage. Build completion is provisional until that run and independent audit succeed. Original PR author(s) must be named in commit Co-authored-by trailers, sourced from original commits/API; report authors may also be acknowledged accurately. Source-of-truth sync uses relevant existing structure and docs-site pages.
3 changes: 3 additions & 0 deletions devlog/_plan/260907_axis1_bugfixes/010_roadmap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# wp0: scope roadmap

Read current source, prior issue disposition and PR #3809 before choosing changes. Independent Astra high reviewers map each bounded issue. Confirm existing launcher behavior and bounded recovery reasons are already in dev; plan only residual fixes. Record exact file boundaries and acceptance scenarios in 020. Success: all three slices have verifiable requirements, main-owned shared files, original author anchors and explicit policy exclusions. Local evidence is documentation and source inspection; no runtime claim.
5 changes: 5 additions & 0 deletions devlog/_plan/260907_axis1_bugfixes/011_audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# wp0 audit disposition

Independent Astra high reviewer Hooke: VERDICT: GO-WITH-FIXES (blockers=1). Shared-flight failure propagation was the blocker. Accepted: 000/020 now assign cache and bounded-body ownership and define shared typed outcomes, success-only cache, caller-local cancellation and capacity semantics. Source scouts independently identified and confirmed these requirements. Fixed stale CLI test path. Windows runtime proof requires final workflow_dispatch, now explicit in 030.

No runtime code changed. Documentation source/ownership inspection and git diff --check are the wp0 evidence. Runtime verification remains wp2.
5 changes: 5 additions & 0 deletions devlog/_plan/260907_axis1_bugfixes/012_roadmap_lock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Roadmap lock

The second independent audit returned VERDICT: PASS with no remaining blockers. The three accepted slices are ready for scoped implementation. Original quota author: Éverton Toffanetto (everton-dgn), commit identity from 4f3779c04753 and 3ef0ade296c3. Issue reporters: garysassano (10464497) and Hu9956 (282876394). Reporter acknowledgement is separate from code authorship.

Preserve raw unequal version diagnostics. Detailed recovery outcomes must travel in the shared flight, not caller-local closures. Quota observations use immutable dispatch identity. Final verification is hosted workflow_dispatch for full Windows coverage; local suites remain prohibited.
20 changes: 20 additions & 0 deletions devlog/_plan/260907_axis1_bugfixes/020_bounded_fixes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# wp1: implement bounded bug fixes

## Quota
Carry only the source PR diff onto current dev, with original-author trailer. Header utilization fraction -> percentage; reset epoch -> timestamp. Creation: parser; serialization: account quota cache; deserialization: existing hydration; consumers: account ranking/health and management reading. Account-bound writer generation is captured with serving credentials, including retry/sidecar/continuation rebinds. Header observations merge model-specific windows and cannot indefinitely postpone probes. Existing 429 eligibility and retry count stay unchanged. Explicit reset evidence must not be truncated by an invented six-hour policy; any unresolved policy piece is deferred.
Scenarios: 200 and 429 on main/sidecar/continuation attribute only the serving account; generation invalidation discards writes; partial/malformed headers preserve known fields; no prior probe means model-window probe is still due; weekly rejected reset outlasts five-hour reset; absent evidence retains existing fallback. Verify with focused tests included in final hosted CI.

## Version guidance
Compare CLI and running proxy using existing semantic-version utilities if present. CLI newer points to service restart; proxy newer points to upgrading/PATH resolution of CLI; equal/unknown retain suppression; incomparable differing builds use neutral wording. status and doctor share advice. Preserve whether requests are allowed and do not perform repair. Test both directions, prereleases, placeholders, malformed versions and consumer projection.

## Recovery reasons
Keep existing public wrapper returning boolean and typed detailed result. Classify actual upstream HTTP refusal, transport error, timeout/caller cancellation, response-body/decode failures with a bounded vocabulary. Creation: request/collector; propagation: detailed recovery result; consumers: existing response reason projection/tests/docs. No raw upstream body/errors/tokens/ciphertext in output. Strict admission, one attempt, same credential and unchanged request mutation guarantees. Exercise each failure branch, cancellation races, malformed terminal output and successful recovery in final hosted CI.

Main owns src/server/responses/core.ts and layout metadata. Source/security review must check public boundaries and negative cases, not only implementation-mirroring tests. Source-only C evidence does not claim runtime correctness; wp2 is mandatory.

## Source-map clarification from independent #3464 research
Use src/lib/strict-semver.ts unchanged. Raw unequal versions remain skewed; equal precedence with different build metadata and invalid/whitespace/v-prefixed values get neutral wording, not normalization or a guessed direction. Placeholder suppression is unchanged. src/cli/doctor.ts must not call suppressed placeholders a confirmed match. Focused files: tests/cli/cli-version-skew.test.ts, tests/cli/cli-status-json.test.ts, tests/codex-integration/doctor.test.ts. Documentation: reference/cli/lifecycle.md and directly affected Korean/Russian pages. Existing launcher landed via #3616 (4e2246c32); no service runtime changes.

## Audit refinements
Quota: observe physical responses at the existing oauthDispatch boundary before any main/continuation replacement or return. Use immutable request binding to pair response with selected account; skip when final authorization headers do not prove that bearer or credentialGeneration has changed. An active-account switch alone does not invalidate another account's in-flight observation. Native Claude passthrough and single-account expansion remain outside #3809 carry. Preserve Retry-After precedence; only reject nonfinite/unrepresentable deadlines rather than invent an anomaly ceiling. Header-only rows are probe-due; hydrated Anthropic observations must be probe-due unless probe time is proven. Failed probes settle with the most recent committed observation for all joiners.
Recovery: worker owns agent-task-recovery-cache.ts and bounded-body.ts narrow decode discriminator alongside focused tests. Shared flight carries typed outcome, cache retains only success plaintext, cancelled waiters remain local. Recognized caller cancellation precedes owned timeout, which precedes decode/transport classification. Fatal UTF-8 discriminator must identify actual decoder exceptions without reclassifying fetch/body-reader TypeErrors. Rejected-response cancellation is nonblocking best effort. Keep current public wrappers and combo error projection. Update documented reason lists in structure/04_transports-and-sidecars.md and docs-site/reference/architecture.md.
7 changes: 7 additions & 0 deletions devlog/_plan/260907_axis1_bugfixes/030_delivery.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# wp2: hosted proof and manual-stack landing

Publish task-owned branches with --no-verify. Standard PR template, source links, truthful skipped-local/lower-CI disclosure and contributor trailers. Lower layers use [skip ci], final cumulative head runs existing Cross-platform CI; never modify shared workflow filters or fabricate checks. On final failure inspect failing jobs, fix owned defects, and only then use lower CI to localize ambiguity. Leave unrelated/unresolvable slices unmerged with evidence.

Before admin merge: source/security review findings resolved, final CI SHA/run pinned, current PR head and manual membership inspected. Record owner-authorized admin review/lower-CI exception. Merge bottom-up with original commits preserved; do not delete parent branches while children depend on them. Retarget child to dev after parent landing. Reconcile concurrent dev before claiming final integrated proof. Verify every merge SHA is ancestor of refreshed origin/dev. Close #3809 only after its accepted replacement scope lands; keep #3661 open for multipart/retry and #3464 open if broader original acceptance remains unresolved. No release/deploy.

Final full platform evidence uses workflow_dispatch ci.yml on the final cumulative branch, because ordinary PR CI excludes the Windows runtime job. Cancel only duplicate task-owned PR CI runs; skipped/cancelled runs are not passing evidence.
15 changes: 12 additions & 3 deletions docs-site/src/content/docs/guides/claude-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,18 @@ rotation does not protect against provider enforcement.

Operational contract when enabled:

- Upstream **429** cools that account using `Retry-After` when present (else a default backoff),
clears its affinities, and may rotate to another eligible account within the same request
(bounded).
- Upstream **429** cools that account, clears its affinities, and may rotate to another eligible
account within the same request (bounded). The cooldown uses a usable `Retry-After` when present,
otherwise the latest valid reset time among windows Anthropic marks `rejected`, including
weekly windows. Valid upstream deadlines are not shortened to a fixed cooldown ceiling.
A refusal with no usable deadline falls back to a 60-second default backoff.
- Responses report the serving account's 5-hour and weekly utilization, and whichever of those
two the response carries is recorded for that account — each window independently, and a
refusal counts as well as a success. Usage-aware selection works from ordinary traffic,
without waiting for a dashboard poll. Headers preserve model-specific quota windows and do
not postpone usage probes or clear a failed usage probe's unavailable status. Measurements
whose known reset time has passed are discarded as unknown, including retained model-specific
windows. Values without a known reset are preserved; missing data is never reported as zero usage.
- Affinity is **process-local** (lost on proxy restart).
- **401/403** credential failures quarantine the account (`needsReauth`) so it is excluded from
selection until re-authenticated.
Expand Down
25 changes: 21 additions & 4 deletions docs-site/src/content/docs/reference/configuration/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,14 +430,31 @@ rotation may trigger provider restrictions.
| `anthropicAccountPool.enabled?` | `boolean` | `false` | Enable sticky session affinity and quota-ranked new-session selection. **429 failover is not gated here**: it activates whenever two or more usable accounts are stored, exactly like every other multi-credential provider, and cannot be switched off. |
| `anthropicAccountPool.autoSwitchThreshold?` | `number` | `80` | For new sessions, when the active account reaches this threshold, choose the lowest known cached usage in the configured window; the account chosen does not itself have to be at or above the threshold. `0` disables **proactive** usage-based switching only — new-session selection and routing recovery after an eligible 429 still consult `quotaWindow`. |
| `anthropicAccountPool.strategy?` | `"quota" \| "round-robin" \| "fill-first"` | `"quota"` | New-session strategy; `quota` ranks accounts by the window set by `quotaWindow`, and `fill-first` evaluates its drain threshold in that same window. |
| `anthropicAccountPool.quotaWindow?` | `"five-hour" \| "weekly" \| "max-utilization"` | `"five-hour"` | The cached provider-reported utilization bar used for usage-aware account selection. `five-hour` keeps the original behavior. `weekly` scores the weekly bar and skips accounts whose 5-hour bar is exhausted while another eligible account remains, but falls back to exhausted candidates when none do. `max-utilization` scores the highest known bar, so it can use 5-hour usage before weekly usage is available; if neither is known, the account follows unknown-usage ordering. Known usage ranks before unknown usage under the opt-in `weekly` and `max-utilization` windows only; an omitted or explicit `five-hour` preserves the legacy ordering. If every eligible account is unknown, selection still returns one in eligible order. After the documented lower-5-hour tie-break, exact ties preserve eligible order. A healthy affinity-bound session is not proactively rebalanced. For new-session assignment and routing recovery after an eligible 429 replacement, `quota` ranks eligible candidates directly with this window; `fill-first` advances in stable order using this window's threshold and exhaustion rules; `round-robin` ignores it. Cooldown, failover limits, and reauthentication eligibility remain separate local state. Per-account weekly bars are only known once the dashboard Providers page has polled them. |
| `anthropicAccountPool.quotaWindow?` | `"five-hour" \| "weekly" \| "max-utilization"` | `"five-hour"` | The cached provider-reported utilization bar used for usage-aware account selection. `five-hour` keeps the original behavior. `weekly` scores the weekly bar and skips accounts whose 5-hour bar is exhausted while another eligible account remains, but falls back to exhausted candidates when none do. `max-utilization` scores the highest known bar, so it can use 5-hour usage before weekly usage is available; if neither is known, the account follows unknown-usage ordering. Known usage ranks before unknown usage under the opt-in `weekly` and `max-utilization` windows only; an omitted or explicit `five-hour` preserves the legacy ordering. If every eligible account is unknown, selection still returns one in eligible order. After the documented lower-5-hour tie-break, exact ties preserve eligible order. A healthy affinity-bound session is not proactively rebalanced. For new-session assignment and routing recovery after an eligible 429 replacement, `quota` ranks eligible candidates directly with this window; `fill-first` advances in stable order using this window's threshold and exhaustion rules; `round-robin` ignores it. Cooldown, failover limits, and reauthentication eligibility remain separate local state. Per-account weekly bars come from usage probes or observed response headers. |
| `anthropicAccountPool.stickyLimit?` | `number` | `1` | Successful new-session binds retained on one round-robin selection. Range 1–100. |

When enabled, 429 records bounded cooldown from `Retry-After` or a default backoff and may rotate
within the request. Affinity is process-local and size-bounded. Credential 401/403 marks the account
as needing reauthentication. If all eligible accounts are cooling, clients receive 429 with
When enabled, 429 records a cooldown and may rotate within the request. The cooldown length comes
from a usable `Retry-After`, otherwise from the latest valid reset time among rate-limit windows
Anthropic reports as `rejected`, including weekly windows. Valid upstream deadlines are not
shortened to a fixed cooldown ceiling; non-finite or unrepresentable deadlines are ignored.
A refusal with no usable deadline falls back to a 60-second default backoff. Affinity is process-local
and size-bounded. Credential 401/403 marks the account as needing reauthentication. If all eligible accounts are cooling, clients receive 429 with
`Retry-After` when known, not an authentication error.

Anthropic responses also report the serving account's 5-hour and weekly utilization, and whichever
of those two a given response carries is recorded against that account — each window independently,
on refusals as well as successes. Usage-aware selection therefore works from the accounts you
actually use, without waiting for the dashboard Providers page to poll them. These readings refresh
the existing row rather than replacing it, so the model-scoped weekly bars that only the usage
endpoint reports are preserved until their known reset time passes. Expired measurements become
unknown, including retained standard windows omitted by later headers. A reset-only header cannot
extend an older utilization measurement. Values with no known reset retain their existing behavior;
missing measurements are never replaced with zero usage.

Header observations do not postpone usage probes or clear a failed
probe's unavailable status. After restart, cached Anthropic observations remain available while
the next quota read probes again, because the saved observations do not include the probe clock.

:::caution[Experimental]
Leave this disabled unless you understand Anthropic account policy risk. Prefer manual
`ocx account use anthropic <id>` switching when unsure.
Expand Down
2 changes: 2 additions & 0 deletions scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,8 @@
"anthropic-image-retry.test.ts": "adapters/anthropic",
"anthropic-pool-toggle-copy.test.ts": "adapters/anthropic",
"anthropic-quorum-cache.test.ts": "routing",
"anthropic-quota-dispatch.test.ts": "adapters/anthropic",
"anthropic-ratelimit-headers.test.ts": "adapters/anthropic",
"anthropic-reasoning.test.ts": "adapters/anthropic",
"anthropic-sidecar-account-failover.test.ts": "adapters/anthropic",
"anthropic-stream-hardening.test.ts": "adapters/anthropic",
Expand Down
Loading
Loading