Skip to content

refactor(oauth): move the pool rotation kernel out of the Codex namespace #4279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lidge-jun merged 3 commits into from
Sep 11, 2026
Merged

refactor(oauth): move the pool rotation kernel out of the Codex namespace #4279

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
80d4a31
refactor(oauth): move the pool rotation kernel out of the Codex names…
codex Sep 11, 2026
3015be8
docs(devlog): record the second-half audit findings for the pool kernel
codex Sep 11, 2026
e1e8b25
docs(devlog): re-verify phase 3 anchors and record what blocks it
codex Sep 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions devlog/_plan/260911_account_pool_unification/020_phase2_shared_kernel.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,42 @@ would have forced an edit inside lane L3's freeze. The second was that dropping
which requires `pool.kernel` to default off and the old behaviour to be exactly
restorable.

## Second-half audit (the flagged behaviour change)

The extraction shipped as PR #4279. A separate audit of the remaining half returned
FAIL, and its findings change that half materially. Recorded here so the next cycle
starts from them rather than rediscovering them.

1. **BLOCKER. Branching the final ranking expression is not enough.**
`preferredInitialAccount` encodes the quota strategy BEFORE its tail: the
healthy-active early return tests `isAccountQuotaExhausted` (:262) and the
roster-wide `hasHeadroomEvidence` check (:272) returns null when a provider has
no quota data at all. Leave those untouched and round-robin can never run for a
provider without quota evidence, and fill-first never reaches
`autoSwitchThreshold` because the healthy active account already returned. Both
guards have to be strategy-gated: skip the evidence requirement for round-robin,
and use the threshold rather than exhaustion for fill-first.
2. **BLOCKER. The preference must peek, not pick.**
`pickRoundRobinAccount` mutates live ring state, but
`preferredInitialAccount` is explicitly a discardable proposal that the caller
drops on a resolver throw or a missing project. Mutating there desyncs the
cursor against requests that never happened. Use `peekRoundRobinAccount` and
mutate with `pickRoundRobinAccount` plus `notePoolRotationSuccess` only after
the selection is admitted, which is what Anthropic already does.
3. **The 429 path is safe to branch but fill-first must still move.** That tail has
no evidence guard, so a strategy branch is structurally fine. Fill-first there
cannot mean keep-active: the account that just returned 429 is already cooled,
so staying put would skip rotation entirely.
4. **`stickyLimit` does not exist for the generic kind yet.** The
`oauthAccountFailover` type carries only `enabled`, `strategy` and
`autoSwitchThreshold`. Lifting the 400 at `oauth-account-routes.ts:395` before
adding the field to the type, the DTO, GET and the PUT writer would accept a
value and then drop it. The kernel default is 1.
5. **The flag lands in a lane-owned file.** `OcxConfig` has no `pool` key today,
so `pool.kernel` belongs in `src/types/config.ts` (around :363) - which lane L3
owns. This half therefore inherits the same freeze as work-phases 1 and 2 until
that ownership clears, or the flag needs a different home.

- `tests/codex-integration/codex-pool-rotation.test.ts` — unchanged behaviour
through the re-export (`pickRoundRobinAccount` `:270`, `selectPriorityTier` `:111`)
- `tests/oauth/generic-oauth-failover.test.ts` — a configured strategy changes the
Expand Down
32 changes: 32 additions & 0 deletions devlog/_plan/260911_account_pool_unification/030_phase3_cache_affinity.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,35 @@ passing the threshold, because a threshold rebind throws away a warm cache.
A cache-affine account is chosen over a higher-headroom one; an exhausted affine
account still yields; concurrent distinct sessions keep distinct accounts; a
shared-cohort cache key does not collapse every session onto one account.

## Staleness re-verification and why this phase is not open yet

Re-verified at the wp3 P entry against `origin/dev` `1da8dae96`. Every anchor this
document relies on is unchanged from the original reading:

| Symbol | File | Line |
|---|---|---|
| `CODEX_THREAD_AFFINITY_MAX_ENTRIES` | `src/codex/routing.ts` | 135 |
| `pruneLruThreadAffinities` | `src/codex/routing.ts` | 1212 |
| `reevaluateAffinityQuota` | `src/codex/routing.ts` | 1942 |
| `MAX_AFFINITY_ENTRIES` | `src/oauth/anthropic-routing.ts` | 48 |
| `anthropicSessionKeyFromParts` | `src/oauth/anthropic-routing.ts` | 877 |
| `promptCacheKeyIsSharedCohort` | `src/oauth/anthropic-routing.ts` | 883 |
| `MAX_CACHE_BREAKPOINTS` | `src/adapters/anthropic.ts` | 60 |

The design is therefore current. Two things still stop this phase from opening,
and neither is a documentation gap:

1. **Its three open assumptions are genuine product decisions, not research gaps.**
The affinity key shape, what to do when a `prompt_cache_key` looks like a shared
cohort, and whether to add a minimum-token cache gate all change observable
behaviour and none is settled by reading the code. They need a human answer.
Under an active goal the Interview is suppressed, so this phase cannot resolve
them from inside the loop.
2. **The Codex half is frozen.** `src/codex/routing.ts` carries three of the seven
anchors above and is owned by lane L3 for the dispatch round in flight.

The Anthropic and generic halves are not frozen, so a narrower first slice exists:
unify the affinity key for those two kinds only, leaving the Codex thread-affinity
map on its current key until the freeze lifts. That slice still needs assumption 1
answered, which is why this phase stays closed rather than being re-scoped now.
1 change: 1 addition & 0 deletions scripts/test-layout/layout.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -665,6 +665,7 @@
"gemini-inline.test.ts": "images",
"gemini-web-search.test.ts": "adapters/google",
"generic-oauth-failover.test.ts": "oauth",
"pool-kernel-generic-sweep.test.ts": "oauth",
"github-copilot-account-origin.test.ts": "providers/github-copilot",
"github-copilot-oauth.test.ts": "providers/github-copilot",
"github-copilot-sse-rewrite.test.ts": "providers/github-copilot",
Expand Down
300 changes: 8 additions & 292 deletions src/codex/pool-rotation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,295 +1,11 @@
import type { OcxAccountPoolRotationStrategy } from "../types";
import type { GenerationContext } from "../lib/state-store-sweeper";

export const POOL_KEY_CODEX = "codex";
export const POOL_KEY_ANTHROPIC = "anthropic";

interface SelectionState {
activeKey?: string;
successes: number;
currentWeights: Map<string, number>;
}

const selectionState = new Map<string, SelectionState>();
let lastReconciledGeneration = 0;

const DEFAULT_STICKY_LIMIT = 1;
const MIN_STICKY_LIMIT = 1;
const MAX_STICKY_LIMIT = 100;
const DEFAULT_STRATEGY: OcxAccountPoolRotationStrategy = "quota";
const VALID_STRATEGIES = new Set<OcxAccountPoolRotationStrategy>(["quota", "round-robin", "fill-first"]);

/** Selection order for an account with no stored preference: one flat tier. */
export const DEFAULT_ACCOUNT_PRIORITY = 0;
export const MIN_ACCOUNT_PRIORITY = -100;
export const MAX_ACCOUNT_PRIORITY = 100;

/** Strict parse for management APIs — returns null instead of defaulting. */
export function parseAccountPoolStrategy(raw: unknown): OcxAccountPoolRotationStrategy | null {
if (typeof raw === "string" && VALID_STRATEGIES.has(raw as OcxAccountPoolRotationStrategy)) {
return raw as OcxAccountPoolRotationStrategy;
}
return null;
}

/** Strict parse for management APIs — returns null instead of defaulting. */
export function parseAccountPoolStickyLimit(raw: unknown): number | null {
if (typeof raw === "number" && Number.isInteger(raw) && raw >= MIN_STICKY_LIMIT && raw <= MAX_STICKY_LIMIT) {
return raw;
}
return null;
}

export function normalizeAccountPoolStrategy(raw: unknown): OcxAccountPoolRotationStrategy {
return parseAccountPoolStrategy(raw) ?? DEFAULT_STRATEGY;
}

export function normalizeAccountPoolStickyLimit(raw: unknown): number {
return parseAccountPoolStickyLimit(raw) ?? DEFAULT_STICKY_LIMIT;
}

/** Strict parse for management APIs — returns null instead of defaulting. */
export function parseAccountPriority(raw: unknown): number | null {
if (
typeof raw === "number"
&& Number.isInteger(raw)
&& raw >= MIN_ACCOUNT_PRIORITY
&& raw <= MAX_ACCOUNT_PRIORITY
) {
return raw;
}
return null;
}

export function normalizeAccountPriority(raw: unknown): number {
return parseAccountPriority(raw) ?? DEFAULT_ACCOUNT_PRIORITY;
}

/**
* Narrow an already-eligible account list to the highest selection-order tier that
* still has usable quota. Priority is an *ordering* boundary layered on top of
* eligibility: it never admits an account the caller already filtered out, and it
* never keeps the pool on a tier whose every member is drained.
* Compatibility shim. The rotation primitives moved to `src/oauth/pool-kernel.ts`
* so every credential kind can share them, not only Codex and Anthropic.
*
* Contract (each clause is load-bearing for "no behavior change when unconfigured"):
* - one distinct priority across `ids` (the unconfigured case) returns `ids` unchanged,
* so today's pick sequence is preserved byte for byte;
* - input order is preserved inside the returned tier, which keeps the `__main__`
* head-of-list bias and the first-index tie-break used by SWRR/lowest-usage;
* - every tier drained returns `ids` unchanged, reproducing today's
* stay-put-until-429 behavior rather than inventing a pick;
* - a `pinnedId` that is present *and* has headroom lowers the ceiling to its own
* tier, which is what makes a manual "use this now" survive round-robin and
* fill-first without any mutable selection state. A drained or absent pin is
* ignored, so the pin expires on its own once the account crosses the threshold.
* This file stays because the move is behaviour-preserving and its importers are
* spread across files that other work owns right now. Re-exporting keeps
* `routing.ts`, `auth-api.ts`, `account-priority.ts` and
* `state-store-registrations.ts` on their existing import path, so the extraction
* lands without editing any of them.
*/
export function selectPriorityTier(
ids: readonly string[],
priorityOf: (id: string) => number,
hasHeadroom: (id: string) => boolean,
pinnedId?: string,
): readonly string[] {
// Readonly out as well as in: the no-change cases return the caller's own array, so a
// mutating caller would corrupt its input in exactly the cases that must not change.
const list = ids;
if (list.length <= 1) return list;

const priorities = list.map(priorityOf);
const firstPriority = priorities[0]!;
if (priorities.every(priority => priority === firstPriority)) return list;

let ceiling = Number.POSITIVE_INFINITY;
if (pinnedId !== undefined) {
const pinnedIndex = list.indexOf(pinnedId);
if (pinnedIndex >= 0 && hasHeadroom(pinnedId)) ceiling = priorities[pinnedIndex]!;
}

const tiers = [...new Set(priorities)].sort((a, b) => b - a);
for (const tier of tiers) {
if (tier > ceiling) continue;
const members = list.filter((_, index) => priorities[index] === tier);
if (members.some(hasHeadroom)) return members;
}
return list;
}

function getOrCreateState(poolKey: string): SelectionState {
let state = selectionState.get(poolKey);
if (!state) {
state = { successes: 0, currentWeights: new Map() };
selectionState.set(poolKey, state);
}
return state;
}

function cloneSelectionState(state: SelectionState): SelectionState {
return {
activeKey: state.activeKey,
successes: state.successes,
currentWeights: new Map(state.currentWeights),
};
}

function smoothWeightedIndex(ids: readonly string[], state: SelectionState): number {
let best = -1;
let bestScore = Number.NEGATIVE_INFINITY;
let total = 0;
const weight = 1;
for (let i = 0; i < ids.length; i++) {
const id = ids[i]!;
const score = (state.currentWeights.get(id) ?? 0) + weight;
state.currentWeights.set(id, score);
total += weight;
if (score > bestScore) {
best = i;
bestScore = score;
}
}
if (best >= 0) {
const key = ids[best]!;
state.currentWeights.set(key, (state.currentWeights.get(key) ?? 0) - total);
}
return best;
}

/**
* Shared pick core. Mutates `state` the same way live resolve does; callers pass
* either the live map entry or a scratch/clone for dry-run peek.
*/
function pickRoundRobinFromState(
eligibleIds: readonly string[],
stickyLimit: number,
state: SelectionState,
commitSticky: boolean,
): string | null {
if (eligibleIds.length === 0) return null;

const limit = normalizeAccountPoolStickyLimit(stickyLimit);

if (state.activeKey && eligibleIds.includes(state.activeKey)) {
return state.activeKey;
}

if (state.activeKey) {
delete state.activeKey;
state.successes = 0;
}

const index = smoothWeightedIndex(eligibleIds, state);
if (index < 0) return null;

const picked = eligibleIds[index]!;
if (commitSticky && limit > 1) {
state.activeKey = picked;
state.successes = 0;
}
return picked;
}

export function pickRoundRobinAccount(
poolKey: string,
eligibleIds: readonly string[],
stickyLimit: number,
): string | null {
return pickRoundRobinFromState(eligibleIds, stickyLimit, getOrCreateState(poolKey), true);
}

/**
* Dry-run of {@link pickRoundRobinAccount}: returns the same account resolve would
* pick without advancing ring weights, activeKey, or successes.
*/
export function peekRoundRobinAccount(
poolKey: string,
eligibleIds: readonly string[],
stickyLimit: number,
): string | null {
const live = selectionState.get(poolKey);
const scratch = live
? cloneSelectionState(live)
: { successes: 0, currentWeights: new Map<string, number>() };
return pickRoundRobinFromState(eligibleIds, stickyLimit, scratch, false);
}

export function notePoolRotationSuccess(
poolKey: string,
accountId: string,
stickyLimit: number,
): void {
const limit = normalizeAccountPoolStickyLimit(stickyLimit);
const state = selectionState.get(poolKey);
if (!state) return;
if (state.activeKey !== accountId) {
state.activeKey = accountId;
state.successes = 0;
}
state.successes += 1;
if (state.successes >= limit) {
delete state.activeKey;
state.successes = 0;
}
}

export function notePoolRotationFailure(poolKey: string, accountId: string): void {
const state = selectionState.get(poolKey);
if (state?.activeKey === accountId) {
delete state.activeKey;
state.successes = 0;
}
}

/**
* Force the next sticky/RR pick onto `accountId` (manual dashboard selection).
* Clears sticky success counters and ring weights so the seeded account is held
* for the next new-session pick before ordinary rotation resumes.
*/
export function seedPoolRotationAccount(poolKey: string, accountId: string): void {
const state = getOrCreateState(poolKey);
state.activeKey = accountId;
state.successes = 0;
state.currentWeights.clear();
}

export function clearPoolRotationState(poolKey?: string): void {
if (poolKey === undefined) {
selectionState.clear();
return;
}
selectionState.delete(poolKey);
}

export function reconcilePoolRotationState(context: GenerationContext): number {
if (context.generation <= lastReconciledGeneration) return 0;
const anthropicIds = new Set<string>();
for (const key of context.oauthAccountKeys) {
const separator = key.indexOf("\0");
if (separator > 0 && key.slice(0, separator) === "anthropic") {
anthropicIds.add(key.slice(separator + 1));
}
}
let removed = 0;
for (const [poolKey, state] of selectionState) {
const valid = poolKey === POOL_KEY_ANTHROPIC
? anthropicIds
: poolKey === POOL_KEY_CODEX || poolKey.startsWith(`${POOL_KEY_CODEX}:`)
? context.codexAccountIds
: null;
if (!valid) continue;
if (valid.size === 0) {
selectionState.delete(poolKey);
removed += 1;
continue;
}
if (state.activeKey && !valid.has(state.activeKey)) {
delete state.activeKey;
state.successes = 0;
removed += 1;
}
for (const accountId of state.currentWeights.keys()) {
if (valid.has(accountId)) continue;
state.currentWeights.delete(accountId);
removed += 1;
}
}
lastReconciledGeneration = context.generation;
return removed;
}
export * from "../oauth/pool-kernel";
Loading
Loading