diff --git a/devlog/_plan/260912_accounts/060_capacity.md b/devlog/_plan/260912_accounts/060_capacity.md index f4c5acb023..d222802811 100644 --- a/devlog/_plan/260912_accounts/060_capacity.md +++ b/devlog/_plan/260912_accounts/060_capacity.md @@ -2,14 +2,14 @@ Cycle capacity depends on history. Source: `src/usage/log.ts` already persists accountLogLabel, timestamp, reported/estimated usage and per-attempt attribution; `src/codex/account-label.ts` owns safe labels. Use those existing records instead of storing credentials or duplicating request attribution. -NEW `src/codex/quota-capacity.ts`: a pure estimator receives copied raw history and account-attributed reported usage observations. For each short/weekly/monthly window, pair adjacent fresh percentage observations only when reset identity matches, time increases and percentage delta is positive. Sum reported token usage in that interval, count per-attempt records once, exclude estimated/local/unattributed usage and reset/refund crossings. Estimate tokens per full window as observedTokens * 100 / percentageDelta; aggregate defensible intervals with median and report sampleCount plus observed-token lower-bound caveat. No valid interval returns null, never zero or a fabricated capacity. Bounded scan is invoked on management request, never routing; estimation is informational and does not overrule live quota. +NEW `src/codex/quota-capacity.ts`: a pure estimator receives copied raw history and account-attributed reported usage observations. For each short/weekly/monthly window, pair adjacent fresh percentage observations only when reset identity matches, time increases and percentage delta is positive. Sum reported token usage in that interval, count per-attempt records once, exclude estimated/local/unattributed usage and reset/refund crossings. Estimate tokens per full window as observedTokens * 100 / percentageDelta; aggregate defensible intervals with median and report sampleCount plus an explicit low-confidence inference caveat. No valid interval returns null, never zero or a fabricated capacity. Bounded scan is invoked on management request, never routing; estimation is informational and does not overrule live quota. ```ts export type CodexCapacityEstimate = { window: "short" | "weekly" | "monthly"; estimatedTokens: number; sampleCount: number; - confidence: "observed-lower-bound"; + confidence: "low"; }; ``` @@ -18,3 +18,15 @@ MODIFY history read API/CLI projection to attach per-window estimates with sampl A2 accepted: use readUsageSnapshotForManagement; if truncatedPrefixBytes>0, entriesTruncated, entriesDropped>0, missing revision, or invalid timing then return insufficient-evidence with no estimate. Treat each request as interval [timestamp, timestamp+durationMs] (request-log.ts:1039/1072); include only requests wholly contained in a quota-observation interval. Boundary-spanning requests contribute nothing. For included requests count reported physical attempts matching the exact pool label once; do not count both request total and attempts. Without attempts accept request-level reported usage only with matching label and no recovery ambiguity. Native main is excluded from token capacity because its historical label cannot establish identity after replacement. Current pool logLabel must be unique; legacy fallback labels/id reuse require insufficient evidence unless continuity is proven by history generation. Same-reset positive deltas only. Hand-worked boundary-spanning, truncation, missing identity and retry rows are mandatory regression fixtures. P future refinement from history sidecar: do not call estimate a mathematical lower bound. It is an observed effective token estimate under rounded/delayed quota and local coverage assumptions. Admit only single-send reported nonestimated attempts; present-but-empty attempt arrays cannot fall back to parent totals. Deduplicate requestId+ordinal and reject conflicting duplicates. Use interval (left,right] with whole request containment to avoid zero-duration double counting. Existing parser can skip malformed rows without a rejected counter: report retained-valid-ledger-only assumption explicitly or add rejected-row metadata before claiming complete coverage. Loglabel alone is not history identity; history publication UUID and current stable unique configured label must bind sample period. All source tests remain hosted-only. + +## Resumed capacity contract + +Depends on history PR4404/0d98205fcd. Add pure quota-capacity.ts estimator receiving public sanitized observations, validated usage rows and the current explicit unique random pool logLabel; no native-main/fallback labels. Per account short/weekly/monthly, pair adjacent raw observations only with same source/reset boundary, increasing localtime and percentage delta>=1. Count only whole requests within (left,right], single-send reported nonestimated nonlocal attempts matching that label. Presence of an empty attempts array never falls back to request totals. Deduplicate requestId; conflicting duplicates yield insufficient evidence. No inferred absolute attempt start. Exclude boundary-spanning requests and unknown/multisend usage; no valid pair yields insufficient-evidence. + +Use reported totalTokens or input+output exactly once, not reasoning/cache detail additions. Median effective tokens per100percentage over defensible intervals, sampleCount explicit. Output confidence low and assumptions array: rounded/delayed quota, only retained valid proxy ledger rows, label continuity assumed inside the observation interval, external usage not observed. This is an observed effective estimate, never a provider token limit or proven lower bound. The private credential publication UUID must match before/after async ledger read; current explicit logLabel and uniqueness must still match config. Any mismatch yields insufficient-evidence, not mixed identity. No estimate is used for scheduling. + +Extend existing history GET result with capacity:{status:estimated|insufficient-evidence,estimates:[{window,estimatedTokens,sampleCount,confidence:low}],reason?,assumptions}. Cached history remains visible on ledger read failure. Use readUsageSnapshotForManagement; reject truncatedPrefixBytes/entriesTruncated/entriesDropped, missingrevision and >10000 retainedrows before estimator scan. This deliberately does not attest missing/rejected historical ledger lines; assumptions state that limitation. CLI history humanoutput renders estimates and sample counts/caveat; JSON carries fullobject. No new config, timer, persistence, GUI surface or inference call. + +Tests handcomputed10→20% plus1000reportedtokens→10000estimate; mixed sources/reset/refund/0delta/rounding/timestampintervals, duplicate request IDs, absent-vs-empty attempts, multisend, local/estimated/unattributed tokens, nonfiniteoutput, truncatedledger andidentitychangedawait. Sample storage/read provides current publication evidence; retrospective label continuity is explicitly low-confidence inference, not independently verified identity. This clarification replaces earlier mathematically unprovable lower-bound wording without reducing raw-data/identity fences. Local suites/build/typecheck/install NOTRUN. Independent source design/review plus final cumulative tip hostedCI required; hostFSMblockedB remains unchanged. + +Implementation refinements: reject absent physicalattempts, deduplicate ordinals, countonlyaccountfamily/sharedmodelscope and matchingwindowduration/primaryprovenance withresetnotelapsed. Preserve locallyAnswered duringexistingusagenormalization so capacitycanexcludeit. CaptureUUIDbeforehistoryread and recheckbefore/afterasyncledgerread; usefullboundedhistory forestimationindependentofdisplaylimit. Labels re-read fromcurrentruntimeconfig. diff --git a/devlog/_plan/260912_accounts/061_capacity_delivery.md b/devlog/_plan/260912_accounts/061_capacity_delivery.md new file mode 100644 index 0000000000..da434e3e12 --- /dev/null +++ b/devlog/_plan/260912_accounts/061_capacity_delivery.md @@ -0,0 +1,7 @@ +# Informational effective quota capacity + +This child of #4404 estimates observed reported tokens per100percentage from bounded raw observation intervals. It preserves private publication UUID checks and requires an explicit unique pool log label. The estimate is low-confidence with disclosed rounding, retained-valid-row, external-usage and label-continuity assumptions; it is not a provider limit or scheduling policy. + +Regression sources cover a hand-computed1000tokens/10points=10000, duplicates, single-send evidence, provenance/reset/interval/independent-model conditions, numeric overflow, bounded ledger rejection, populated API/CLI output and identity replacement during async usage read. Existing local-answer provenance now survives attempt normalization. No local suite/build/typecheck/install was run. Independent design source audit passed; implementation source review and final cumulative hostedCI remain pending. Actual hostgoal blocked/FSMB untouched; no persisted capacity PABCD cycle is claimed. + +Source review corrections: API accepts only explicit shared quota scope, excluding blank/undefined model identity through an actual populated API regression. CLI prints insufficient-evidence reasons through the closed reason parser, with estimated/insufficient human+JSON fixtures. A positive fraction that rounds to zero yields no estimate. Local suites remain NOTRUN. diff --git a/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md b/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md index 4f832d3dbb..bbed36cb4b 100644 --- a/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/ko/reference/cli/providers-accounts.md @@ -369,4 +369,6 @@ ocx models remove deepseek/deepseek-v4 --yes `ocx account history openai [--limit 1-200] [--json]`은 제공자에게 요청하지 않고 저장된 관측을 읽습니다. 관측 시각, WHAM·응답 헤더 출처, 한도 종류와 사용률을 구분해 표시합니다. 계정마다 최대 200개를 30일간 보관하며 전체 저장량에도 제한이 있습니다. -일반 토큰 갱신은 기록을 유지합니다. 재로그인·삭제·계정 교체는 이전 기록과 분리합니다. 네이티브 메인 계정과 로그인 저장 전 조회는 포함하지 않습니다. 기록이 없다는 것은 관측 부족이며 사용량 0을 뜻하지 않습니다. 이 명령은 토큰 용량을 추정하거나 쿼터를 소비하지 않습니다. +일반 토큰 갱신은 기록을 유지합니다. 재로그인·삭제·계정 교체는 이전 기록과 분리합니다. 네이티브 메인 계정과 로그인 저장 전 조회는 포함하지 않습니다. 기록이 없다는 것은 관측 부족이며 사용량 0을 뜻하지 않습니다. 이 명령은 쿼터를 소비하지 않습니다. 관측을 바탕으로 한 용량 추정에는 아래 한계가 적용됩니다. + +같은 초기화 구간의 관측과 계정별 사용 기록이 있으면 보고된 토큰 기준 용량 추정도 표시합니다. 표본 수와 낮은 신뢰도를 함께 표시하며, 쿼터 반올림·외부 사용량·로그 라벨 유지 여부 때문에 제공자의 실제 토큰 한도와 다를 수 있습니다. 기록이 없거나 잘렸으면 근거 부족으로 표시합니다. `--limit`은 표시할 기록 수만 제한하며 추정 입력은 전체 보관 범위입니다. diff --git a/docs-site/src/content/docs/reference/cli/providers-accounts.md b/docs-site/src/content/docs/reference/cli/providers-accounts.md index f39c1f3a57..30f289551d 100644 --- a/docs-site/src/content/docs/reference/cli/providers-accounts.md +++ b/docs-site/src/content/docs/reference/cli/providers-accounts.md @@ -586,4 +586,6 @@ all refuse the bad value rather than storing something the catalog writer would `ocx account history openai [--limit 1-200] [--json]` reads stored observations without contacting the provider. The output separates actual observation time, WHAM or response-header source, window family and usage percentage. At most 200 observations per account are retained for 30 days, with global storage bounds. -Ordinary token refresh preserves history. Reauthentication, removal or account replacement retires the old publication. Native main and probes performed before a login is published are not included. Missing history means insufficient observations, not zero usage. This command does not estimate token capacity or spend quota. +Ordinary token refresh preserves history. Reauthentication, removal or account replacement retires the old publication. Native main and probes performed before a login is published are not included. Missing history means insufficient observations, not zero usage. This command does not spend quota. Effective estimates, when supported by observations, carry the limitations below. + +The history output also includes effective reported-token estimates when same-window observations and attributable usage support them. Each estimate includes a sample count and low confidence. Quota rounding, external usage and assumed log-label continuity limit the inference; it is not your provider’s token allowance. Missing or truncated ledger evidence returns insufficient evidence. `--limit` controls displayed history, not the bounded estimate input. diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 7b60d1e5ea..82e06531e1 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -488,6 +488,7 @@ "codex-prompt-text-probe.test.ts": "codex-integration", "codex-quota-auto-refresh-main-admission.test.ts": "codex-integration", "codex-quota-auto-refresh.test.ts": "codex-integration", + "codex-quota-capacity.test.ts": "codex-integration", "codex-quota-history.test.ts": "codex-integration", "codex-quota-parser-parity.test.ts": "codex-integration", "codex-quota-prime.test.ts": "codex-integration", diff --git a/src/cli/account-history.ts b/src/cli/account-history.ts index c7662d3ad4..9534cf30fd 100644 --- a/src/cli/account-history.ts +++ b/src/cli/account-history.ts @@ -1,3 +1,4 @@ +import { parseCapacityReason } from "../codex/quota-capacity"; import { isValidCodexAccountId } from "../codex/account-id"; import { apiError, apiJson, proxyUnreachable, resolveBaseUrl, type AccountDeps } from "./account-api"; @@ -41,5 +42,19 @@ export async function cmdAccountHistory(args: string[], deps: AccountDeps): Prom console.log(`${historyDate(observation.observedAt)}\t${observation.source}\t${window.family}/${window.window}\t${window.usedPercent}%\t${historyDate(window.resetAtMs)}`); } } + const capacity = result.json.capacity; + if (capacity && typeof capacity === "object" && "status" in capacity && capacity.status === "estimated" + && "estimates" in capacity && Array.isArray(capacity.estimates)) { + console.log("Effective capacity estimate (low confidence; not a provider token limit):"); + for (const estimate of capacity.estimates) { + if (estimate && Number.isFinite(estimate.estimatedTokens) && Number.isSafeInteger(estimate.sampleCount)) { + console.log(`${estimate.window}\t~${estimate.estimatedTokens} reported tokens / 100%\t${estimate.sampleCount} samples`); + } + } + } + if (capacity && typeof capacity === "object" && "status" in capacity && capacity.status === "insufficient-evidence") { + const reason = "reason" in capacity ? parseCapacityReason(capacity.reason) : undefined; + console.log(`Effective capacity: insufficient evidence${reason ? ` (${reason})` : ""}.`); + } return 0; } diff --git a/src/codex/auth-api.ts b/src/codex/auth-api.ts index 4c112d03b0..c128e781d8 100644 --- a/src/codex/auth-api.ts +++ b/src/codex/auth-api.ts @@ -1,3 +1,7 @@ +import { CODEX_ACCOUNT_LOG_LABEL_RE } from "./account-label"; +import { poolQuotaHistoryIdentity } from "./account-store"; +import { estimateCodexQuotaCapacity, insufficientCodexCapacity, type CodexCapacityResult } from "./quota-capacity"; +import { readUsageSnapshotForManagement } from "../usage/log"; import { capturePoolQuotaWriter } from "./account-store"; import type { PoolQuotaWriter } from "./quota-types"; import { getAccountQuotaHistory, isValidWhamHistoryObservation } from "./quota"; @@ -44,6 +48,7 @@ import { } from "./account-priority"; import { claimDueCodexQuotaRecoveryProbes, + codexQuotaScopeForModel, claimManualResetCooldowns, settleManualResetCooldown, type ManualResetCooldownClaim, @@ -2543,8 +2548,35 @@ export async function handleCodexAuthAPI( || (rawLimit !== null && !/^(?:[1-9]|[1-9][0-9]|1[0-9]{2}|200)$/.test(rawLimit))) { return jsonResponse({ error: "A stored pool accountId and optional limit from 1 to 200 are required" }, 400); } + const runtimeConfig = getRuntimeConfig(config); + const account = configuredPoolAccount(runtimeConfig, accountId); + if (!account) return jsonResponse({ error: "Unknown pool account" }, 404); + const identity = poolQuotaHistoryIdentity(accountId); + const allHistory = getAccountQuotaHistory(accountId); + const limit = rawLimit === null ? 200 : Number(rawLimit); + const history = { ...allHistory, observations: allHistory.observations.slice(-limit), truncated: allHistory.observations.length > limit }; + const label = account.logLabel; + const labelStillUnique = () => { + const current = getRuntimeConfig(config); + return configuredPoolAccount(current, accountId)?.logLabel === label + && current.codexAccounts?.filter(row => codexAccountLogLabel(row) === label).length === 1; + }; + let capacity: CodexCapacityResult = insufficientCodexCapacity("identity_unavailable"); + if (identity && identity === poolQuotaHistoryIdentity(accountId) && label && CODEX_ACCOUNT_LOG_LABEL_RE.test(label) && labelStillUnique()) { + try { + const usage = await readUsageSnapshotForManagement(); + if (poolQuotaHistoryIdentity(accountId) !== identity || !labelStillUnique()) capacity = insufficientCodexCapacity("identity_changed"); + else if (!usage.revision) capacity = insufficientCodexCapacity("ledger_unavailable"); + else if (usage.truncatedPrefixBytes > 0 || usage.entriesTruncated || usage.entriesDropped > 0) capacity = insufficientCodexCapacity("ledger_truncated"); + else capacity = estimateCodexQuotaCapacity(allHistory.observations, usage.entries, label, + model => codexQuotaScopeForModel(model) === "shared"); + } catch { capacity = insufficientCodexCapacity("ledger_unavailable"); } + } if (!configuredPoolAccount(getRuntimeConfig(config), accountId)) return jsonResponse({ error: "Unknown pool account" }, 404); - return jsonResponse({ accountId, ...getAccountQuotaHistory(accountId, rawLimit === null ? 200 : Number(rawLimit)) }); + if (identity !== poolQuotaHistoryIdentity(accountId) || (identity && label && !labelStillUnique())) { + return jsonResponse({ accountId, ...getAccountQuotaHistory(accountId, limit), capacity: insufficientCodexCapacity("identity_changed") }); + } + return jsonResponse({ accountId, ...history, capacity }); } if (url.pathname === "/api/codex-auth/quota" && req.method === "GET") { diff --git a/src/codex/quota-capacity.ts b/src/codex/quota-capacity.ts new file mode 100644 index 0000000000..d22d53b511 --- /dev/null +++ b/src/codex/quota-capacity.ts @@ -0,0 +1,98 @@ +import type { QuotaHistorySample, QuotaHistoryWindow } from "./quota-history"; +import type { PersistedUsageAttempt, PersistedUsageEntry } from "../usage/log"; + +export const CAPACITY_ASSUMPTIONS = [ + "Quota percentages can be rounded or delayed.", + "Only retained valid proxy usage rows are observed; external usage is unknown.", + "Account log labels are assumed stable within each observation interval.", + "This low-confidence effective-token estimate is not a provider token limit or lower bound.", +] as const; +export const CAPACITY_REASONS = ["insufficient_intervals", "ledger_unavailable", "ledger_truncated", "identity_unavailable", "identity_changed", "ambiguous_usage"] as const; +export type CapacityReason = typeof CAPACITY_REASONS[number]; +export function parseCapacityReason(value: unknown): CapacityReason | undefined { + return CAPACITY_REASONS.find(reason => reason === value); +} +export interface CodexCapacityResult { + status: "estimated" | "insufficient-evidence"; + estimates: Array<{ window: QuotaHistoryWindow["window"]; estimatedTokens: number; sampleCount: number; confidence: "low" }>; + reason?: CapacityReason; + assumptions: readonly string[]; +} +export function insufficientCodexCapacity(reason: CapacityReason): CodexCapacityResult { + return { status: "insufficient-evidence", estimates: [], reason, assumptions: [...CAPACITY_ASSUMPTIONS] }; +} +const nonnegative = (value: unknown): value is number => typeof value === "number" && Number.isFinite(value) && value >= 0; + +function reportedTokens(attempt: PersistedUsageAttempt): number | undefined { + if (attempt.sendCount !== 1 || attempt.usageStatus !== "reported" || attempt.locallyAnswered === true + || !attempt.usage || attempt.usage.estimated === true + || !nonnegative(attempt.usage.inputTokens) || !nonnegative(attempt.usage.outputTokens)) return undefined; + const total = attempt.usage.totalTokens ?? attempt.usage.inputTokens + attempt.usage.outputTokens; + return nonnegative(total) ? total : undefined; +} + +/** Informational inference over raw same-window observations, never an account-selection input. */ +export function estimateCodexQuotaCapacity( + observations: ReadonlyArray>, + entries: readonly PersistedUsageEntry[], + label: string, + sharedQuotaModel: (model: string) => boolean, +): CodexCapacityResult { + if (entries.length > 10_000) return insufficientCodexCapacity("ledger_truncated"); + const requests = new Map(); + for (const entry of entries) { + const previous = requests.get(entry.requestId); + if (previous && JSON.stringify(previous) !== JSON.stringify(entry)) return insufficientCodexCapacity("ambiguous_usage"); + requests.set(entry.requestId, entry); + } + const sorted = [...observations].sort((a, b) => a.observedAt - b.observedAt); + const estimates: CodexCapacityResult["estimates"] = []; + for (const windowName of ["short", "weekly", "monthly"] as const) { + const points = sorted.flatMap(row => { + const window = row.windows.find(candidate => candidate.family === "account" && candidate.window === windowName); + return window ? [{ ...window, at: row.observedAt, source: row.source }] : []; + }); + const samples: number[] = []; + for (let index = 1; index < points.length; index++) { + const left = points[index - 1], right = points[index]; + const delta = right.usedPercent - left.usedPercent; + if (left.source !== right.source || !nonnegative(left.resetAtMs) || left.resetAtMs !== right.resetAtMs + || left.resetAtMs <= right.at || left.windowSeconds !== right.windowSeconds + || left.monthlyIsPrimaryWindow !== right.monthlyIsPrimaryWindow + || right.at <= left.at || delta < 1 || delta > 100 || !Number.isFinite(delta)) continue; + let tokens = 0; + let valid = true; + for (const entry of requests.values()) { + if (!nonnegative(entry.timestamp) || !nonnegative(entry.durationMs)) continue; + const end = entry.timestamp + entry.durationMs; + if (!Number.isFinite(end) || entry.timestamp <= left.at || end > right.at) continue; + // Untimed or absent physical-attempt evidence cannot be reconstructed from parent totals. + if (!entry.attempts?.length) continue; + const attempts = new Map(); + for (const attempt of entry.attempts) { + const prior = attempts.get(attempt.ordinal); + if (prior && JSON.stringify(prior) !== JSON.stringify(attempt)) { valid = false; break; } + attempts.set(attempt.ordinal, attempt); + } + if (!valid) break; + for (const attempt of attempts.values()) { + if (attempt.accountLogLabel !== label || attempt.adapter !== "openai-responses" || !sharedQuotaModel(attempt.model)) continue; + const reported = reportedTokens(attempt); + if (reported === undefined) continue; + tokens += reported; + } + } + const inferred = tokens * 100 / delta; + if (valid && tokens > 0 && Number.isFinite(inferred) && inferred > 0) samples.push(inferred); + } + if (samples.length) { + samples.sort((a, b) => a - b); + const middle = Math.floor(samples.length / 2); + const median = samples.length % 2 ? samples[middle] : samples[middle - 1] / 2 + samples[middle] / 2; + const estimatedTokens = Math.round(median); + if (estimatedTokens > 0) estimates.push({ window: windowName, estimatedTokens, sampleCount: samples.length, confidence: "low" }); + } + } + return estimates.length ? { status: "estimated", estimates, assumptions: [...CAPACITY_ASSUMPTIONS] } + : insufficientCodexCapacity("insufficient_intervals"); +} diff --git a/src/usage/log.ts b/src/usage/log.ts index 953195b7ca..2e81ebd681 100644 --- a/src/usage/log.ts +++ b/src/usage/log.ts @@ -457,6 +457,7 @@ function normalizeUsageAttempt(raw: unknown): PersistedUsageAttempt | null { durationMs: attempt.durationMs, // Absent by default; only the literal `true` marker survives the round trip. ...(attempt.streamAborted === true ? { streamAborted: true } : {}), + ...(attempt.locallyAnswered === true ? { locallyAnswered: true } : {}), ...(isNonNegativeFiniteNumber(attempt.firstOutputMs) ? { firstOutputMs: attempt.firstOutputMs } : {}), diff --git a/structure/catalog.md b/structure/catalog.md index 1055f6c0c4..a37d2c606e 100644 --- a/structure/catalog.md +++ b/structure/catalog.md @@ -303,3 +303,5 @@ Live sideband admission and its bounded upstream handshake follow the [runtime c Private pool credential metadata follows the [quota-history publication identity contract](providers/openai-tiers.md#quota-history-publication-identity); credential-only and account DTO projections omit it. Pool quota producers and account commands follow the [bounded raw-observation contract](providers/openai-tiers.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. + +The account history response can include a [low-confidence effective capacity estimate](providers/openai-tiers.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. diff --git a/structure/clients/claude-desktop.md b/structure/clients/claude-desktop.md index aaed5ce98a..5e2ec4983c 100644 --- a/structure/clients/claude-desktop.md +++ b/structure/clients/claude-desktop.md @@ -115,3 +115,5 @@ Live sideband admission and its bounded upstream handshake follow the [runtime c OpenCode is a separate launcher: its management catalog read retains local admin authority in the parent, while generated provider blocks reference only the child admission environment. It does not change Desktop configuration ownership. Pool quota producers and account commands follow the [bounded raw-observation contract](../providers/openai-tiers.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. + +The account history response can include a [low-confidence effective capacity estimate](../providers/openai-tiers.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. diff --git a/structure/codex-home.md b/structure/codex-home.md index d91210a1ed..529c5f655b 100644 --- a/structure/codex-home.md +++ b/structure/codex-home.md @@ -245,3 +245,5 @@ Incomplete compensation still raises the explicit partial-write error. Private pool credential metadata follows the [quota-history publication identity contract](providers/openai-tiers.md#quota-history-publication-identity); credential-only and account DTO projections omit it. Pool quota producers and account commands follow the [bounded raw-observation contract](providers/openai-tiers.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. + +The account history response can include a [low-confidence effective capacity estimate](providers/openai-tiers.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. diff --git a/structure/config.md b/structure/config.md index 27a9b7d2e2..06e3074f8f 100644 --- a/structure/config.md +++ b/structure/config.md @@ -229,3 +229,5 @@ The lightweight top-level CLI help counts Cline CLI among the fifteen registered The OpenCode launcher resolves the existing local management origin from the live bind and configured hub ingress. Its admin credential comes from the existing admin environment/file policy; an absent credential fails without substituting a data key. Pool quota producers and account commands follow the [bounded raw-observation contract](providers/openai-tiers.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. + +The account history response can include a [low-confidence effective capacity estimate](providers/openai-tiers.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. diff --git a/structure/gui-and-management-api.md b/structure/gui-and-management-api.md index c588fccf1d..856055b381 100644 --- a/structure/gui-and-management-api.md +++ b/structure/gui-and-management-api.md @@ -592,3 +592,5 @@ Live sideband admission and its bounded upstream handshake follow the [runtime c Dashboard overview polling observes authorization failures independently of stalled or rejected peer requests, cancels remaining child requests after a decisive result, and exposes resource-level deadline failures without rewriting them as authentication failures. Pool quota producers and account commands follow the [bounded raw-observation contract](providers/openai-tiers.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. + +The account history response can include a [low-confidence effective capacity estimate](providers/openai-tiers.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. diff --git a/structure/ops/docs-and-release.md b/structure/ops/docs-and-release.md index 0625489ac8..830637ec37 100644 --- a/structure/ops/docs-and-release.md +++ b/structure/ops/docs-and-release.md @@ -343,3 +343,5 @@ Translated Chat request construction uses the [inline-image budget](../transport OpenCode launcher verification distinguishes the local management catalog request from the inference child. Its transport regressions cover proxy environment, redirects, endpoint validation, credential precedence and child-env separation on hosted CI. Pool quota producers and account commands follow the [bounded raw-observation contract](../providers/openai-tiers.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. + +The account history response can include a [low-confidence effective capacity estimate](../providers/openai-tiers.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. diff --git a/structure/providers/openai-tiers.md b/structure/providers/openai-tiers.md index 6530ae35e7..a131b503bb 100644 --- a/structure/providers/openai-tiers.md +++ b/structure/providers/openai-tiers.md @@ -488,3 +488,9 @@ materialized headers pass the proxy-credential exclusion check before owner matc WHAM and response-header producers pass the exact captured pool writer, including refreshed replay and compact outcomes. Admission rechecks credential generation and publication UUID. Same-account refresh preserves prior history; replacement/removal invalidates it. Raw invalid percentages discard the entire trusted observation before display clamping; carried windows, reset credits alone, native main and staged-login probes never become durable pool history. `GET /api/codex-auth/quota/history` and `ocx account history openai ` read only cached, identity-checked observations. The optional limit is 1–200. Public results omit the internal publication UUID and credential generation. These observations are inputs for capacity estimation; percentages alone do not establish absolute token capacity. + +## Observed effective token capacity + +`src/codex/quota-capacity.ts` joins raw account-family observations with reported single-send usage attempts wholly contained within matching, unexpired reset intervals. Source, window duration and monthly-primary provenance must match; percentage delta must be at least one point. Duplicate request/attempt identities never multiply usage. Local, estimated, multi-send, independent-model and absent-attempt evidence does not supply a capacity sample. + +The history read API reports a median effective token estimate and interval sample count with low confidence and explicit coverage/rounding/label-continuity assumptions. It is not a provider token limit or mathematical lower bound and never affects account selection. Truncated, unavailable or excessive usage-ledger reads produce insufficient evidence while retaining history. Publication UUID and explicit unique account label are checked around the asynchronous read; identity changes discard the estimate and refresh the returned history. diff --git a/structure/runtime.md b/structure/runtime.md index 71e8f1a170..25ed8caefe 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -276,3 +276,5 @@ Translated Chat request construction uses the [inline-image budget](transports/s OpenCode catalog discovery in `src/cli/opencode.ts` uses the local admin credential and a validated numeric-loopback management origin. It dials through `src/server/direct-local-http.ts`, rejects redirects and preserves the request/body deadline. Hub ingress selection stays separate from exported inference settings. Pool quota producers and account commands follow the [bounded raw-observation contract](providers/openai-tiers.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. + +The account history response can include a [low-confidence effective capacity estimate](providers/openai-tiers.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. diff --git a/structure/subagents.md b/structure/subagents.md index 4f328b04bb..2e4c66cdd6 100644 --- a/structure/subagents.md +++ b/structure/subagents.md @@ -270,3 +270,5 @@ Live sideband admission and its bounded upstream handshake follow the [runtime c Private pool credential metadata follows the [quota-history publication identity contract](providers/openai-tiers.md#quota-history-publication-identity); credential-only and account DTO projections omit it. Pool quota producers and account commands follow the [bounded raw-observation contract](providers/openai-tiers.md#bounded-pool-quota-observations), separate from the latest display snapshot and capacity estimates. + +The account history response can include a [low-confidence effective capacity estimate](providers/openai-tiers.md#observed-effective-token-capacity); usage normalization retains local-answer provenance so local responses cannot supply samples. diff --git a/tests/cli/cli-account.test.ts b/tests/cli/cli-account.test.ts index a41187e27f..cb18aa5f7c 100644 --- a/tests/cli/cli-account.test.ts +++ b/tests/cli/cli-account.test.ts @@ -586,6 +586,19 @@ afterEach(() => { }); describe("ocx account CLI (issue #180 matrix)", () => { + test.each(["estimated", "insufficient-evidence"] as const)("human and JSON history preserve capacity status %s", async status => { + const capacity = status === "estimated" ? { status, estimates: [{ window: "weekly", estimatedTokens: 10000, sampleCount: 2, confidence: "low" }] } + : { status, reason: "ledger_truncated", estimates: [] }; + const deps: AccountDeps = { baseUrl: "http://127.0.0.1:10100", fetchImpl: (async () => Response.json({ observations: [{ + observedAt: 1_800_000_000_000, source: "wham", windows: [{ family: "account", window: "weekly", usedPercent: 20 }], + }], capacity })) as typeof fetch }; + const human = await run(["history", "openai", "pool-a"], deps); + expect(human.code).toBe(0); + expect(human.stdout).toContain(status === "estimated" ? "~10000 reported tokens / 100%\t2 samples" : "insufficient evidence (ledger_truncated)"); + const json = await run(["history", "openai", "pool-a", "--json"], deps); + expect(JSON.parse(json.stdout).capacity).toEqual(capacity); + }); + test("human quota history renders populated rows and safely handles oversized reset dates", async () => { const result = await run(["history", "openai", "pool-a"], { baseUrl: "http://127.0.0.1:10100", fetchImpl: (async () => Response.json({ observations: [{ observedAt: 1_800_000_000_000, source: "wham", windows: [ diff --git a/tests/codex-integration/codex-auth-api.test.ts b/tests/codex-integration/codex-auth-api.test.ts index 51a05d9170..fe6eca6ed2 100644 --- a/tests/codex-integration/codex-auth-api.test.ts +++ b/tests/codex-integration/codex-auth-api.test.ts @@ -1,3 +1,4 @@ +import * as usageHistoryModule from "../../src/usage/log"; import { getAccountQuotaHistory } from "../../src/codex/quota"; import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test"; import type { ServerWebSocket } from "bun"; @@ -1052,6 +1053,49 @@ describe("codex-auth API", () => { } }); + test("history capacity uses reported intervals and invalidates after identity changes during the ledger read", async () => { + const config = makeConfig(); + seedPoolAccount(config, { id: "capacity-a", email: "capacity@example.test", plan: "plus" }); + config.codexAccounts![0].logLabel = "pabcdef"; + const { capturePoolQuotaWriter } = await import("../../src/codex/account-store"); + const record = readCodexAccountRecord("capacity-a")!; + const writer = capturePoolQuotaWriter("capacity-a", { ...record.credential!, generation: record.generation })!; + const now = Date.now(); + for (const [observedAt, weeklyPercent] of [[now - 2000, 10], [now, 20]]) { + const raw = { weeklyPercent, weeklyResetAt: now + 100_000 }; + setAccountQuotaFromParsed("capacity-a", raw, undefined, undefined, raw, { writer, observedAt, source: "wham", raw }); + } + usageHistoryModule.appendUsageEntry({ requestId: "capacity-request", timestamp: now - 1000, durationMs: 100, provider: "openai", model: "gpt-5.5", status: 200, usageStatus: "reported", attempts: [{ + ordinal: 1, provider: "openai", model: "gpt-5.5", adapter: "openai-responses", status: 200, durationMs: 100, sendCount: 1, + recoveryKinds: [], usageStatus: "reported", accountLogLabel: "pabcdef", usage: { inputTokens: 800, outputTokens: 200, totalTokens: 1000 }, + }] }); + const request = () => new Request("http://localhost/api/codex-auth/quota/history?accountId=capacity-a&limit=1"); + const req = request(); + const result = await handleCodexAuthAPI(req, new URL(req.url), config); + const body = await result!.json() as { observations: unknown[]; capacity: { status: string; estimates: unknown[] } }; + expect(body.observations).toHaveLength(1); + expect(body.capacity.estimates).toEqual([{ window: "weekly", estimatedTokens: 10000, sampleCount: 1, confidence: "low" }]); + const stored = usageHistoryModule.readUsageEntries(); + expect(stored).toHaveLength(1); + stored[0].attempts![0].model = " "; + writeFileSync(usageHistoryModule.usageLogPath(), JSON.stringify(stored[0]) + "\n"); + const blankModel = request(); + const blankResult = await handleCodexAuthAPI(blankModel, new URL(blankModel.url), config); + expect((await blankResult!.json()).capacity).toMatchObject({ status: "insufficient-evidence", estimates: [] }); + const originalRead = usageHistoryModule.readUsageSnapshotForManagement; + const read = spyOn(usageHistoryModule, "readUsageSnapshotForManagement").mockImplementation(async () => { + const snapshot = await originalRead(); + saveCodexAccountCredential("capacity-a", record.credential!); + return snapshot; + }); + try { + const next = request(); + const response = await handleCodexAuthAPI(next, new URL(next.url), config); + expect(read).toHaveBeenCalledTimes(1); + expect(await response!.json()).toMatchObject({ observations: [], capacity: { status: "insufficient-evidence", reason: "identity_changed", estimates: [] } }); + } finally { read.mockRestore(); } + }); + test("GET /api/codex-auth/accounts returns array with main", async () => { const req = new Request("http://localhost/api/codex-auth/accounts", { method: "GET" }); const url = new URL(req.url); diff --git a/tests/codex-integration/codex-quota-capacity.test.ts b/tests/codex-integration/codex-quota-capacity.test.ts new file mode 100644 index 0000000000..3bca1e8391 --- /dev/null +++ b/tests/codex-integration/codex-quota-capacity.test.ts @@ -0,0 +1,66 @@ +import { describe, expect, test } from "bun:test"; +import { estimateCodexQuotaCapacity } from "../../src/codex/quota-capacity"; +import type { QuotaHistorySample } from "../../src/codex/quota-history"; +import type { PersistedUsageEntry, PersistedUsageAttempt } from "../../src/usage/log"; + +const label = "pabcdef"; +const shared = (model: string) => model !== "independent"; +const point = (at: number, percent: number): Omit => ({ + observedAt: at, source: "wham", windows: [{ family: "account", window: "weekly", usedPercent: percent, resetAtMs: 10_000 }], +}); +const attempt = (overrides: Partial = {}): PersistedUsageAttempt => ({ + ordinal: 1, provider: "openai", model: "gpt-test", adapter: "openai-responses", status: 200, durationMs: 10, + sendCount: 1, recoveryKinds: [], usageStatus: "reported", accountLogLabel: label, + usage: { inputTokens: 800, outputTokens: 200, totalTokens: 1000 }, ...overrides, +}); +const entry = (overrides: Partial = {}): PersistedUsageEntry => ({ + requestId: "r1", timestamp: 1100, durationMs: 100, provider: "openai", model: "gpt-test", status: 200, usageStatus: "reported", + attempts: [attempt()], ...overrides, +}); +const points = [point(1000, 10), point(2000, 20)]; + +describe("observed effective quota capacity", () => { + test("hand-calculated 1000 reported tokens over ten percentage points estimates 10000", () => { + const result = estimateCodexQuotaCapacity(points, [entry()], label, shared); + expect(result.status).toBe("estimated"); + expect(result.estimates).toEqual([{ window: "weekly", estimatedTokens: 10000, sampleCount: 1, confidence: "low" }]); + expect(result.assumptions.length).toBeGreaterThan(0); + }); + + test("duplicate rows and ordinals count once while conflicts refuse estimation", () => { + expect(estimateCodexQuotaCapacity(points, [entry(), entry()], label, shared).estimates[0].estimatedTokens).toBe(10000); + expect(estimateCodexQuotaCapacity(points, [entry({ attempts: [attempt(), attempt()] })], label, shared).estimates[0].estimatedTokens).toBe(10000); + expect(estimateCodexQuotaCapacity(points, [entry(), entry({ durationMs: 101 })], label, shared).status).toBe("insufficient-evidence"); + expect(estimateCodexQuotaCapacity(points, [entry({ attempts: [attempt(), attempt({ sendCount: 2 })] })], label, shared).status).toBe("insufficient-evidence"); + }); + + test.each([ + entry({ timestamp: 1000 }), entry({ timestamp: 1999, durationMs: 2 }), entry({ attempts: [] }), entry({ attempts: undefined }), + entry({ attempts: [attempt({ sendCount: 2 })] }), entry({ attempts: [attempt({ locallyAnswered: true })] }), + entry({ attempts: [attempt({ usage: { inputTokens: 1, outputTokens: 1, estimated: true } })] }), + entry({ attempts: [attempt({ usageStatus: "unreported" })] }), entry({ attempts: [attempt({ accountLogLabel: "p123456" })] }), + entry({ attempts: [attempt({ model: "independent" })] }), + ])("unknown or outside-interval usage supplies no sample", row => { + expect(estimateCodexQuotaCapacity(points, [row], label, shared).status).toBe("insufficient-evidence"); + }); + + test("window/provenance/reset changes, refunds and tiny deltas are not capacity intervals", () => { + for (const right of [point(2000, 9), point(2000, 10), point(2000, 10.1), { ...point(2000, 20), source: "response-header" as const }, + { ...point(2000, 20), windows: [{ ...point(2000, 20).windows[0], resetAtMs: undefined }] }, + { ...point(2000, 20), windows: [{ ...point(2000, 20).windows[0], resetAtMs: 20_000 }] }, + { ...point(2000, 20), windows: [{ ...point(2000, 20).windows[0], family: "spark" as const }] }, + ]) expect(estimateCodexQuotaCapacity([points[0], right], [entry()], label, shared).status).toBe("insufficient-evidence"); + }); + + test("overflow and bounded scan cannot produce a finite-looking false result", () => { + const oversized = entry({ attempts: [attempt({ usage: { inputTokens: Number.MAX_VALUE, outputTokens: Number.MAX_VALUE } })] }); + expect(estimateCodexQuotaCapacity(points, [oversized], label, shared).status).toBe("insufficient-evidence"); + expect(estimateCodexQuotaCapacity(points, Array.from({ length: 10001 }, () => entry()), label, shared).reason).toBe("ledger_truncated"); + }); +}); + + +test("a positive fractional inference never publishes zero capacity after rounding", () => { + const small = entry({ attempts: [attempt({ usage: { inputTokens: 0.1, outputTokens: 0 } })] }); + expect(estimateCodexQuotaCapacity([point(1000, 0), point(2000, 90)], [small], label, shared).status).toBe("insufficient-evidence"); +}); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index cfb2a0ac08..35413d8adc 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -321,6 +321,7 @@ "codex-prompt-text-probe.test.ts": "codex-integration", "codex-quota-auto-refresh-main-admission.test.ts": "codex-integration", "codex-quota-auto-refresh.test.ts": "codex-integration", + "codex-quota-capacity.test.ts": "codex-integration", "codex-quota-history.test.ts": "codex-integration", "codex-quota-parser-parity.test.ts": "codex-integration", "codex-quota-prime.test.ts": "codex-integration", diff --git a/tests/server/account-pool-management-api.test.ts b/tests/server/account-pool-management-api.test.ts index 63a1feefc3..90805e538b 100644 --- a/tests/server/account-pool-management-api.test.ts +++ b/tests/server/account-pool-management-api.test.ts @@ -687,7 +687,7 @@ describe("unified pool-settings contract (#695 wp5c)", () => { await unknown.text(); const response = await fetch(new URL(`${endpoint}?accountId=history-row&limit=1`, server.url)); expect(response.status).toBe(200); - expect(await response.json()).toEqual({ accountId: "history-row", observations: [], retention: { maxObservations: 200, maxAgeDays: 30 }, truncated: false }); + expect(await response.json()).toEqual({ accountId: "history-row", observations: [], retention: { maxObservations: 200, maxAgeDays: 30 }, truncated: false, capacity: { status: "insufficient-evidence", reason: "identity_unavailable", estimates: [], assumptions: expect.any(Array) } }); const { saveCodexAccountCredential, capturePoolQuotaWriter } = await import("../../src/codex/account-store"); const { setAccountQuotaFromParsed } = await import("../../src/codex/quota"); const credential = { accessToken: "history-secret-access", refreshToken: "history-secret-refresh", expiresAt: Date.now() + 3600_000, chatgptAccountId: "private-history-account" }; diff --git a/tests/usage/usage-log.test.ts b/tests/usage/usage-log.test.ts index f39c778d2b..9cd97855a6 100644 --- a/tests/usage/usage-log.test.ts +++ b/tests/usage/usage-log.test.ts @@ -579,6 +579,14 @@ describe("usage log", () => { expect(valid.attempts?.[0]?.reasoningWireValue).toBe(false); }); + test("local-answer provenance survives attempt normalization for capacity exclusion", () => { + const value = normalizeUsageEntryForTest({ requestId: "local-capacity", timestamp: Date.now(), provider: "openai", model: "m", status: 200, durationMs: 1, usageStatus: "reported", attempts: [{ + ordinal: 1, provider: "openai", model: "m", adapter: "openai-responses", status: 200, durationMs: 1, sendCount: 1, + recoveryKinds: [], usageStatus: "reported", locallyAnswered: true, accountLogLabel: "pabcdef", usage: { inputTokens: 1, outputTokens: 1 }, + }] }); + expect(value.attempts?.[0]?.locallyAnswered).toBe(true); + }); + test("drops only malformed persisted attempts while preserving valid siblings", () => { const valid = (ordinal: number) => ({ ordinal,