Cost estimates ignore published long-context pricing tiers (OpenAI >272k, xAI >=200k)

[brunoflma]

Area

Dashboard

What are you trying to accomplish?

I want the ~$ column and /api/usage to be right for the requests that matter most — the big ones.

Several vendors reprice the entire request once the prompt crosses a token threshold. The two that affect me:

Vendor	Threshold	Effect on the whole request	Source
OpenAI GPT-5.6 Sol / Terra / Luna	> 272,000 input tokens	2× input, 2× cached input, 1.5× output	https://developers.openai.com/api/docs/pricing
xAI Grok 4.5	≥ 200,000 input tokens	2× on every rate	https://docs.x.ai/developers/pricing
MiniMax M3	> 512,000 input tokens	2× on every rate	https://platform.minimax.io/docs/guides/pricing-paygo

OpenAI publishes these as a separate "long context" row per model, not as a footnote — Sol is 5 / 0.50 / 30 short and 10 / 1.00 / 45 long.

Long requests are exactly the expensive ones, so an estimator that ignores the tier is wrong where being right is worth the most.

What prevents this today?

Cost4 is a flat four-tuple, and resolveMatchedPrice(provider, modelId) never sees a token count — so there is nowhere to express "this rate depends on how big the prompt is". Every request bills at the short rate regardless of size.

applyPriorityMultiplier() already establishes the shape for a conditional repricing step, but it keys off serviceTier, not off prompt size, so it cannot be reused as-is.

There is also a subtlety that makes this easy to get wrong: the threshold is measured on the raw usage.inputTokens the vendor counted, whereas normalizeCostTokens() subtracts cache read and write to produce billable input. A 280k-token prompt with a 200k cache read has 80k billable input but still crosses OpenAI's 272k threshold. Deciding the tier after normalization would silently under-bill exactly the cache-heavy long requests.

What should OpenCodex do?

When a request's raw input token count crosses a model's published threshold, the whole request should be costed at that model's long rate, and the estimate should say so — a flag on CostEstimate so the dashboard can distinguish "long" from "standard" rather than just showing a larger number.

Models with no published tier must be completely unaffected, and the tier must compose with the existing Fast-mode multiplier instead of replacing it.

Example usage or interface

Same request, openai/gpt-5.6-sol, 300,000 input tokens and 20,000 output tokens:

today:     5 * 0.300 + 30 * 0.020 = $2.10
should be: 10 * 0.300 + 45 * 0.020 = $3.90

What I run locally: tiers stored as multipliers rather than a second rate table, so a tier composes with whatever base rate resolved (jawcode bundle or overlay) instead of duplicating every price and drifting from it.

export interface ContextTier {
  thresholdInputTokens: number;
  /** true: `>= threshold` (xAI). false: `> threshold` (OpenAI). */
  inclusive: boolean;
  input: number; output: number; cacheRead: number; cacheWrite: number;
  source: string;
}

/** Exact (provider, modelId) key. No fuzzy matching. */
export const CONTEXT_TIERS: Readonly<Record<string, ContextTier>>;
export function findContextTier(provider: string, modelId: string): ContextTier | undefined;
export function isLongContext(tier: ContextTier, rawInputTokens: number): boolean;

Applied in estimateRequestCost() / estimateAttemptCost() before the priority multiplier, so a long Fast request bills long-rate × Fast-multiplier:

const [tieredCost4, contextTier] =
  applyContextTier(price.cost4, input.provider, input.model, input.usage.inputTokens);
const [effectiveCost4, multiplier] =
  applyPriorityMultiplier(tieredCost4, input.provider, input.model, input.serviceTier);
// ...
...(contextTier ? { contextTier } : {}),   // CostEstimate.contextTier?: "long"

estimateComboCost() propagates contextTier: "long" when any attempt crossed.

The inclusive flag is not over-engineering — xAI documents "once a prompt reaches 200k" and OpenAI documents "prompts above 272K", and a model sitting exactly on 272,000 bills differently under the two readings.

Measured on 30 days of my own usage.jsonl (same requests, same normalized tokens, only the rate table differs):

openai/gpt-5.6-sol    390/6283 requests over 272k   max prompt 337,247
                      short $76.47  ->  long $150.38
openai/gpt-5.6-luna     1/424  requests over 272k
                      short  $0.31  ->  long   $0.62
TOTAL  391 requests under-estimated by $74.22

Those 390 Sol requests were billed at roughly half what OpenAI charges. It is ~4% of my 30-day total, concentrated in <2% of requests.

Alternatives or workarounds

• A second overlay row per long variant (e.g. gpt-5.6-sol-long) — rejected: nothing in the log distinguishes them, so the resolver still could not pick between the two without a token count.
• Absolute long rates instead of multipliers — works, but doubles the table and lets the long rate drift from the short one when a vendor reprices. Multipliers stay correct through a price change; only the threshold is model-specific data.
• Correcting it after the fact in the dashboard — the tier has to be applied per request before summing, so it cannot be a display-layer adjustment.
• Leaving it — defensible while ~$ is explicitly a list-price estimate (Clarify that usage/logs \$ estimates are API list-price math, not actual charges (esp. subscription OAuth) #198), but the error is systematically one-directional and lands on the priciest requests.

Additional context

Related: I have just filed the stale gpt-5.6-terra / gpt-5.6-luna bundle rates separately. These interact — a gpt-5.6-luna prompt over 272k is currently costed with a 5×-too-high base rate and a 2×-too-low tier — but they are independent fixes and I did not want to bundle them.

Vendor sources: OpenAI, xAI, MiniMax.

I have been running the above against 2.10.0 with a test suite covering both threshold edges (199,999 vs 200,000 for xAI; 272,000 vs 272,001 for OpenAI), the raw-vs-normalized input case, models with no tier, and the interaction with the Fast multiplier. The existing tests/usage-cost.test.ts priority block needs its expectations updated, because its shared 1M-token fixture also crosses 272k — that is the change surfacing, not breaking, but it is worth knowing before review.

Happy to send it as a PR if the shape looks right to you. If you would rather keep Cost4 flat and put the threshold somewhere else entirely, say where and I will follow that instead.

Checks

• I searched existing issues and documentation.
• This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
• I removed secrets and personal data.

[github-actions]

_{Automated translation bookkeeping — detected language: unknown.}

[Ingwannu]

Confirmed on current dev (3cd75ab0): this is a real cost-estimation bug, not a GUI-only enhancement.

usage.inputTokens already preserves the raw prompt size including cached input, but the estimator resolves one flat Cost4, normalizes cache tokens, and immediately calculates the request. There is currently no step that can select the published long-context rate, so both /api/logs and /api/usage under-estimate qualifying requests. Your 300k Sol example reproduces exactly: the current estimate is $2.10 and the published long-context estimate is $3.90.

Your proposed direction is sound with these boundaries:

• keep resolveMatchedPrice() token-independent because it is memoized by provider/model;
• select the context tier from raw usage.inputTokens after base-price resolution and before calculateCost();
• keep the tier registry exact and provider-scoped so Cursor/OpenRouter routes are not charged using first-party API rules;
• record both source and verifiedAt;
• expose the optional tier on each attempt and propagate it to a combo result;
• compose it with the existing Fast multiplier;
• take care with exact MiniMax casing because the bundle currently contains both MiniMax-M3 and a lowercase minimax-m3 row with different prices.

Please send this as a core-only PR targeting dev, limited to src/usage/expected-prices.ts, src/usage/cost.ts, and focused tests. Do not include GUI, i18n, styling, or visible-copy changes.

Required coverage: OpenAI 272000/272001, xAI 199999/200000, MiniMax 512000/512001, raw-vs-normalized cached input, routed-provider isolation, an unaffected model, long-context + Fast composition, and per-attempt combo propagation. Please also keep the existing shared priority fixture below the long-context threshold and add a separate explicit composition case.

Thanks for the unusually complete report and measured reproduction. I changed the type label from enhancement to bug; the gui area label stays because that is one visible surface, but the implementation belongs in the usage core.