-
Notifications
You must be signed in to change notification settings - Fork 1.1k
fix(devin): report the context windows Cognition actually serves #4323
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1722,11 +1722,23 @@ async function fetchProviderModelsWithAuth( | |
| if (liveResult.ok) { | ||
| // Live catalog is the source of truth — use the discovered base models | ||
| // directly, not a filtered subset of the static seed. | ||
| const result = liveResult.models.map((id) => ({ | ||
| id, | ||
| provider: name, | ||
| ...catalogHintsFromProviderConfig(name, prov, id, contextCap, metadataModelIdCaseFold, captured.effectiveAlias), | ||
| }) as CatalogModel); | ||
| // | ||
| // That extends to the context window. Cognition publishes no window | ||
| // anywhere, so the per-account catalog is the only first-party number, | ||
| // and the shipped static table is a degraded-mode guess that was wrong | ||
| // for nine of its eleven rows. The live value is applied first and the | ||
| // config hints run after it, so an explicit per-model override and an | ||
| // enabled Context cap still win — this only replaces the number nobody | ||
|
Comment on lines
+1728
to
+1731
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This changes the provider catalog source-of-truth, context-window precedence, and persisted-config startup repair across AGENTS.md reference: src/AGENTS.md:L11-L11 Useful? React with 👍 / 👎. |
||
| // chose. | ||
| const result = liveResult.models.map((id) => { | ||
| const liveWindow = liveResult.contextWindows[id]; | ||
| return { | ||
| id, | ||
| provider: name, | ||
| ...(liveWindow ? { contextWindow: liveWindow } : {}), | ||
| ...catalogHintsFromProviderConfig(name, prov, id, contextCap, metadataModelIdCaseFold, captured.effectiveAlias), | ||
|
Comment on lines
+1738
to
+1739
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
For a normal registry-enriched Devin provider, AGENTS.md reference: src/AGENTS.md:L18-L18 Useful? React with 👍 / 👎. |
||
| } as CatalogModel; | ||
| }); | ||
| const forCache = withConfiguredRetention(result, { retainComboTargets: false }); | ||
| if (!setCached(name, forCache, Date.now(), cacheGeneration)) { | ||
| return observed(withConfiguredRetention(configured), "degraded"); | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| /** | ||
| * Repair context windows that a saved config inherited from a wrong registry seed. | ||
| * | ||
| * `enrichProviderFromRegistry` is fill-only: it seeds `modelContextWindows` when | ||
| * the row has none and never rewrites it afterwards. That posture is right — a | ||
| * hand-tuned window must survive an upgrade — but it means a registry table that | ||
| * shipped WRONG numbers is frozen into every config that was saved while those | ||
| * numbers were current. Correcting the registry alone fixes new installs and | ||
| * leaves existing ones reporting the old figure forever. | ||
| * | ||
| * This rewrites one thing: a window whose saved value is still byte-for-byte the | ||
| * wrong number this file names, on a provider that still carries the registry's | ||
| * adapter. A value the user changed does not match `from` and is left alone, and | ||
| * nothing else in the row is touched. Same shape and the same restraint as | ||
| * `model-rename-migration`, for the case where the id was right and the number | ||
| * was not. | ||
| */ | ||
| import { PROVIDER_REGISTRY } from "./registry"; | ||
| import type { OcxConfig } from "../types"; | ||
|
|
||
| export interface StaleContextWindow { | ||
| /** Registry provider id whose saved rows may carry the wrong window. */ | ||
| provider: string; | ||
| model: string; | ||
| /** The wrong value this migration is allowed to replace, and nothing else. */ | ||
| from: number; | ||
| to: number; | ||
| } | ||
|
|
||
| export interface StaleContextWindowProjection { | ||
| config: OcxConfig; | ||
| changed: boolean; | ||
| warnings: string[]; | ||
| } | ||
|
|
||
| /** | ||
| * Cognition windows corrected against a live `GetCascadeModelConfigs` response. | ||
| * | ||
| * The shipped table had been assembled from each model's ORIGINAL vendor window | ||
| * rather than from what Cognition serves, so the Claude rows claimed 200k against | ||
| * an actual 1M and Grok claimed 256k against 500k. Cognition documents no window | ||
| * anywhere, so the per-account catalog is the only first-party source; these are | ||
| * the degraded-mode figures, and live discovery supersedes them when it runs. | ||
| */ | ||
| export const STALE_CONTEXT_WINDOWS: readonly StaleContextWindow[] = [ | ||
| { provider: "devin", model: "swe-1-7", from: 256_000, to: 262_000 }, | ||
| { provider: "devin", model: "swe-1-7-lightning", from: 256_000, to: 202_752 }, | ||
| { provider: "devin", model: "gpt-5-6-sol", from: 1_050_000, to: 1_000_000 }, | ||
| { provider: "devin", model: "gpt-5-6-luna", from: 1_050_000, to: 1_000_000 }, | ||
| { provider: "devin", model: "gpt-5-6-terra", from: 1_050_000, to: 1_000_000 }, | ||
| { provider: "devin", model: "claude-opus-4-8", from: 200_000, to: 1_000_000 }, | ||
| { provider: "devin", model: "claude-fable-5-1", from: 200_000, to: 1_000_000 }, | ||
| { provider: "devin", model: "claude-sonnet-5", from: 200_000, to: 1_000_000 }, | ||
| { provider: "devin", model: "kimi-k2-7", from: 256_000, to: 262_144 }, | ||
| { provider: "devin", model: "grok-4-5", from: 256_000, to: 500_000 }, | ||
| ]; | ||
|
|
||
| function providerStillMatchesRegistry(id: string, adapter: unknown): boolean { | ||
| const entry = PROVIDER_REGISTRY.find(row => row.id === id); | ||
| return entry !== undefined && entry.adapter === adapter; | ||
| } | ||
|
|
||
| /** Pure projection. The caller decides whether to persist. */ | ||
| export function projectStaleContextWindows( | ||
| config: OcxConfig, | ||
| entries: readonly StaleContextWindow[] = STALE_CONTEXT_WINDOWS, | ||
| ): StaleContextWindowProjection { | ||
| const warnings: string[] = []; | ||
| const repaired = new Map<string, string[]>(); | ||
|
|
||
| for (const entry of entries) { | ||
| const prov = config.providers?.[entry.provider]; | ||
| if (!prov) continue; | ||
| if (!providerStillMatchesRegistry(entry.provider, prov.adapter)) continue; | ||
| const windows = prov.modelContextWindows; | ||
| if (!windows || windows[entry.model] !== entry.from) continue; | ||
| windows[entry.model] = entry.to; | ||
| const list = repaired.get(entry.provider) ?? []; | ||
| list.push(`${entry.model} ${entry.from} -> ${entry.to}`); | ||
| repaired.set(entry.provider, list); | ||
| } | ||
|
|
||
| for (const [provider, list] of repaired) { | ||
| warnings.push( | ||
| `corrected ${list.length} context window(s) on "${provider}" that the saved config ` | ||
| + `inherited from a wrong registry seed: ${list.join(", ")}.`, | ||
| ); | ||
| } | ||
|
|
||
| return { config, changed: repaired.size > 0, warnings }; | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Use a null-prototype record for catalog model IDs.
If the upstream catalog returns
modelUid"__proto__",collapseDevinModelUidpreserves it. The normal object lookup atsrc/adapters/devin/live-models.ts:122returnsObject.prototype, and the assignment at line 123 cannot create a numeric own property.src/codex/catalog/provider-fetch.ts:1734then reads that inherited object and can emit it asCatalogModel.contextWindow.📝 Committable suggestion
🤖 Prompt for AI Agents