Skip to content
17 changes: 17 additions & 0 deletions devlog/_plan/260906_grok_catalog_and_patch/000_research.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Grok catalog selection and Codex patch parity

Class C3, spec-satisfaction repair. The user clarified that filtering means enabled model visibility in Pi/Aside, not assistant output filtering. No output-filter changes are authorized by this unit. Prior Chat patch fixes must remain effective at every Codex Responses tool completion boundary.

Scope: export catalog selection, refresh of already-owned Pi/Aside integrations, native Responses custom-tool repair, focused regression tests, matching docs. No live user config changes, deployment, release, unrelated cleanup, local test suites or local typecheck. Push --no-verify and admin merge are explicitly authorized. Probes and hosted CI are authorized.

Evidence: src/clients/config-export/constants.ts selects openai-completions for Pi; Aside uses the same builder. src/server/management/model-rows.ts:218 filters disabled only. src/cli/opencode.ts:373 likewise ignores selectedModels. src/codex/catalog/provider-fetch.ts:1992 is canonical for allowlist plus disabled and pending selection. src/server/management/model-routes.ts visibility writes converge Codex only. Both explicit sync paths refresh only MCode among owned file integrations. Live read-only snapshot: /v1/models and existing Aside managed file currently contain xai/grok-4.6 only; do not claim that the live snapshot reproduced a full catalog leak. Synthetic allowlist and stale-owned-file scenarios will establish the gaps.

Independent patch analysis: native custom exec is skipped by repairable||aliased in responses-custom-tool-repair.ts:206. Its item.done/input.done preserve raw patch while response.completed repairs it. Function apply_patch helper aliases can stream raw patch before compiling final JS. Existing arbitrary JS and foreign namespace boundaries stay byte-exact.

Dependencies: 010 export selection -> 020 owned file convergence -> 030 Codex patch completion parity. Each has its own PABCD and reviewable PR. The patch layer is a separate user-requested stabilization concern published after catalog layers in the requested stack.

Resource scope: existing GitHub repo credentials, read-only local configuration with no secret output; at most 24 synthetic live-provider calls, each <=120 seconds; six-hour execution window. No explicit token budget. Probe scripts stay ignored .tmp; public notes contain no credentials or private requests. DONE = regression probes, actual hosted exact-head test/typecheck CI, independent review, registered stack merged and fetched-dev ancestry. BLOCKED only for a persistent external dependency; no stopping on CI queueing. Each later P rechecks current source and carries earlier evidence. Escalation: reclaim failed delegated scope; new write delegation requires P amendment.

## Baseline probes

`bun .tmp/grok-stabilization/catalog-probe.ts` exit 0: management/CLI × pi/aside each emitted grok-4.3, grok-4.5, grok-4.6 despite selectedModels=[grok-4.6]; full management roster was three. `bun .tmp/grok-stabilization/patch-probe.ts` exit 0: native custom exec emitted one raw delta, input.done uncompiled and item.done uncompiled; function apply_patch alias emitted one raw preview despite compiled final. Both are observation probes before repair, not passing acceptance assertions. `python3 .tmp/grok-stabilization/verify-roadmap.py` exit 0 checks numbered roadmap artifacts and actual existing source target paths. No local suite or typecheck was run.
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# 010 Export catalog visibility

Loop: spec-satisfaction repair; trigger: selectedModels ignored by export projection. Goal: Pi/Aside export obeys the same provider allowlist, blocklist, pending selection as routed catalog. No change to full management catalog or routing authorization.

MODIFY src/server/management/model-rows.ts: import filterCatalogVisibleModels. In loadExportModels compute visible routed row identities from filterCatalogVisibleModels(rows.filter(row => !row.native), config); return only !row.disabled and (row.native || visible set contains row), then map toExportModel. Preserve native visibility semantics.
MODIFY src/cli/opencode.ts: use same canonical filter once for rows with non-native provider/id identity, then exclude those not retained before seen.add. Do not infer provider identities for legacy rows missing them; keep existing disabled and Direct-native checks. Preserve order, custom/combo aliases and per-row metadata; do not duplicate allowlist matching.
MODIFY tests/server/management-client-config-route.test.ts and tests/cli/cli-export-command.test.ts: fixtures with xai selectedModels=[grok-4.6], full three-model roster, blocklist override, empty allowlist, slash-bearing ids, disabled duplicate. Render both pi and aside through production loader and CLI projection. A nonempty allowlist retains only selected IDs; a ready provider with an empty allowlist retains its full otherwise-visible roster; pending initial selection keeps routed rows hidden. Management still offers all IDs. CLI consumers must reload their configured state after discovery because the request can persist the initial selection.
MODIFY docs-site/src/content/docs/guides/integrations.md: explain selected list applies to generated catalogs.

Verifier: standalone synthetic imports of loadExportModels/exportModelsFromProxyRows plus Pi/Aside serializers; no bun:test. CI runs existing focused regressions, typecheck and full platform suite. Before C record exact source SHA and probe output. Stop after export boundaries agree; next cycle refreshes old owned files.
17 changes: 17 additions & 0 deletions devlog/_plan/260906_grok_catalog_and_patch/020_owned_refresh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# 020 Converge already-owned Pi/Aside catalogs

Depends on 010 filtered loader. Loop spec-satisfaction repair. Goal: a model visibility/selection change and explicit sync refresh existing connected Pi/Aside files. No adoption of unowned/manual files, no recreation of removed blocks, no override of drift.

NEW src/integrations/catalog-refresh.ts: bounded helper refreshOwnedCatalogIntegrations(input) iterates explicit client list [pi, aside] (sync may include mcode), passes lazy cached models loader to refreshOwnedIntegration, catches per-client errors and returns existing outcome shape. Use existing ownership store, mutation flight and coordinated writer; never bypass fingerprints.
MODIFY src/server/management/model-routes.ts: local async convergence helper calls existing convergeCodexCatalog then new owned refresh for pi/aside with port from URL/config and lazy loadExportModels(config); attach clientIntegrations outcome to disabled-models, model-visibility, selected-models and model-preset writes. Keep successful config persistence even when one file refuses refresh; return warning outcome.
MODIFY src/server/management/config-routes.ts and src/cli/dispatch.ts: expand current MCode-only owned refresh to mcode/pi/aside via helper; preserve native Grok/Desktop gates and refused-sync behavior.
Comment thread
lidge-jun marked this conversation as resolved.
MODIFY existing tests/clients/sync-client-integrations.test.ts and tests/server/management-integration-routes.test.ts: fake IO/store or isolated home seeds owned pi/aside with two models, refresh with selected one, assert hidden row removed and other provider fields preserved. Prove unowned, removed and drifted configs untouched; one failure does not block other client. Add route-driven visibility refresh coverage using injected convergence.
UPDATE structure/09_client-integrations.md and owning docs page with ownership/refusal semantics.

Verification: standalone isolated writer probe using synthetic models and temp homes, then exact-head hosted CI. C4 care for automatic owned-file writes: independent review must confirm ownership/no-clobber and per-client failure boundaries. Final enforcement is existing coordinated writer; refresh helper is an early caller, not a permission boundary. Known bypass: manually calling writer with explicit adoption; no such call in this unit. Stop when file projection converges or produces truthful refusal.

## Audit amendment: overlapping refreshes

The existing constant refresh mutation-flight key incorrectly joins different model selections. MODIFY src/integrations/owned-refresh.ts to use a unique per-refresh operation key (crypto.randomUUID), making overlapping refreshes explicitly busy rather than reporting another desired catalog as success. Implicit refresh never joins an explicit HTTP mutation. Add controlled overlap with distinct old/new rosters: second call reports integration_mutation_busy; first result describes only its own write. Subsequent retry applies the new roster. Return per-client failures; never retry stale snapshots automatically.

Add a ManagementApiDeps refreshOwnedCatalogIntegrations seam for route verification, defaulting to the real helper. Creation: exported helper/deps type; consumption: model routes and explicit sync. No serialization/deserialization: runtime-only dependency injection. Tests use fake IO/store or temporary home, never actual user-owned files.
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# 030 Codex native Responses patch completion parity

Depends on recorded export layers for stack delivery; runtime independent. Class C3, spec-satisfaction repair. Goal: same repaired executable input at deltas/input.done/item.done/response.completed for complete patches misrouted as exec. No arbitrary JavaScript rewriting.

MODIFY src/server/responses-custom-tool-repair.ts: register same-name routed custom calls in addition to aliases. Track original wire name and target; hold custom input deltas when code-mode exec may be a patch envelope or when helper alias requires compilation. Accumulate under TranslatorBudget, release on done/dispose. Run restoreRoutedCustomCalls for same-name custom items, and use existing resolveCodeModeHelperName/compileCodeModeHelperInput at input.done. Do not place exec in repairNames. Preserve ordinary JavaScript streaming where monotonic; once raw prefix would diverge, withhold to authoritative completion. Suppress function helper-alias progressive previews rather than emitting raw patch before compiled JS.
MODIFY tests/responses/responses-custom-tool-repair.test.ts: native custom exec raw/wrapped complete patch, fragmented marker, input.done and output_item.done plus terminal snapshots; function apply_patch wrapper alias; invalid/incomplete envelopes and valid JS remain exact; flat catalogs and foreign namespaces do not retarget; cancellation frees retained buffers.
UPDATE existing patch compatibility docs and structure/11_compatibility-contracts.md to describe completion-boundary parity.

Verifier: pure standalone synthetic SSE-block imports, compare outputs at each lifecycle edge and execute generated JS against a recording tools.apply_patch stub (no filesystem writes). Probe must assert monotonic preview or held preview, one call, exact canonical patch data. CI runs added regressions and existing bridge/native compatibility tests plus full suite/typecheck. Complete only after independent review and exact-head CI; register requested stack and admin merge after verified heads. Fetch dev and prove every merge SHA ancestor. D records parity inventory and public PR links.

## Audit amendment

Executable repair is limited to authorized code-mode exec and recognized helper aliases; unrelated same-name native custom tools keep raw input byte-for-byte. Explicit negative: render_diagram input JSON string {"input":"literal"} is not unwrapped. Separate scenarios cover missing input.done, terminal-only completion, failed/incomplete after held deltas, and disposal. Authoritative completion wins over previews. Failure never synthesizes successful completion. All retained buffers release. One simulated execution means choose the client-consumed completed item once, not execute every redundant lifecycle representation.
4 changes: 4 additions & 0 deletions docs-site/src/content/docs/guides/integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ file, and removes it again. Twelve clients work this way, each with a switch:
| ZCode | `~/.zcode/v2/config.json` | JSON | on restart | loopback placeholder |
| Aside | `~/.aside/u/<account>/models.json` | JSON | after fully quitting and reopening Aside | loopback placeholder |

Generated catalogs include only enabled models from each provider selection. This applies to both
downloads and managed integrations, including Pi and Aside. The management model list still shows
the full roster so you can enable additional models.

The managed OpenCode integration owns two fragments: `provider.opencodex` (opencode V1) and
`providers.opencodex` (opencode V2). Only the V2 block carries the per-model reasoning-effort
variants, so both are written and kept in sync; they name the same provider and model ids, and
Expand Down
4 changes: 3 additions & 1 deletion src/cli/export-command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -171,12 +171,14 @@ export async function handleExportCommand(argv: string[], deps: ExportCommandDep
rejectArgs(args, USAGE);

const spec = EXPORT_CLIENTS[client];
const config = (deps.configImpl ?? loadConfig)();
const root = await runtimeBaseUrl(deps);
const rows = await runtimeRequest<ExportProxyModelRow[]>("/api/models", {}, { ...deps, baseUrl: root });
if (!Array.isArray(rows)) {
throw new RuntimeApiError("Management API returned an unexpected /api/models payload.", 502, rows);
}
// Discovery can persist pending -> ready selection. Read from the caller's
// config source after the response, rather than filtering with a stale snapshot.
const config = (deps.configImpl ?? loadConfig)();
const models = exportModelsFromProxyRows(rows, config);
// The text is the client's OWN format — YAML, TOML and JSON5 clients would
// otherwise receive a JSON rendering their parser reads differently.
Expand Down
15 changes: 11 additions & 4 deletions src/cli/opencode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ import type {
OpencodeProviderBlocks,
OpencodeV2ProviderBlock,
} from "../clients/config-export";
import { visibleNativeSlugs } from "../codex/catalog";
import { filterCatalogVisibleModels, visibleNativeSlugs } from "../codex/catalog";
import { commandInvocation } from "../lib/win-exec";
import { loadServiceTokenFromFile, serviceApiTokenFilePath } from "../lib/service-secrets";
import { providerCodexAccountMode } from "../providers/registry";
Expand Down Expand Up @@ -376,12 +376,17 @@ export function opencodeCatalogFromProxyRows(
config: OcxConfig,
): OpencodeCatalogModel[] {
const omitNative = providerCodexAccountMode("openai", config.providers?.openai) === "direct";
const routedRows = rows.filter((row): row is OpencodeProxyModelRow & { provider: string; id: string } =>
row.native !== true && typeof row.provider === "string" && typeof row.id === "string");
const visibleRouted = new Set<OpencodeProxyModelRow>(filterCatalogVisibleModels(routedRows, config));
Comment thread
lidge-jun marked this conversation as resolved.
const seen = new Set<string>();
const catalog: OpencodeCatalogModel[] = [];
for (const row of rows) {
const namespaced = row.namespaced?.trim();
if (!namespaced || row.disabled === true) continue;
if (omitNative && row.native === true) continue;
if (row.native !== true && typeof row.provider === "string" && typeof row.id === "string"
&& !visibleRouted.has(row)) continue;
if (seen.has(namespaced)) continue;
seen.add(namespaced);
catalog.push({
Expand Down Expand Up @@ -632,14 +637,14 @@ export function opencodeNotFoundHint(
}

export async function cmdOpencode(args: string[]): Promise<number> {
const config = loadConfig();
const live = await ensureProxyForOpencode(config);
const startupConfig = loadConfig();
const live = await ensureProxyForOpencode(startupConfig);
if (!live) {
console.error("❌ Proxy did not become healthy after starting.");
return 1;
}

const apiKey = opencodeApiKey(config);
const apiKey = opencodeApiKey(startupConfig);
let proxyModels: OpencodeProxyModelRow[];
try {
proxyModels = await fetchOpencodeProxyModels(live, apiKey);
Expand All @@ -648,6 +653,8 @@ export async function cmdOpencode(args: string[]): Promise<number> {
console.error(`❌ Could not fetch the model catalog from the proxy: ${reason}`);
return 1;
}
// /api/models may have completed and persisted initial provider selection.
const config = loadConfig();
const catalog = opencodeCatalogFromProxyRows(proxyModels, config);
const blocks = buildOpencodeProviderBlocksFromCatalog(live.port, catalog, live.hostname, config);
const baseUrl = blocks.v1.options.baseURL;
Expand Down
6 changes: 5 additions & 1 deletion src/server/management/model-rows.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import type { CatalogModel } from "../../codex/catalog";
import {
catalogModelSlug,
filterCatalogVisibleModels,
accountBoundNativeOpenAiSlugsBySelector,
nativeDefaultReasoningEffort,
NATIVE_OPENAI_MODELS,
Expand Down Expand Up @@ -217,5 +218,8 @@ export function toExportModel(row: ManagementModelRow): ExportModel {
*/
export async function loadExportModels(config: OcxConfig): Promise<ExportModel[]> {
const rows = await listManagementModelRows(config);
return rows.filter(row => !row.disabled).map(toExportModel);
// Management deliberately lists the full roster so hidden models can be enabled.
// A client picker must also honor the provider selection, not just its blocklist.
const visibleRouted = new Set(filterCatalogVisibleModels(rows.filter(row => !row.native), config));
return rows.filter(row => !row.disabled && (row.native || visibleRouted.has(row))).map(toExportModel);
}
7 changes: 7 additions & 0 deletions structure/09_client-integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,13 @@ TOML temporal scalars cannot survive the JSON-cloned merge representation with t
intact. The common parser refuses documents containing them before either status or mutation
proceeds, including nested arrays and inline tables. Quoted date strings remain supported.

## Catalog visibility

Management export and CLI export apply the canonical routed catalog visibility filter before
serialization: provider selections, disabled models, and pending initial selection all constrain
the client roster. The full management list remains available for selection. Native rows retain
their existing visibility rules.

## Fast model selectors

The serving proxy resolves `fastRowAvailable` on every management model row, including its
Expand Down
Loading
Loading