feat: [routing] send altimate routing hint and register altimate-auto (Phase 0a client) - #1356
Draft
anandgupta42 wants to merge 2 commits into
Draft
anandgupta42 wants to merge 2 commits into
anandgupta42 wants to merge 2 commits into
Conversation
Phase 0 client instrumentation for the gateway's model-routing work (see
docs/internal/2026-09-22-gateway-model-routing-research.md, client section).
No routing decision changes client-side; this only adds the observability
hint and a new selectable model alias.
- Verified `@ai-sdk/openai-compatible@2.0.41`'s chat language model spreads
unrecognized `providerOptions[<providerID>]` keys directly into the request
body (only `user`/`reasoningEffort`/`textVerbosity`/`strictJsonSchema` are
filtered out), so `metadata` rides through the existing providerOptions
plumbing with no fetch-wrapper hack needed.
- `session/llm.ts`: attach `metadata.altimate` (`task_kind`, `agent`, `tools`,
`session_pos`, `message_id`, optional `min_tier`) to the outgoing body for
the `altimate-free`/`altimate-backend` providers only. Injected in
`stream()` itself (not inside `ProviderTransform.options()`) because
small-model calls use `ProviderTransform.smallOptions()` and never reach
`options()`.
- Stamp `task_kind` explicitly at every call site: main loop / subagent /
summary (`session/prompt.ts`, branching on `session.parentID` and agent
name), title (`ensureTitle`), compaction (`session/compaction.ts`),
skill-selector, enhance-prompt, ai-review, project-copy. Anything else
reports `"other"`.
- `provider/transform.ts`: new `AltimateTaskKind` enum,
`isAltimateManagedModel()` (covers `altimate-base` and `altimate-auto`,
replacing three `id.includes("altimate-base")` checks), and
`isAltimateManagedProviderID()`.
- `provider/provider.ts` + `altimate/free/client.ts`: register a second
hand-registered model, `altimate-auto` ("Altimate Auto"), under the
`altimate-free` provider with the same shape/limits as Altimate Base.
Default-model selection is unchanged — the whole `altimate-free` provider
is already excluded from `Provider.defaultModel()`'s ordinary scan.
- `skill-selector.ts`/`system.ts`/`prompt.ts`: when the session's model is
already Altimate-managed, skill selection reuses it instead of always
resolving `Provider.defaultModel()`. Not applied to `enhance-prompt.ts`,
`ai-review.ts`, or the `skill` tool's own init-time description builder —
none of those call sites currently have a session/model in scope without a
larger refactor (documented in code comments).
- Tests: `test/session/llm.test.ts` adds an end-to-end request-capture test
(real `@ai-sdk/openai-compatible` serialization against a local HTTP
server) asserting the outgoing body carries `metadata.altimate` with the
expected fields, the `"other"` default, and that non-Altimate providers
never get the field.
Verified: marker guard clean (no upstream-shared files touched), `tsgo
--noEmit` clean, and `bun test` green for every touched file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t gate, id join, markers) Fixes for f503676's Codex review ("needs changes"): 1. BLOCKER — rollout gate. `altimate-auto` was registered as an active, pickable model unconditionally, but the gateway 403s the alias until its own Phase 0a rollout ships. Gated registration behind a new `Flag.ALTIMATE_AUTO_MODEL` (env `ALTIMATE_AUTO_MODEL`, default off, read once at `flag.ts` import time like every other Flag.* gate). When off, `provider.ts`'s `baseModels` does not include the `altimate-auto` entry at all — the routing-hint metadata plumbing in session/llm.ts is unaffected either way. Flip the default only after the gateway's Phase 0a allowlist/rewrite deploys. Tests: `provider.test.ts` covers both states — the flag itself (subprocess, matching `external-skills-flag.test.ts`'s established pattern for import-time env flags) and the actual registration behavior in-process (off by default; on via a direct Flag-object mutation, restored after). 2. MAJOR — message_id join with the generation telemetry event. session/llm.ts previously always used `input.user.id` for the hint's `message_id`, but processor.ts's `generation` telemetry event (Telemetry.track({ type: "generation", message_id: input.assistantMessage.id })) uses the assistant message's id — a different id for every call that goes through a processor turn. Added `StreamInput.messageId`, set by processor.ts to `assistantMessage.id` right before calling LLM.stream (covers main, subagent, summary, AND compaction uniformly, since compaction also goes through SessionProcessor.create()/process() and creates its own summary assistant message — note this differs slightly from the review's parenthetical list, which named compaction among "calls with no assistant message"; compaction does have one, and joining it to the same id as its own generation event is the technically correct behavior). Call sites with no processor turn at all set `messageId` explicitly to the message they're about when one exists (title: the first real user message) or leave it unset — `message_id` is then omitted from the hint entirely (skill-selector, enhance-prompt, ai-review, project-copy all construct synthetic per-call messages with no real "message this is about"). Tests: llm.test.ts's omission/explicit-id cases, plus a new processor-effect.test.ts end-to-end test that drives a real main-turn against the real "altimate-backend" provider and asserts the captured request's `metadata.altimate.message_id` equals the assistant message's own id (the same id `generation` telemetry reads). 3. MAJOR — marker gate. Two real gaps: server/routes/.../project-copy.ts:52 and session/prompt.ts:1446 had single-line `// altimate_change — ...` comments instead of a `start`/`end` wrap. Fixed both. Also fixed a pre-existing imbalance in provider/transform.ts from f503676 itself: an Edit there had unintentionally split one existing marked block into two, leaving a duplicate `// altimate_change end` orphaned after `sanitizeSurrogates()` — restructured so the original block (const + sanitizeSurrogates) closes exactly where it did before, and the new routing-hint block (AltimateTaskKind, isAltimateManagedModel, isAltimateManagedProviderID) is its own separate, correctly closed block after it. `bun run script/upstream/analyze.ts --markers --base origin/main --strict` now exits 0. 4. MINOR — hardening + tests. - `isAltimateManagedModel` is now an exact Set membership check ("altimate-base" / "altimate-auto") instead of substring `.includes()`. - The hint's `tools` count is clamped to 512 (the gateway's cap) instead of sent uncapped. - The hint's `agent` field is validated against the gateway's own ^[a-z][a-z0-9_-]{0,31}$ allowlist and dropped (not sent) on a mismatch, rather than relying on the gateway to silently drop an invalid key. - New tests in llm.test.ts: a `small: true` + real "altimate-free" provider path (mirrors the actual title/enhance-prompt/project-copy shape — small-model calls use ProviderTransform.smallOptions(), not .options(), which is exactly why the hint is injected in stream() itself rather than inside options()), and a tools-clamp + invalid-agent-name-dropped case. - New tests in skill-filtering.test.ts covering skill-selector's session-model reuse: an Altimate-managed session model skips Provider.defaultModel() entirely; a non-Altimate session model (or no session model at all) still falls back to it. Verified: marker guard clean, `tsgo --noEmit` clean, and `bun test` green for every touched file. One pre-existing, order-dependent flake was observed once in processor-effect.test.ts (`it.live`-based Effect fiber cleanup racing across combined test files — the same "Unhandled error between tests: All fibers interrupted without error" pattern reproduces on the unmodified "capture llm input cleanly" test when run filtered/isolated) and did not reproduce on three immediate retries of the identical combined file set; not caused by this change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
full receipts (1 session)
builder ·
|
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes # (no issue; first slice of the gateway model-routing plan,
docs/internal/2026-09-22-gateway-model-routing-research.md, Phase 0a)Type of change
What does this PR do?
The gateway cannot tell what kind of LLM call it is serving: every request from the CLI carries the same model id and no hint about whether it is a main agent turn, a subagent, a title, a compaction or a skill-selection call. That blocks any routing decision on the gateway side and it also blocks joining gateway traces to CLI outcomes. This PR adds the client half of the plumbing.
metadata.altimate = { task_kind, agent, tools, session_pos, message_id }in the request body for thealtimate-freeandaltimate-backendproviders only. It rides@ai-sdk/openai-compatible2.0.41'sproviderOptionspassthrough (verified in the SDK source:getArgsspreadsproviderOptions[providerId]into the body, filtering only four known keys). It is injected inLLM.streamrather thanProviderTransform.optionsbecausesmall: truecalls (title, enhance, project-copy) usesmallOptionsand never reachoptions.task_kindat each call site: main / subagent / summary (shared processor call, branched onparentIDand agent name), title, compaction, skill_select, enhance, review, project_copy; everything else isother.message_idis the assistant message id, the same id thegenerationtelemetry event records, so gateway traces and CLI telemetry join on it; processor-less call sites omit it rather than send a throwaway id.toolsis clamped to 512 andagentis validated before sending because the gateway drops invalid keys silently.altimate-autobesidealtimate-baseunder the free provider, behindALTIMATE_AUTO_MODEL=1(default off). The gateway onmainstill returns 403 for any model other thanaltimate-base, so the flag stays off until the gateway's Phase 0a alias lands. When the flag is off the model is not registered at all. Default model selection is unchanged. The threeid.includes("altimate-base")checks intransform.tsbecome an exact-matchisAltimateManagedModelhelper.Provider.defaultModel(). Enhance-prompt and the review AI lane are unchanged because their call chains carry no session; documented in code.Sending the hint before the gateway change is deployed is safe: today's gateway
_strip_client_paramsremoves clientmetadatawithout rejecting the request.How did you verify your code works?
test/session/llm.test.tsdrive the realLLM.stream→ProviderTransform→@ai-sdk/openai-compatible→fetchpath against a localBun.serveserver and assert the outgoing/chat/completionsbody: hint present with the right fields for the backend provider, thesmall: truefree-provider path,otherdefault, and nometadatakey for a non-Altimate provider.test/session/processor-effect.test.ts: a real main turn asserts the hint'smessage_idequals the assistant message id used by thegenerationtelemetry event.test/provider/provider.test.ts:altimate-autoabsent by default and present with the flag (subprocess env test plus in-process registration test).test/altimate/skill-filtering.test.ts: Altimate session model bypassesdefaultModel(); non-Altimate still falls back.packages/opencode:bun run typecheckclean;bun teston the 17 touched/related files: 932 pass, 17 skip, 43 todo, 0 fail.bun run script/upstream/analyze.ts --markers --base origin/main --strict: all custom code in upstream-shared files properly marked.processor-effect.test.tswas observed once in a large combined run and reproduces on the unmodified test when isolated; it is not caused by this change.Screenshots / recordings
Not a UI change. The model picker shows "Altimate Auto" only when
ALTIMATE_AUTO_MODEL=1.Checklist
🤖 Generated with Claude Code
Summary by cubic
Adds the client half of the Altimate gateway model-routing work: requests to
altimate-freeandaltimate-backendnow carry ametadata.altimatehint describing the call, and a newaltimate-automodel becomes available behind a default-off flag.What changed
metadata.altimatewithtask_kind,agent,tools,session_pos, andmessage_id(matching thegenerationtelemetry event id) for Altimate-managed requests only; other providers are untouched.main,subagent,title,summary,compaction,skill_select,enhance,review,project_copy), defaulting tootherwhen unstamped.altimate-autoas a pickable model underaltimate-freeonly whenALTIMATE_AUTO_MODEL=1(default off).Provider.defaultModel().Rollout
metadatawithout rejecting the request.ALTIMATE_AUTO_MODELonly after the gateway’s Phase 0a alias change ships; until then,altimate-autorequests return 403.Written for commit b041cd6. Summary will update on new commits.