Skip to content

A chat's model, effort and Fast belong to the chat, not to a model - #30561

Merged
brennanb2025 merged 36 commits into
mainfrom
brennanb2025/chat-option-values
Oct 20, 2026
Merged

brennanb2025 merged 36 commits into
mainfrom
brennanb2025/chat-option-values

Conversation

@brennanb2025

@brennanb2025 brennanb2025 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 42 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​961 $\color{#cf222e}{\Huge{\mathbf{−}}}$​121 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​840
Prod 49 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​1276 $\color{#cf222e}{\Huge{\mathbf{−}}}$​364 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​912

ELI5

A chat now keeps the model it actually runs. Each chat remembers its own model, effort and speed in one place, the same place the computer running the agent already keeps them, and a chat that never picked anything remembers what its first turn really ran instead of quietly following whatever the agent's default happens to be at each restart. Before, the app filed those choices under each model, so a choice that arrived without a model had nowhere to go, and a chat with no saved model could switch models on its own after the agent's default changed.

What Changed

The problem

A native chat's model, effort and speed controls were kept by the app per model, while the host that runs the chat (this computer, an SSH server or a paired server) keeps one set of choices per conversation. Two owners with two shapes for one fact, so any value that arrived without a model was dropped or hung on an invented model. And a chat that never saved a model (one picked up from a terminal or from Session History, one created before Orca started saving models, or any Codex chat where nobody touched the picker) had no model of its own at all, so it ran whatever the agent's default was at each start. What users saw:

  • The incident tracked in STA-10947: a Claude chat with no saved model silently followed the Claude command-line tool's new default after Claude Code updated, and changed model between sessions. fix(native-chat): a running Claude chat keeps its model when two Claude Code versions list differently #30022 and fix(native-chat): a Claude chat keeps a model its CLI stopped listing #30436 fixed the running-chat cases; chats with no saved choices still followed every default change.
  • A resumed chat that had never picked a model was labelled with the model list's default as if it had been running it, and could send that guess on the next message.
  • A Codex speed (Fast, Flex, Priority) saved under one model could show under a model that doesn't offer it, so the picker showed one speed while the turn ran another, or flipped by itself once the chat started.
  • On a Codex chat with no saved choices, a message that set only an effort also switched the chat to the model list's default model, and a message that set only a model ran that model's list-default effort instead of the effort the thread was using.
  • After switching to a model that doesn't offer a saved effort (for example xhigh on a model that tops out at high), every later Send was refused.
  • An earlier attempt to fix the labels (Keep model choices coherent across Claude updates #30107) stopped inventing the model, and then effort turned disabled and Codex Fast was lost before the first Send after a restart.

What you see now

  • A chat keeps the model it actually runs. A chat with no saved choices records the model, effort, speed tier and Fast setting its first accepted turn really ran, each only where nothing is saved yet. It never records a filled-in default, and a pick you make always wins over a recorded value. After that the chat shows and sends those values as its own, so a later change to the agent's default no longer moves it. This works the same on every agent: Claude, Codex, ACP agents (such as Grok and OpenCode) and Pi.
  • Only what really ran, and only what the chat's model runs, is saved. An option is recorded only if the turn actually ran it and the chat's model offers it. Claude records the effort the command-line tool actually applied, never the effort written in its settings file alone (so a settings effort on a model with no effort control, such as Haiku, is not saved). Codex records the request Orca actually sent. A follow-up message sent into a turn that is already running records nothing of its own, because it runs that turn's settings.
  • One check for every saved choice. The same "does this chat's model offer this value" check applies to every write of a chat's saved choices: a new chat's first choices (including a chat an older app creates), a pick made while the chat is idle, and recording what ran. A pick the model doesn't offer is refused with the existing message instead of being saved. A pick on a running chat is not refused just because the saved model list is coarser than the agent's live menu (for example Pi's saved list stops at high while the running chat offers xhigh).
  • Nothing is cleaned up behind your back. If a saved effort or Fast later stops being offered (for example after a model list update), it stays saved; the existing refusal message and replacement pick handle it, as before. Orca never silently swaps in another value.
  • Reopened chats with no saved model name their default straight away, as main does, but don't send it: their next turn still runs the agent's own choice, and that is what gets recorded. They never send a guessed default.
  • New chats still show the default model and effort, and the first Send pins exactly those, so a later change to the default can't move them. This still holds if the app restarts before the first Send, and if the first Send is refused and you retry.
  • Codex messages that leave a choice unset keep what the thread is running: no silent drop to the model's list-default effort, and no switch to the list's default model when you only pick an effort. Only switching to a different model takes that model's default effort.
  • A model pick drops what the new model can't run. A saved effort or Fast that the newly picked model doesn't offer is dropped, so later Sends are no longer refused. The speed tier keeps the earlier rule: it stays saved and runs as the new model's effective speed.
  • What Send carries is exactly what the controls show: the chat's own choices plus any pick you just made. Same on desktop, for a new chat's first message, and on the phone.
  • Codex speed. The speed shown is the speed the next turn runs. A speed the selected model doesn't offer shows and runs as Standard instead of showing one speed and running another. A speed set in Codex's own config that no model in the list offers (Flex, in practice) keeps showing and running, as on main. Showing Standard never overwrites your pick: switch back to a model that offers it and it shows again.
  • Fast and effort survive an app restart and a model switch whenever the new model offers them. If the host refuses one, you get the existing refusal message instead of the choice silently disappearing.
  • A Claude chat on a model the command-line tool stopped listing keeps that model (unchanged from fix(native-chat): a Claude chat keeps a model its CLI stopped listing #30436).

How it works

  • One selection per conversation. A single flat selection (model, effort, speed, Fast, each optional) replaces the per-model records in native chats. Which controls appear is worked out separately, from the model list and the agent's capabilities, so a value never needs a model to exist.
  • Recording what ran: one shared host step. When the host hands an accepted message to the agent, the agent's adapter returns, in memory only, what that turn actually ran. Claude: the message choices the command-line tool applied, plus the effort it reports applying when the turn runs the model it was started on. Codex: the final request Orca sent, including the effective speed; a follow-up steered into a running turn reports that turn's values. ACP agents and Pi, which have no such evidence, fall back to the message's own values and the agent's own report. A default the host filled in, an empty value, or the settings file alone is never evidence. One step shared by every agent then saves those values, each only where the chat has none saved. A refused message records nothing. A failed write is logged and never blocks the Send; while an option stays unsaved, the next accepted turn tries again. No new saved field and no wire change: the evidence is neither saved nor sent between apps.
  • One admission check inside the store write. Every write of a chat's saved choices (a new chat's first choices, including one created through an older app's attach request; picks while the chat is idle; recording) goes through one ordered update in the record store that checks the values against the chat's model, using the saved model list read beforehand. A listed model must offer the effort, and must support Fast before Fast is saved. If the chat's saved model differs from the model the turn ran, none of that turn's dependent options are recorded, so a pick made meanwhile always wins and values never cross from one model to another. A model pick drops an inherited effort or Fast the new model doesn't offer; an explicit pick the model doesn't offer is refused with the existing message. Unknown model lists and models the list doesn't name pass through as before, and the saved speed tier keeps its earlier rule. The check runs only on new values; it never removes a value already saved.
  • Codex unset choices resolve to the thread's own model and effort (what the thread reported at start), the same rule ACP agents and Pi already follow. A thread effort the model doesn't list is resolved before the request is sent, and the request is what gets recorded.
  • One optional answer field. The options answer the host already sends gains current.defaulted, naming the model or effort values that came from a fallback default rather than from the chat or the agent. The app names such a default on every chat but sends it only from a chat that has never sent anything (a new chat). When the field is missing, the answer came from an older server and the app behaves exactly as main.
  • "Never sent" comes from facts the host already holds. Any submitted message, any turn (even one before a context clear), a rewind, a chat picked up from elsewhere, or missing history all count as "has sent".
  • One speed rule for Codex, in one shared file, used by the picker, the host's answers and each Codex turn: a speed the model lists runs as is; a speed it doesn't list runs as Standard, unless no model in the list offers it and Codex itself reports running it. The host applies the rule to what it sends and answers, never to what it saves.
  • Main merge. Main's Accept messages while any agent starts, and bound each startup step #28046 (messages accepted while an agent starts) now carries each message's choices on the message itself; the merge keeps that, and this PR's recording step is what saves a chat's first-run values.
  • A new chat's create keeps your picks until it succeeds, and pick replies only add or overwrite the values they carry, because several agents reply with only the value that changed.
  • Per-model preferences for terminal launches are unchanged.

Why

The common pattern keeps a conversation's selection on the conversation, separate from the description of what each model supports, and stores the model, effort and speed a run used so the next run starts from them; a conversation never has "no model". This PR adopts that.

This reverses #27925's rule that "later starts without an explicit choice use the current account or folder default", for chats with no saved choices. The maintainer approved the reversal: a chat keeps what it ran, which is what users expect and what the common pattern does.

Alternatives considered and rejected:

  • Patch the per-model store again (a side map for values with no model, plus read-only rows): keeps two copies of the same fact, and made Fast read-only, which is worse than main.
  • Send the shown default from a reopened chat so it gets pinned: the guessed default may not be what the agent is configured to run. That is the STA-10947 incident.
  • Work out a reopened chat's model from its Session History transcript: needs a reader for each agent's transcript format and still guesses; recording what the host handed to the agent covers every agent with one step.
  • Record when the agent starts instead of when a message is accepted: would save values for a start whose message is then refused.
  • Separate speed rules for the picker and the host: this is how the speed bugs above kept coming back. One shared rule means what you see and what runs can't drift apart.
  • Treat every pick reply as the complete set of values: several agents reply with only the changed value, so effort and Fast disappeared until the next refresh.

Differences from the common pattern

  • Intended: each chat keeps a flat set of values, each filled only where nothing is saved, rather than a complete resolved set with defaults filled in for every option. This follows the earlier binding rulings on one selection per chat and per-option recording.
  • Intended: we record the first run's values, not every run's. After that, your picks own later changes, and each message's own choices travel with the message (Accept messages while any agent starts, and bound each startup step #28046), so recording every run would add nothing but extra writes.
  • Intended: a chat's options stay unsaved until a turn has run, and a reopened chat names its default but doesn't send it until then. Orca picks up chats whose model only the agent's own config knows (terminal sessions, Session History, older chats); sending a guessed default is the STA-10947 incident.
  • Intended: the agent's own report, when the turn proves it ran (the effort Claude applied, the request Codex was sent), can supply recorded values, not only values Orca resolved itself. Agents also resolve their own configuration, so only they know what ran; a settings file on its own is still never evidence.
  • Intended: the "does this model offer this value" check happens in one step at the moment of saving, rather than leaving it to the agent to validate or to the moment a turn starts. Orca has several writers that can race (a pick and the recording of a turn), and checking inside the one ordered write protects both.
  • Intended: unknown or custom values pass through when the model list is unknown or doesn't name the model, where the common pattern accepts only standard effort levels for unknown models. Servers and agent tools update independently of the app, and this keeps the existing behavior for models the list doesn't name.
  • Intended: the saved speed tier is kept as your choice while the shown and sent speed follows the model's effective speed, so a model pick keeps the saved tier. This preserves the earlier speed ruling.
  • Intended: an effort or Fast that becomes invalid later stays saved and is handled by the existing refusal message, instead of being silently swapped for another value when a turn runs. This keeps your choice and current behavior; a silent fallback would be a separate product decision.
  • Intended: a Codex speed set only in Codex's own config isn't recorded: it is not in the request Orca sends and Codex doesn't report it, so such a chat keeps following that config for speed.
  • Temporary: after a change to Claude's settings file or the ANTHROPIC_MODEL environment variable, new Claude chats can keep showing, and pin on first Send, the old default; a reopened Claude chat's idle effort label can also be stale until a turn runs. Orca learns Claude's configured default only from a start without a chosen model, and recorded chats now always start with one. Main already has this for chats created in Orca; this PR makes it more common. Follow-up tracked in STA-11271 (have the background model-list check also ask Claude for its configured default).
  • Intended: the host owns one selection per conversation shared by desktop, phone and SSH-hosted chats, instead of a local draft per window, because Orca has several clients per chat. Replies are matched to the chat and session that asked, so a late answer can't overwrite a newer one.
  • Intended: speed handling specific to Codex, following how Codex itself runs an unoffered speed; other agents are unaffected.
  • Intended: with an older server, a saved Priority under a model that doesn't offer it shows as Standard (main showed Priority), because Codex runs Standard there. A live model switch still saves the host's adjusted speed, as on main.
  • Inherited from main: when the model list is unknown or the chat's model is unnamed, the saved value is shown as is.

Deferred / follow-ups

  • Temporary: STA-11271, above.
  • Inherited from main: with Flex set in Codex's own config, an idle chat shows Flex and the outgoing request asks for Flex; QA could not complete a Flex reply to confirm what the provider runs.
  • Temporary: if you pick Flex while Codex's config sets it and later remove Flex from that config, the idle chat still shows Flex while its next turn runs Standard. Nothing the host holds while idle tells the two apart.
  • Temporary: the reply to a pick on an idle chat works out the speed from the account's model list, while the options answer prefers the folder's list. Visible only when those lists differ and the app's follow-up refresh also fails. A one-line follow-up.
  • Inherited from main: a new Claude chat can show another model for a few seconds before the model list loads.
  • Open PRs fix(native-chat): open a New Chat's composer at once, before its host answers #29358 and fix(native-chat): the account's model list follows the newest agent process, not the last answer #30042 touch the same files and will rebase after this lands.

Linked Issue

Tracked in STA-10947. Follow-up tracked in STA-11271. #30107 keeps its model-list work and is updated on top of this after it lands. Builds on #30022 and #30436.

Visual Proof

Screenshots below are from real-app QA of an earlier head (8886f2f) against main, with real signed-in agents in dedicated QA homes. They predate the recording change; QA on the current head is pending (see Testing).

New Claude chat before the first Send (both sent --model opus --effort high and kept Opus High after restart)

main this branch
main: new Claude chat shows Opus High before the first Send this branch: new Claude chat shows Opus High before the first Send

Claude chats after an app restart (this branch; main passed the same checks)

never-sent chat still names its default chat cleared with /clear keeps the model it ran
this branch after restart: never-sent chat shows default Opus High, effort editable this branch after restart: cleared chat keeps Opus High, no default replaces it

Claude chat on a model the command-line tool stopped listing (opus[1m]): its own row stays selected, effort is read-only, and Send passes the model literally

main this branch, after restart
main: retired opus[1m] row selected, effort read-only this branch after restart: retired opus[1m] row kept, effort read-only

Codex speed and other agents

Codex on Fast Codex on Standard OpenCode picker
Codex chat with Fast selected Codex chat with Standard selected OpenCode model picker, unchanged from main

Testing

  • I manually tested these changes locally (ticked once QA on the current head passes)
  • Automated tests added/updated, or explained why not below

QA: real Electron QA on the 1c2e087 build (shipped code identical to the current head) passed the main flows against main: chats keep the model and effort they ran across restart and a CLI version swap, Codex keeps its model and effort after a config change, Haiku with a high settings effort and model picks no longer refuse sends, recorded Fast carries over, every agent re-applies its own model, new chats pin their default, no layout shift. Pi xhigh and real Grok/OpenCode sends in a folder were only partly covered. The original STA-10947 case also passed on the current head: an opus[1m] chat kept opus[1m] and High after a Claude CLI update that stopped listing it and a restart, while main relabelled it to Opus and silently ran plain Opus. Screenshots are in the review status comment.

Main paths QA on the current head must cover: a reopened Claude chat with no saved model names its default, Sends once, and after a restart shows and sends the model it ran; the same for a Codex chat with a configured effort, which keeps that effort after the config changes; an ACP agent (Grok or OpenCode) re-applies the model it reported; a new chat still pins the default it shows; a model pick drops an effort the new model doesn't offer; a Claude chat on Haiku with an effort in the settings file records no effort and keeps sending; a pick the model doesn't offer is refused with the existing message; the Claude launch command line after a recorded Fast.

Earlier real-app QA on 8886f2f (before the main merge and the recording change) passed: new Claude chat default shown, pinned on first Send and kept after restart; never-sent chat still names its default after restart; a chat cleared with /clear keeps its model; a retired Claude model keeps its own row; Codex Flex from Codex's config is sent as Flex; Codex keeps Fast across a startup retry; Grok and OpenCode pickers; no lag from picks. Blocked or not run there: a completed Flex reply (no provider capacity), Codex rewind, Priority to a model without Priority, a refused-then-retried create, older and newer version pairs, and the phone.

Unit tests, run as named files. On the current head, 31 named files (414 tests) pass, plus two further runs of 13 files (205 tests) and 11 files (154 tests) on the admission commits. Files run on the commits since that QA include: src/main/native-chat/agent-session-wire/structured-agent-session-message-choices.test.ts, src/main/native-chat/agent-session-wire/structured-agent-session-conversation-stop.test.ts, src/main/native-chat/agent-session-wire/structured-agent-session-options-read.test.ts, src/main/native-chat/agent-session-wire/structured-agent-session-startup-attempt-host.test.ts, src/main/native-chat/agent-session-wire/structured-agent-session-send-options.test.ts, src/main/native-chat/agent-session-wire/structured-agent-session-permission-mode.test.ts, src/main/native-chat/agent-session-wire/structured-agent-session-host.test.ts, src/main/native-chat/agent-session-wire/structured-conversation-compaction.test.ts, src/main/codex/codex-structured-message-options.test.ts, src/main/codex/codex-structured-dispatch-boundary.test.ts, src/main/codex/codex-structured-turn-steer.test.ts, src/main/native-chat/agent-session-wire/structured-agent-session-operation-start-refusal.test.ts, src/main/codex/codex-structured-service-tier-shown.test.ts, src/main/codex/codex-structured-session-options.test.ts, src/main/claude/claude-structured-message-options.test.ts, src/main/claude/claude-structured-effort-default-at-rest.test.ts, src/main/claude/claude-structured-options.test.ts, src/main/claude/claude-context-usage-unloaded-turn.test.ts, src/main/runtime/structured-agent-session-pi-startup.test.ts, src/main/runtime/structured-agent-session-runtime-provider-started.test.ts, src/main/runtime/structured-agent-session-codex-scripted-child.test.ts, src/main/runtime/claude-structured-retired-saved-model.test.ts, src/main/runtime/structured-agent-new-chat-effort-at-rest.test.ts, src/main/runtime/structured-agent-model-catalog-first-frame.test.ts. Earlier rounds also ran src/shared/structured-agent-session-options.test.ts, src/shared/structured-agent-session-option-view.test.ts, src/shared/structured-agent-session-first-frame-picker.spec.ts, src/renderer/src/lib/structured-agent-session-launch-options.test.ts, src/renderer/src/lib/structured-agent-session-launch-create-message.test.ts, src/renderer/src/components/native-chat/use-structured-agent-session-options.test.tsx, tests/e2e/claude-auto-mode-host-catalog.unit.test.ts and mobile/src/session/use-mobile-structured-agent-options.test.ts.

The new tests follow whole flows, including: options are saved only under the model the chat runs, whichever of a pick and a turn's recording lands first, with no value crossing to another model even when both models offer it, and a value kept after the model list later drops it; Claude records the effort the command-line tool applied rather than the settings file's (including Haiku, where none is applied); a Codex thread effort the model doesn't list is resolved before the request is sent and the request is what gets recorded, and a follow-up steered into a running turn records that turn's values only; a Codex chat records the model and effort it ran (configured xhigh), its config then moves to another model and effort, and the next Send still carries the recorded pair; a Claude start that names only a filled-in default records nothing until the agent's settings report the real model, then records that; a Codex message that sets only an effort keeps the thread's model; Codex Fast, then Standard, then a model pick, then a restart; a speed the new model doesn't offer showing and running as Standard; a resumed chat never sending a default; a new chat pinning the default it shows after a restart and after a refused-then-retried create; both arrival orders of the options answer and the model list; a phone pick whose reply is lost; and an older server's answer behaving as main.

CI: to be confirmed on the current head.

AI Disclosure

Written with Claude Code. Reviewed over several rounds by automated reviewers, with real-app QA driven through the app's developer tools protocol.

Review

Multi-round review completed on 8886f2f. A narrow re-review then covered the main merge and every commit since (recording what ran, the per-option admission check and their fixes) and is done on that narrow scope: every finding is fixed on the current head (3631abf) or labelled above. QA is pending on the current head. The review status comment will be refreshed for the current head.

Agent skill upstream boundary

  • Not applicable, or this change copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.

Notes

  • Mixed versions: the PR's only wire change is the one optional answer field; the recording step adds none. An older server behaves as main: it names and sends the default and records nothing. An older app talking to an updated server shows the recorded values as the chat's own and sends them; for a reopened Claude chat it sends the default it shows, and the updated server records that because it is what ran. An older app also gets the speed the next start runs, in both the idle answer and the reply to an idle pick. A chat an older app creates through its attach request goes through the same "does this model offer this value" check as any other new chat.
  • SSH and paired servers: the host that owns the chat records what ran, produces the field, decides "never sent" and applies the speed rule; every Send from desktop, phone or SSH goes through the same host step. Nothing assumes a local chat.
  • Mobile: the phone uses the same shared selection and Send logic.
  • Cross-platform, security, performance: pure logic. Each Send now reads the saved model list (no network call, no new discovery) and runs one record-store update; it writes to disk only when a turn ran an option the chat had no saved value for, usually once per chat. The "never sent" history scan runs only for a chat with no submitted messages.
  • Main merge formatting: eight files from main that landed unformatted were reformatted by the commit hook during the merge; formatting only.

Checklist

  • This PR is small and focused
  • I explained what changed and why (ELI5, the user-facing before/after, the mechanism, and why over the alternatives)
  • Before/after screenshots or videos attached for UI changes, or N/A with reason
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH/remote, and path/shortcut impact considered (or N/A)
  • pnpm lint, pnpm typecheck, pnpm test, and pnpm build pass (or CI will cover; local preferred)

Tracked in STA-10947.

Author: Brennan Benson (X: @BrennanKB5)

… not to a model

Structured chats kept their option values per model on the client, so a value
the host saved without a model (a resumed chat that never picked one, Codex
Fast saved on the conversation) had nowhere to live. The client either dropped
it or labelled the chat with the listing's default model to hold it.

- One flat record per chat holds model, effort, speed and Fast independently;
  a model pick changes only the model and the host reconciles the rest.
- Controls come from the model they apply to, separately from the values; with
  no model known, a saved Codex speed still shows and Standard turns it off.
- The options answer marks values a catalog default supplied
  (current.defaulted). A resumed chat never names or sends such a model as its
  own; a new chat still shows the default the host says it runs. An answer
  without the mark reads as before.
- Every send, including a new chat's first message, carries the chat's own
  model and effort and held picks, never a default read off the picker.
- A held launch pick supersedes only an earlier pick of the same option.

Tracked in STA-10947.
…l it shows

A new chat that never picked a model carries the default its picker names, as
before, so it keeps the model it actually ran after the CLI's default changes.
Only a resumed chat's defaulted model is never named or sent.

Tracked in STA-10947.
@brennanb2025
brennanb2025 marked this pull request as ready for review October 19, 2026 04:24
@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b3ee0e1b-91cd-482f-8d48-0ac0901b6b9f
📥 Commits

Reviewing files that changed from the base of the PR and between 4f3ef91 and cfc2a14.

📒 Files selected for processing: 7
  • src/main/claude/claude-structured-effort-default-at-rest.test.ts
  • src/main/claude/claude-structured-session-reading-control.test.ts
  • src/main/native-chat/agent-session-wire/structured-agent-session-options-read.test.ts
  • src/main/native-chat/agent-session-wire/structured-agent-session-options-read.ts
  • src/main/native-chat/agent-session-wire/structured-agent-session-registered-definition.test.ts
  • src/main/runtime/structured-agent-new-chat-effort-at-rest.test.ts
  • tests/e2e/claude-auto-mode-host-catalog.unit.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

📝 Walkthrough

The change introduces structured session option records with tracked value sources and default attribution. Host readers report defaulted model and effort values. Snapshot and send-selection logic now use structured option views. Renderer and mobile hooks pass launch context, retain compatible held picks, and provide option callbacks during sends. Tests cover default handling, option refreshes, model changes, and service-tier reconciliation.

Priority: ➖ Normal

Merge Risk: 🔵 Low · up to cfc2a

Resumed Pi chats can treat a fallback model as one the chat selected. The risk is bounded, but the option attribution should be corrected or explicitly accepted before merging.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 59.02% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 61 functions across 39 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly summarizes the primary change: model, effort, and Fast selections now belong to the chat rather than to individual models.
Description check Passed The description is complete and follows the required template. It includes the ELI5 summary, scope, rationale, linked issue, visual proof, testing details, AI disclosure, review notes, compatibility c…
Full details: Description check

Explanation

The description is complete and follows the required template. It includes the ELI5 summary, scope, rationale, linked issue, visual proof, testing details, AI disclosure, review notes, compatibility considerations, and checklist. It also clearly identifies pending current-head QA and CI confirmation.

✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

🟡 Minor · Update the stale structured-agent assertion. · structured-agent-model-catalog-first-frame.test.ts:135-142

src/main/runtime/structured-agent-model-catalog-first-frame.test.ts:135-142
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Update the stale structured-agent assertion.

StructuredAgentSessionOptionRecord no longer exposes model directly. The assertion must read state.record.values.model. The current assertion is also vacuous at runtime because state.record.model is not populated and therefore evaluates to undefined.

Suggested fix
-    expect(state.record.model).toBeUndefined()
+    expect(state.record.values.model).toBeUndefined()

ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fe754e83-e571-4056-ae2a-8e5576e06280
📥 Commits

Reviewing files that changed from the base of the PR and between e37d527 and c4da436.

📒 Files selected for processing: 31
  • mobile/src/session/use-mobile-host-model-catalog-upgrade.ts
  • mobile/src/session/use-mobile-structured-agent-options.ts
  • mobile/src/session/use-mobile-structured-agent-session.ts
  • mobile/src/session/use-mobile-structured-send-with-outcome.ts
  • src/main/claude/claude-structured-effort-default-at-rest.test.ts
  • src/main/claude/claude-structured-model-catalog.ts
  • src/main/claude/claude-structured-session-options.ts
  • src/main/codex/codex-structured-model-catalog.ts
  • src/main/codex/codex-structured-session-options.test.ts
  • src/main/native-chat/agent-session-wire/structured-agent-session-options-read.test.ts
  • src/main/native-chat/agent-session-wire/structured-agent-session-options-read.ts
  • src/main/native-chat/agent-session-wire/structured-agent-session-registered-definition.test.ts
  • src/main/runtime/structured-agent-new-chat-effort-at-rest.test.ts
  • src/renderer/src/components/native-chat/use-host-model-catalog-upgrade.ts
  • src/renderer/src/components/native-chat/use-structured-agent-session-option-state.ts
  • src/renderer/src/components/native-chat/use-structured-agent-session-send-options.ts
  • src/renderer/src/lib/structured-agent-session-launch-create-message.ts
  • src/renderer/src/lib/structured-agent-session-launch-options.test.ts
  • src/renderer/src/lib/structured-agent-session-launch-options.ts
  • src/shared/agent-session-wire.ts
  • src/shared/native-chat-session-option-snapshot.ts
  • src/shared/structured-agent-session-discovered-model.ts
  • src/shared/structured-agent-session-first-frame-picker.spec.ts
  • src/shared/structured-agent-session-option-snapshot.ts
  • src/shared/structured-agent-session-option-state.ts
  • src/shared/structured-agent-session-option-view.test.ts
  • src/shared/structured-agent-session-option-view.ts
  • src/shared/structured-agent-session-options.test.ts
  • src/shared/structured-agent-session-options.ts
  • src/shared/structured-agent-session-permission-option.ts
  • src/shared/structured-agent-session-unlisted-effort.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

Comment thread src/shared/structured-agent-session-options.ts Outdated
…listing names the default

The flag lives only on a listed catalog, so the check confirms the catalog is not the unknown
answer before reading it; the first-frame test reads the chat's flat model value.
…ds the model it now runs

After a retired saved model is reset, the options answer no longer marks the model the child runs
as a default, so the next send records it instead of replaying the refused model on every reopen.
… a model pick keeps only what the model offers

A chat this view started names the default model the host fills in, whichever of the model list and
the options answer lands first, and its first send carries the shown model and effort. A resumed
chat still names and sends only its own values. A model pick, held or live, keeps an effort, speed
or Fast only where the new model offers it, so a value the model cannot take is never shown, sent
or remembered.
… host

A live pick no longer drops values on the client; the host's reply, when it carries its full
value set, replaces the chat's own values as a read does. Only a pick held before launch, with no
host yet, keeps just what the picked model offers.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 95244860-cbdf-41b8-81f1-6f6613ec865d
📥 Commits

Reviewing files that changed from the base of the PR and between 37dcf23 and 715825f.

📒 Files selected for processing: 14
  • mobile/src/session/use-mobile-structured-agent-options.ts
  • src/main/claude/claude-structured-options.test.ts
  • src/main/native-chat/agent-session-wire/structured-agent-session-options-read.ts
  • src/main/runtime/claude-structured-retired-saved-model.test.ts
  • src/main/runtime/structured-agent-new-chat-effort-at-rest.test.ts
  • src/renderer/src/components/native-chat/use-structured-agent-session-option-state.ts
  • src/renderer/src/components/native-chat/use-structured-agent-session-options.test.tsx
  • src/renderer/src/components/native-chat/use-structured-agent-session-options.ts
  • src/renderer/src/lib/structured-agent-session-launch-options.test.ts
  • src/renderer/src/lib/structured-agent-session-launch-options.ts
  • src/shared/structured-agent-session-option-view.test.ts
  • src/shared/structured-agent-session-option-view.ts
  • src/shared/structured-agent-session-options.test.ts
  • src/shared/structured-agent-session-options.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

Comment thread mobile/src/session/use-mobile-structured-agent-options.ts
Comment thread src/main/native-chat/agent-session-wire/structured-agent-session-options-read.ts Outdated
A pick's reply carries only the values the host set or was asked for, so it can't speak for the
rest: it no longer removes values it leaves out. Joined tests pin real reply shapes and a new
Claude chat in a fresh Claude home naming and pinning Opus High.
A Speed value another listed model offers but the named model doesn't is drawn as Standard, which
is what the host reconciles it to, while the chat's record keeps the host's value. Effort, a tier
no listed model offers, an unlisted model and a stand-in model's controls still show what the host
holds. After a lost pick reply the picker no longer shows the old tier as a raw extra choice.
…a reopen

The options answer no longer marks the default model and effort as stand-ins for a chat this host
started that has sent nothing, so a chat reopened after a restart or on the phone names them and
its first send pins them, as before. A chat taken from history, or one that has sent, keeps the mark.
A Codex tier the drawn model lacks shows as Standard, which is how Codex runs it, except Flex, which
Codex passes regardless: established Flex keeps showing, and after a model pick the host hasn't
answered it shows no selection until the reply or a read. Each Codex turn sends the same effective
tier without rewriting the chat's saved one, and a new chat's create carries held picks as the
picker draws them, settling them so they aren't applied again after publish.
…nt needs proof; create keeps held picks

A Codex tier the model doesn't list shows and runs as Standard under one rule shared by the picker,
the host's live reconciliation, each turn and the at-rest answer, which keeps the saved tier for a
model that offers it. An older host's Flex shows as it reports it. A chat counts as never sent only
when no turn, rewind or untrustworthy journal says otherwise. A new chat's create carries held picks
as the picker draws them but keeps them as the user made them until create succeeds, so a refused
create and retry still has the user's own pick, and a newer pick still applies after publish.
…option tests

One typed resting-journal helper replaces six inline mocks and the broad object parameters the
low-evidence check rejects. Cuts tests the joined flows already pin: the resumed list-or-answer
ordering test, the restart-after-first-send view test, the held effort before a model pick, and
the model-pick and restart loop of the saved-Fast test.
Drops the Codex Standard row (the joined tier test pins it), the dispatched-source and literal
effort-choice checks, and the resumed view test restart and model-pick steps; folds the pick table.
… next start runs

The reply to a pick made at rest now carries the tier the chat model runs, through the same rule as
the at-rest answer, so an older app whose follow-up read fails shows what the next turn sends; the
saved record keeps the pick. A saved tier no listed model offers (a Flex from the user Codex config)
is the agent configured tier at rest too, as the live rule and the picker already read it. Restores
the pin that an accepted Codex reply naming Standard shows with no later read.
@brennanb2025

brennanb2025 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Review status: done on 8f1d4a6 (narrow re-review; final main merge checked), CI green, QA done

The problem

A native chat's model, effort and speed were filed by the app under each model, while the computer running the agent (this machine, an SSH server or a paired server) keeps one set per conversation. A chat that never saved a model had no model of its own: it was picked up from a terminal or Session History, was created before Orca saved models, or was any Codex chat where nobody touched the picker. Such a chat ran whatever the agent's default was at each start. That is the STA-10947 incident: a Claude chat silently changed model after Claude Code updated. A reopened chat with no saved model also showed just "Model" until a message started the agent.

What changes for users

  • A chat keeps the model it actually runs. A chat with no saved choices records the model, effort, speed tier and Fast its first accepted turn really ran, each only where nothing is saved. It never records a filled-in default, and your own pick always wins. A later change to the agent's default no longer moves the chat. This works the same for Claude, Codex, ACP agents (Grok, OpenCode) and Pi. It reverses Wait for agent readiness without blocking native chat #27925's "later starts use the current default" for chats with no saved choices, as approved.
  • A reopened chat with no saved model names its default at once, but doesn't send it until a turn has run. A new chat still shows its default and pins it on the first Send.
  • Only what really ran is saved: for Claude, the effort the command-line tool applied, never the settings file alone; for Codex, the request Orca actually sent.
  • One check covers every saved write (new chats, picks at rest, recording): a value is saved only if the chat's model offers it. So a chat can no longer end up with an effort its model refuses, which used to make every Send fail.
  • A Codex message that leaves a choice unset keeps what the thread runs. It no longer silently drops to the model's default effort, and an effort-only pick no longer switches to the list's default model.

What this round fixed (narrow re-review of the main merge, e746e44 and b6d6610)

  • Main merge (c6fe78f): keeps main's per-message options (Accept messages while any agent starts, and bound each startup step #28046) and the PR's design. 8 of main's files only gained oxfmt formatting, because main had merged them unformatted.
  • P1 fixes:
    • After recording, a Codex chat dropped from its thread's effort to the list default.
    • A recorded settings-file effort under a model without effort control (Haiku) made every send fail. This was the second bug of one kind, so the design panel ruled: "only execution facts, through one model-aware admission rule inside the store write".
    • Type and test fixes after the merge.
  • P2/P3 fixes:
    • A follow-up sent during the first turn now joins it.
    • Effort, tier and Fast are now recorded with the model (launcher ruling).
    • A model pick drops an effort or Fast the new model can't run.
    • Fast is recorded only when it really ran on.
    • A live pick isn't refused by a coarser saved model list (Pi xhigh).
    • A new chat is checked against the same per-folder model list its picker showed.
    • Duplicate checks removed.

Deferred or labelled (all in the PR body)

  • Temporary, STA-11271: after a Claude settings or ANTHROPIC_MODEL change, new chats can keep showing and pinning the old default. Orca learns the configured default only from a launch without a model. This already happens on main for chats created in Orca; the fix needs the background model check to ask the CLI for its settings.
  • Intended: a Codex speed set only in Codex's own config isn't recorded, because Codex doesn't report it. A model pick keeps the speed tier (earlier ruling). A value that later becomes invalid stays saved, and the existing refusal and replace path handles it.
  • Concerns noted, not proven: in the ledger.

Verified

  • Final main merge (8f1d4a6): clean, and checked narrowly. Only one production file overlaps, the mobile session hook, and it keeps both main's paging and /clear changes and this PR's send selection. Main's per-agent Auto permission defaults (feat(native-chat): apply per-agent Auto permission defaults #30853) write the permission mode through its own path, which this PR's admission and recording don't touch. CI is green on this head.

  • CI: full PR workflow green on d0a8ab5 ("PR checks passed", run 39358287290); Mobile verify passed; merge state CLEAN.

  • Unit tests: the named test files for every fix pass, including the three joined flows the ruling asked for.

  • QA: real Electron QA on the 1c2e087 build, whose shipped code is identical to d0a8ab5, compared against main. PASS:

    1. A new or reopened Claude chat names a model at once; after Send and restart it keeps the model and effort it ran, even across a CLI swap from 2.1.302 to 2.1.280.
    2. A Codex chat keeps gpt-6-luna/medium after its config changes.
    3. Haiku with settings effortLevel high: the send is accepted.
    4. Effort is kept across model picks, and the send is accepted.
    5. A recorded Fast carries over to the next start.
    6. Grok, OpenCode, Pi, Codex and Claude each re-apply their own model after restart.
    7. A new chat pins its default.
    8. No layout shift (main shifts from Sonnet to Opus).
  • Partial:

    • 5: this build of Pi offers no xhigh; the pick path passes.
    • 6: an ACP chat in a folder displays and picks correctly, but real Grok/OpenCode sends were not run (no load-balancer recipe for them).
  • Same as main, not a regression: a new ACP or Pi chat shows "Model" until a pick.

  • QA, the original STA-10947 case (real Claude through the load balancer, on d0a8ab5 against main a1ea2e39):

    • Setup, same on both builds: with Claude CLI 2.1.280, pick opus[1m] and High and send. Both got a real reply, launched with --model opus[1m] --effort high.
    • Then: switch the CLI to 2.1.302, which no longer lists opus[1m], and restart.
    • Branch (PASS): the picker still shows opus[1m] and High (effort read-only, since the model is no longer listed). The send got a real reply, launched with --model opus[1m] --effort high --resume.
    • Main (FAIL, the original bug): the picker relabels the chat to plain Opus, and the send runs --model opus --effort high, a silent model switch.
    • Hygiene: the real Claude and Codex config hashes were unchanged after the run.

Not verified

  • Real Grok and OpenCode sends in a per-folder chat, because no load-balancer setup exists for them. Their display, pick and model re-apply after restart were verified.
  • Pi xhigh: this Pi build doesn't offer it. The pick path passes.

For Brennan's review

Screenshots: the opus[1m] restart case, branch then main, plus the 2.1.280 starting point:

Branch: after the CLI update and restart, the chat keeps opus[1m]
Main: same restart, the picker relabels to Opus
Main: after Send, it ran plain Opus
Starting point, 2.1.280: opus[1m] and High picked

Other flows (branch):

Flow 1: fresh chat with no saved model
Flow 1: new chat names its default
Flow 1: after switching the Claude CLI to 2.1.280, the chat keeps its model
Flow 3: Haiku with settings effort high, before Send
Flow 3: before Send
Flow 3: after Send, accepted
Flow 4: model pick, before Send
Flow 6: Grok chat in a folder, picked
Flow 8: OpenCode re-applies its model
Baseline main: new chat label

Three places where a human judgment matters:

  1. agent-session-record-options.ts:29 and :114 are the one admission rule every saved option passes. Unknown or unlisted models pass through; a saved value that later becomes invalid is kept, and the existing refusal path handles it.
  2. structured-agent-session-ran-options.ts:89 records what a chat's turn ran. This is the deliberate reversal of Wait for agent readiness without blocking native chat #27925 for chats with no saved choices.
  3. codex-structured-message-options.ts:10: a Codex choice the message leaves unset now keeps the thread's value. One side effect: a chat with a saved model but no saved effort now runs the thread's effort, not the list default.

…ion-values

# Conflicts:
#	src/main/claude/claude-context-usage-unloaded-turn.test.ts
#	src/main/codex/codex-structured-turn-start.ts
#	src/main/native-chat/agent-session-wire/structured-agent-session-options-read.ts
#	src/main/native-chat/agent-session-wire/structured-agent-session-registered-definition.test.ts
#	src/shared/agent-session-wire.ts
…lt at once

A reopened chat that never saved a model showed just "Model" until a send started its agent, while
main names the list default straight away. The picker now names the default the host marks for every
chat, as main does. Whether a send carries that default is a separate rule: a new chat sends it, so
its first send pins it; a reopened chat does not, so its next start still runs the agent's own.
…t turn ran

A chat that never saved a model followed whatever the agent's default was at each start, so a chat
resumed from history or a terminal could quietly switch models after a restart. When a message is
handed to the agent on such a chat, the host now saves the model that turn ran: the message's own
model when it carried one, otherwise the model the agent reports running. A default the host filled
in, no model at all, or a model the agent would refuse as a choice on later sends, is never saved. The write sets the model only while none is saved, inside
the record store's transaction, so a pick made meanwhile always wins and effort, speed and Fast are
untouched. One host step covers Claude, Codex, ACP agents and Pi. No new saved field.
…fort its first turn ran

A Codex message that left the model or effort unset was filled from the model list's defaults, not
from what the running thread uses. Once the host saved the model a chat ran, the next Codex turn
carried that model alone and ran the list's default effort for it (a configured xhigh became
medium), and a message carrying only an effort ran the list's default model while the chat saved
the thread's. A Codex message now resolves an unset model or effort to the thread's own, and only a
switch to a different model takes that model's default effort, the same rule the other agents use.

When a chat with no saved model records what its first turn ran, it now records the effort, service
tier and Fast that turn ran too, in the same one-time store write: each only where the chat has none
saved, only from the message's own value or the agent's real report, never a filled-in default, so
a later change to the agent's configured effort no longer pairs the kept model with an effort it
never ran. No new saved field and no wire change.

A Codex follow-up sent while a chat's first turn runs is compared with what that turn actually runs,
so it steers instead of waiting for a new turn just because the host saved the model meanwhile.

Also fixes the delivery loop's optional recording hook type.
…and drop what a model pick can't run

A Codex message that only switched the tier set the thread's effort as a lasting override, so a later
switch to a model with no listed efforts was refused; the effort is again written only where the
model brought one or an earlier override needs replacing.

Recording what a turn ran now works per option: a chat that already saved its model still records
the effort it ran, and so on for each option it has none saved for. Tier and Fast come only from the
agent's own report, since a message steered into a running turn runs that turn's, and Fast is kept
only when the agent reports it actually routed on: an off state may only mean Fast is unavailable.

A model pick drops a saved effort, or Fast, that the newly picked model's listed row does not offer,
in the one host step both resting and live picks go through, so no later send carries a choice the
agent would refuse. The saved tier is kept, as before: it runs as the model's effective tier.
…the chat's model runs

A chat with a saved model recorded the effort its agent's settings named, not the one the turn
ran: with "effortLevel": "high" in Claude's settings and Haiku saved (no effort control), the chat
saved high and every later send was refused. A Codex thread effort its saved model doesn't list
did the same.

What a turn ran now comes from the agent's own dispatch: Claude reports the choices it applied and
the effort the CLI resolved for the model it applies, never the settings level alone; Codex reports
its final request over the thread's own model and effort, and a steer reports its running turn.
This evidence lives only in memory on the dispatch result; agents without it keep using their own
reports.

One admission rule now decides every saved option, inside the ordered record write: under a model
the saved catalog lists, an effort must be one it offers and Fast must be one it replays (Claude
needs positive support). A late record of a turn that ran on another model than the one the chat
now has saves none of that turn's options. A model pick drops inherited options its new model
doesn't run, an explicit pick or a new chat's option the model doesn't offer is refused with the
existing option refusal, and unknown catalogs or unlisted models pass as before. The saved tier
keeps its effective-tier rule, and nothing already saved is cleaned up later.
…icks are judged inside the write

The handover type was misnamed, which broke the typecheck. Claude and Codex now attach their
dispatch evidence even when it names nothing, so a Claude turn dispatched before its settings
readback never records the settings effort it did not run; the agent's own report still names the
model and stands in only for agents without dispatch evidence. Recording leaves the saved-model
and set-if-unset checks to the store's one admission rule. An explicit effort or Fast pick is now
checked inside the ordered write, with the same refusal, and only at rest: a running chat's own
menu can offer more than its saved catalog row (Pi's probe menu stops at high).
@brennanb2025
brennanb2025 merged commit ad1bcfe into main Oct 20, 2026
37 checks passed
brennanb2025 added a commit that referenced this pull request Oct 20, 2026
Since #30561 a Claude send resolves with the options it ran under; the stop test only needs the send admitted, and the options flow checks the ran options are exactly the per-message pick.
brennanb2025 added a commit that referenced this pull request Oct 21, 2026
#31201)

* test(claude): real-claude suite passes on any signed-in machine and current model aliases

Model ids come from the CLI's own model list instead of dated ids, tests that need a SessionStart hook or an effort bring their own project settings, and the supervised-stop test launches and reads its transcript under one Claude home, as acquisition does.

* test(claude): one real-claude flow for the model, effort and permission pickers and the context meter

Drives the live CLI through the production send path (model list, set model, apply effort, read settings), checks the turn's context report, and relaunches into a new permission mode the way Orca changes it. CI only meets these requests through stand-ins.

* test(claude): accept the ran options a send now reports

Since #30561 a Claude send resolves with the options it ran under; the stop test only needs the send admitted, and the options flow checks the ran options are exactly the per-message pick.

* test(claude): an alias pick's dispatch report may omit the model the CLI resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant