Skip to content

Validate an explicit Codex reasoning effort against the model's discovered capabilities at run start #980

Description

@pat-lewczuk

Follow-up from #815

#815 added per-task Codex reasoning-effort selection. reasoningEffortSchema is deliberately free-form (z.string().max(64)) so a future App Server value works without a contract release — the right call for the contract. The gap is that nothing then checks the value against the set the selected model actually advertises, even though the catalog already knows it.

Consequence: a typo reaches the wire. reasoningEffort: hgih in a workflow YAML passes validation, passes resolveStepReasoningEffort, and is sent on turn/start. What the App Server does with an unknown effort — reject the turn, or silently ignore it and run at the default — decides whether the user sees an error or just quietly gets less reasoning than they asked for. The cockpit picker is safe (it only offers discovered values); this is about YAML, the CLI, and direct API callers.

The reviewer's request

From @wojciechszyjka's review of 2026-08-10 (🟡 Minor):

reasoningEffortSchema is free-form by design, so a typo reaches the wire. Keeping it z.string().max(64) so future App Server values work without a contract release is well argued, and I agree with it for the contract. But the catalog already knows the valid set for the selected model, and nothing checks a value against it — reasoningEffort: hgih in a workflow YAML sails through validation, through resolveStepReasoningEffort, and into turn/start. A start-time check against the discovered efforts when the catalog is available, degrading to pass-through when it is not, would fail fast without giving up forward compatibility.

Deferred out of #815 deliberately: it adds a new rejection path plus catalog plumbing into the run-start path, on a PR already spanning 57 files, and it is better landed once the manual Codex wire verification that PR is gated on has actually happened — that pass is what tells us whether an unknown effort is rejected or ignored, which changes how loud this check should be.

Acceptance criteria

  • At run start, an explicit reasoningEffort is validated against the selected Codex model's discovered reasoningEfforts when a catalog answer is available, and the run fails fast with a message naming the accepted values.
  • When the catalog is unavailable, stale, or the model is not in it, the value passes through unchanged — forward compatibility with App Server values newer than the local catalog must not regress.
  • The check covers the paths the picker does not: workflow YAML step values, the CLI, and direct POST /runs / POST /runs/:id/continue callers.
  • Tests cover: valid value accepted, typo rejected with the accepted set named, and catalog-unavailable passing through.

Related: #815

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew capabilitypriority-lowCosmetic or follow-up workrisk-lowIsolated, low blast radius

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions