Skip to content

Tool calls whose arguments are echoed with a different scalar type break the ChatGPT chain (Bash) #225

Description

@bman654

Symptom

On a ChatGPT/Codex OAuth model, a turn following a Bash tool call can lose its conversation chain
and re-send the whole history uncached, even though the tool call is plainly the same call.

Evidence

The tool-argument canary already reports this. In a 27.6-hour local diagnostics ledger there are 254
requests whose head-decision diagnostic carries
heads[].mismatch.toolArgumentNormalizationGap — all with equalAfterStrip: false, meaning the
existing filler-strip rule cannot reconcile the two sides:

tool records
Edit 177
Bash 77

The Edit ones have a known cause (the client echoes a tool call with its schema default filled in
— see #214). The Bash ones do not: the real Bash schema declares no defaults at all, across
command, timeout, description, run_in_background and dangerouslyDisableSandbox. So whatever
#214 does, these 77 survive it.

Leading mechanism — scalar type coercion, not filler

A probe against a real Claude Code 2.1.267 binary with canned tool-call responses shows the client
rewriting the type of an argument before echoing it:

model emitted client echoed
"timeout": "5000" timeout: 5000
"run_in_background": "false" run_in_background: false

The head snapshot holds the model's raw arguments, so a function_call with the same call_id and
name compares unequal on the coerced property, and the request falls to
history_mismatch_new_head with full context. That is exactly the equalAfterStrip: false signature
above.

Not yet attributed. The ledger stores hashes rather than content, so it cannot prove coercion
caused those specific 77 records — only that the mechanism is real and produces the same signature.
Confirming it means capturing a real Bash call whose model-emitted arguments carry a string where
the schema wants a number or boolean, then checking the next request's echo. GPT-family models do
emit stringified scalars for optional numeric parameters, which is what makes this reachable.

Notes for whoever picks this up

  • Do not fix it by coercing both sides generally. The comparison exists to notice a genuinely
    re-sent value, and src/oauth/responses-websocket.ts already documents that a changed value under
    the same call_id must stay a mismatch. A type-only reconciliation is narrower: same JSON scalar
    after coercing to the type the tool's schema declares for that property.
  • The schema is already available where it would be needed — requiredToolProps(payload) walks
    tools[].parameters.properties for the same array this would read type from.
  • Whatever shape the fix takes, scope the schema lookup to the request or the head being compared,
    not a process-global map — that is the defect being fixed in fix(oauth): keep the ChatGPT chain alive when the client echoes tool defaults #214's review, and it applies here for
    the same reason: a clodex server serves several clients, and a tool name is not unique across
    them.
  • Worth checking whether the same coercion affects other tools with optional numeric or boolean
    parameters before assuming Bash is special.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions