Skip to content

Send temperature through extra_body; check kwargs against SDK signatures - #15

Merged
drewOrc merged 1 commit into
mainfrom
fix/haiku-temperature
Sep 29, 2026
Merged

drewOrc merged 1 commit into
mainfrom
fix/haiku-temperature

Conversation

@drewOrc

@drewOrc drewOrc commented Sep 29, 2026

Copy link
Copy Markdown
Owner

The first make llm-smoke after #14 failed on every call before any request was sent: TypeError: Messages.create() got an unexpected keyword argument 'temperature'. Nothing reached the API, US$0 was spent, and no key appeared in the output.

Cause

anthropic 1.x removed temperature, top_p and top_k from messages.create(). The API did not remove them, and Haiku 4.5 still accepts them. The runner's tests use a fake client that accepts any keyword, so none of them could see the mismatch with the real SDK.

Fix

  • request_params sends extra_body={"temperature": 0.0}. The SDK merges extra_body into the request JSON as is, which is the SDK upgrade guide's advice when the model still accepts the setting and the code depends on it. Temperature 0 is part of the comparison with cost-aware-hybrid-router, so it is moved, not dropped.
  • The identity still records temperature 0, so its hash is unchanged. The failed rows in the smoke journal are retried on the next run.
  • count_tokens only ever sent model, system and messages, so it was not affected. Its arguments now come from count_tokens_params(), the one place they are built, as request_params() is for messages.create.

New tests (for the 2026-09-29 smoke incident)

  • test_the_arguments_we_send_fit_the_installed_sdk_signatures binds the kwargs from request_params() and count_tokens_params() to the installed SDK's Messages.create and Messages.count_tokens signatures. It needs no network and no key.
  • test_the_fake_client_receives_exactly_the_built_arguments checks that what the client receives is exactly those kwargs. The signature check therefore covers what is actually sent.
  • test_request_matches_the_old_project_settings now asserts extra_body["temperature"] == 0.0 and that no top-level temperature is sent.

Mutation check (backup and restore): 6 mutations, all caught:

  • temperature at the top level (also the original bug)
  • temperature dropped
  • temperature added to count_tokens
  • classify or count_tokens sending extra kwargs of their own

make lint, make test (423 passed) and make smoke pass locally.

The first make llm-smoke failed on every call with "Messages.create() got
an unexpected keyword argument 'temperature'": anthropic 1.x removed the
sampling parameters from messages.create, though the API and Haiku 4.5
still accept them. No request reached the API; nothing was spent.

temperature 0 is part of the comparison with the old project, so it now
goes in extra_body, which the SDK merges into the request JSON as is.
The identity still records temperature 0 and its hash is unchanged.

The fake client accepts any keyword, which is why no test caught this. A
new test binds the kwargs we send to the installed SDK's Messages.create
and Messages.count_tokens signatures (no network, no key), and another
checks the fake client receives exactly those kwargs.
@drewOrc
drewOrc merged commit 30e6d66 into main Sep 29, 2026
4 checks passed
@drewOrc
drewOrc deleted the fix/haiku-temperature branch September 29, 2026 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant