Send temperature through extra_body; check kwargs against SDK signatures - #15
Merged
Merged
Conversation
The first make llm-smoke failed on every call with "Messages.create() got an unexpected keyword argument 'temperature'": anthropic 1.x removed the sampling parameters from messages.create, though the API and Haiku 4.5 still accept them. No request reached the API; nothing was spent. temperature 0 is part of the comparison with the old project, so it now goes in extra_body, which the SDK merges into the request JSON as is. The identity still records temperature 0 and its hash is unchanged. The fake client accepts any keyword, which is why no test caught this. A new test binds the kwargs we send to the installed SDK's Messages.create and Messages.count_tokens signatures (no network, no key), and another checks the fake client receives exactly those kwargs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first
make llm-smokeafter #14 failed on every call before any request was sent:TypeError: Messages.create() got an unexpected keyword argument 'temperature'. Nothing reached the API, US$0 was spent, and no key appeared in the output.Cause
anthropic 1.x removed
temperature,top_pandtop_kfrommessages.create(). The API did not remove them, and Haiku 4.5 still accepts them. The runner's tests use a fake client that accepts any keyword, so none of them could see the mismatch with the real SDK.Fix
request_paramssendsextra_body={"temperature": 0.0}. The SDK mergesextra_bodyinto the request JSON as is, which is the SDK upgrade guide's advice when the model still accepts the setting and the code depends on it. Temperature 0 is part of the comparison with cost-aware-hybrid-router, so it is moved, not dropped.count_tokensonly ever sentmodel,systemandmessages, so it was not affected. Its arguments now come fromcount_tokens_params(), the one place they are built, asrequest_params()is formessages.create.New tests (for the 2026-09-29 smoke incident)
test_the_arguments_we_send_fit_the_installed_sdk_signaturesbinds the kwargs fromrequest_params()andcount_tokens_params()to the installed SDK'sMessages.createandMessages.count_tokenssignatures. It needs no network and no key.test_the_fake_client_receives_exactly_the_built_argumentschecks that what the client receives is exactly those kwargs. The signature check therefore covers what is actually sent.test_request_matches_the_old_project_settingsnow assertsextra_body["temperature"] == 0.0and that no top-leveltemperatureis sent.Mutation check (backup and restore): 6 mutations, all caught:
count_tokensclassifyorcount_tokenssending extra kwargs of their ownmake lint,make test(423 passed) andmake smokepass locally.