feat: --extra-body, a request field the server wants and quackd never sends - #22
Merged
Merged
Conversation
Every request an OpenAI-compatible provider sends is built from a fixed set of keys, so a field quackd has no name for cannot be sent at all. Qwen3 on vLLM needs one: thinking is on unless the request body says otherwise, and the switch is a chat template argument rather than a sampling parameter, so it cannot be set at serve time on a server somebody else runs. One step of find-and-kick spent 150 seconds and 1717 output tokens deliberating (#12). `extra_body` is the SDK's own door for this. It merges the object into the top level of the request, so this is a passthrough rather than a new code path, and it reaches every provider that speaks OpenAI's API: nine of the eleven cloud vendors and all five local presets. Anthropic and Gemini have other SDKs and other knobs, and ignore it as they already ignore --base-url. Sent on both APIs. `step` moves a run from Chat Completions to Responses when the API asks it to, and a knob that quietly stops working halfway through a run is worse than one the server ignores. Six keys are refused rather than passed: model, messages, input, instructions, tools, stream. `model` would put a model on the wire that run_start does not name, and `stream` changes the shape of the reply without telling the SDK, which then fails on the content type long after the robot has connected. The other four are the conversation, and `instructions` is the one that is easy to miss: on Chat Completions the system prompt is the first of `messages`, but on Responses it is a field of its own, so leaving it out would have let a passthrough quietly replace the whole contract on one API and not the other. Everything else goes through, tool_choice and reasoning_effort included, because overriding those is the point. run_start records what was sent. A run whose model was told not to think reads nothing like one that was, and the transcript is the only place a reader can tell which of the two they are holding. Refs #12 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…iable The variable alone would have been the cheaper change, and for most of the knobs in this file it is the whole story. This one earns a flag: it is per run rather than per machine, the value differs between two servers a person points at in the same afternoon, and the reporter in #12 is on Windows, where a JSON string in a `.env` file is easier to get right than one on a command line. Both are read, and the flag wins. The factory parses it and hands the provider the object, so a provider that was given one never looks at the environment. An empty object is not nothing, so `--extra-body '{}'` silences a `.env` line for a single run. Threaded through every path that builds a provider, which is more than the one that looks obvious: `run`, `record`, and both kinds of flock. A pilot flock builds one whole pilot per body and a coordinator flock builds one referee for all of them, and they are two different call sites, so the test walks all four routes rather than the one that would have looked like enough. A bad value stops the command before anything connects, and before the branch that would have ignored it. Parsing in one place rather than in the three branches that consume it means a typo is refused the same way whichever provider was named, which also makes `--provider fake` the cheapest way to find out whether a shell mangled the quoting. Each door names itself: a flag somebody has just typed and a line in a `.env` they have forgotten want different answers. Refs #12 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… for it A knob nobody can find is a knob nobody has, and this one is harder to find by reading the source than most: the field it carries belongs to the server, so the name somebody would search for is never in this repository at all. The Knobs table gains a row, and the sentence about what is never sent to local servers now says what can be. The vLLM section gets the case itself, with both ways to turn Qwen3's thinking off rather than one. The issue said this could not be fixed on the server side; vLLM's own --default-chat-template-kwargs does exactly that, once, at serve time. It is the better answer for a server you run yourself, and no answer at all for one you share, which is what --extra-body is for. Saying only the second would have been a smaller truth than the reader needs. Two traps are written down because both were found by running them rather than by reading. No single spelling of a JSON string survives bash, PowerShell 5.1 and cmd.exe, so the docs show the shells and then show the .env line that sidesteps all three. And in that file the quoting is not cosmetic: double quotes around JSON make python-dotenv drop the line and set nothing, so a run reads as though the line had never been written. The browser demo has no such door, and web/README.md keeps the canonical list of what it does not have, so it is recorded there rather than left to be discovered. docs/architecture.md's run_start row names the new field, because that table is what a reader checks a transcript against. The guard is the house shape: one test naming every file that has to mention it. Closes #12 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
6 tasks
rokbenko
added a commit
that referenced
this pull request
Sep 14, 2026
Vallhalen's PR #23, merged whole. His commit lands as written; the corrections are in the commits after this one. Nothing conflicted. His branch is four commits behind main and touches the two lines about local model evidence, both of which main had left alone since #7, so the auto merge took his text onto main's rows without resurrecting anything stale. The CHANGELOG entry auto merged into Unreleased, which is where it belongs, so #7's trap did not repeat. This is the measurement that was asked for on #12 and that #22 could not make here: every test on main proves the extra_body object reaches the SDK call, and none of them proves the thinking stops. Eight of eight llm rows in the thinking-on file carry a thinking field. Zero of five do in the thinking-off file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #12.
Every request an OpenAI-compatible provider sends is built from a fixed set of keys, so a field quackd has no name for cannot be sent at all. @Vallhalen hit that with Qwen3 on vLLM: thinking is on unless the request body says otherwise, the switch is a chat template argument rather than a sampling parameter, and one step of
find-and-kickspent 150 s and 1717 output tokens deliberating before a decision that was correct anyway.What it does
QUACKD_EXTRA_BODYand--extra-bodytake one JSON object and hand it to the SDK's ownextra_body, which merges it into the top of the request. A passthrough, not a new code path.--base-url.step()moves a run from Chat Completions to Responses when the API asks it to, and a knob that quietly stops halfway through a run is worse than one the server ignores..envline for one run.model,messages,input,instructions,tools,stream.instructionsis the one that is easy to miss: on Chat Completions the system prompt is the first ofmessages, but on Responses it is a field of its own, so leaving it out would have let a passthrough replace the whole contract on one API and not the other. Everything else overrides what quackd sends,tool_choiceincluded, because that is the point.--provider fakethe cheapest way to find out whether a shell mangled the quoting.run_startrecords it. A run whose model was told not to think reads nothing like one that was.Two things found by running rather than reading
No single spelling of a JSON string survives bash, PowerShell 5.1 and cmd.exe, so the docs show all three and then show the
.envline that sidesteps them. And in that file the quoting is not cosmetic: double quotes around JSON make python-dotenv drop the line and set nothing, with no warning, so the run reads as though it had never been written.The issue said this could not be fixed server-side. It can:
vllm serve ... --default-chat-template-kwargs '{"enable_thinking": false}'. That is the better answer for a server you run yourself and no answer at all for one you share, so the docs name both.Gate
ruff check,ruff format --checkagainstgit archivebytes,mypyon 3.11 and 3.12, and the full suite. All green locally on Windows. This PR exists to get macOS, Linux and the physics job.Reviewed adversarially before opening
A six-lens review over the diff found six things, all fixed here: the
instructionsgap above; the coordinator-flock path (method: auction) was threaded but only the pilot flock was tested; the scope decision and therun_startemit were both mutation-confirmed unguarded, so both now have tests; "eight cloud vendors" was nine; anddocs/architecture.md'srun_startrow needed the new field.🤖 Generated with Claude Code