Skip to content

feat: --extra-body, a request field the server wants and quackd never sends - #22

Merged
rokbenko merged 3 commits into
mainfrom
feat/extra-body
Sep 14, 2026
Merged

rokbenko merged 3 commits into
mainfrom
feat/extra-body

Conversation

@rokbenko

Copy link
Copy Markdown
Owner

Closes #12.

Every request an OpenAI-compatible provider sends is built from a fixed set of keys, so a field quackd has no name for cannot be sent at all. @Vallhalen hit that with Qwen3 on vLLM: thinking is on unless the request body says otherwise, the switch is a chat template argument rather than a sampling parameter, and one step of find-and-kick spent 150 s and 1717 output tokens deliberating before a decision that was correct anyway.

What it does

QUACKD_EXTRA_BODY and --extra-body take one JSON object and hand it to the SDK's own extra_body, which merges it into the top of the request. A passthrough, not a new code path.

  • Nine of the eleven cloud vendors and all five local presets. Anthropic and Gemini have other SDKs, and ignore it as they already ignore --base-url.
  • Both APIs. step() moves a run from Chat Completions to Responses when the API asks it to, and a knob that quietly stops halfway through a run is worse than one the server ignores.
  • The flag beats the variable. An empty object sends nothing, which silences a .env line for one run.
  • Six keys refused: model, messages, input, instructions, tools, stream. instructions is the one that is easy to miss: on Chat Completions the system prompt is the first of messages, but on Responses it is a field of its own, so leaving it out would have let a passthrough replace the whole contract on one API and not the other. Everything else overrides what quackd sends, tool_choice included, because that is the point.
  • A bad value stops the command before anything connects, and before the branch that would have ignored it, so a typo is refused the same way whichever provider was named. That also makes --provider fake the cheapest way to find out whether a shell mangled the quoting.
  • run_start records it. A run whose model was told not to think reads nothing like one that was.

Two things found by running rather than reading

No single spelling of a JSON string survives bash, PowerShell 5.1 and cmd.exe, so the docs show all three and then show the .env line that sidesteps them. And in that file the quoting is not cosmetic: double quotes around JSON make python-dotenv drop the line and set nothing, with no warning, so the run reads as though it had never been written.

The issue said this could not be fixed server-side. It can: vllm serve ... --default-chat-template-kwargs '{"enable_thinking": false}'. That is the better answer for a server you run yourself and no answer at all for one you share, so the docs name both.

Gate

ruff check, ruff format --check against git archive bytes, mypy on 3.11 and 3.12, and the full suite. All green locally on Windows. This PR exists to get macOS, Linux and the physics job.

Reviewed adversarially before opening

A six-lens review over the diff found six things, all fixed here: the instructions gap above; the coordinator-flock path (method: auction) was threaded but only the pilot flock was tested; the scope decision and the run_start emit were both mutation-confirmed unguarded, so both now have tests; "eight cloud vendors" was nine; and docs/architecture.md's run_start row needed the new field.

🤖 Generated with Claude Code

rokbenko and others added 3 commits September 14, 2026 15:58
Every request an OpenAI-compatible provider sends is built from a fixed set of
keys, so a field quackd has no name for cannot be sent at all. Qwen3 on vLLM
needs one: thinking is on unless the request body says otherwise, and the switch
is a chat template argument rather than a sampling parameter, so it cannot be set
at serve time on a server somebody else runs. One step of find-and-kick spent
150 seconds and 1717 output tokens deliberating (#12).

`extra_body` is the SDK's own door for this. It merges the object into the top
level of the request, so this is a passthrough rather than a new code path, and
it reaches every provider that speaks OpenAI's API: nine of the eleven cloud
vendors and all five local presets. Anthropic and Gemini have other SDKs and
other knobs, and ignore it as they already ignore --base-url.

Sent on both APIs. `step` moves a run from Chat Completions to Responses when the
API asks it to, and a knob that quietly stops working halfway through a run is
worse than one the server ignores.

Six keys are refused rather than passed: model, messages, input, instructions,
tools, stream. `model` would put a model on the wire that run_start does not
name, and `stream` changes the shape of the reply without telling the SDK, which
then fails on the content type long after the robot has connected. The other four
are the conversation, and `instructions` is the one that is easy to miss: on Chat
Completions the system prompt is the first of `messages`, but on Responses it is
a field of its own, so leaving it out would have let a passthrough quietly
replace the whole contract on one API and not the other. Everything else goes
through, tool_choice and reasoning_effort included, because overriding those is
the point.

run_start records what was sent. A run whose model was told not to think reads
nothing like one that was, and the transcript is the only place a reader can tell
which of the two they are holding.

Refs #12

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…iable

The variable alone would have been the cheaper change, and for most of the knobs
in this file it is the whole story. This one earns a flag: it is per run rather
than per machine, the value differs between two servers a person points at in the
same afternoon, and the reporter in #12 is on Windows, where a JSON string in a
`.env` file is easier to get right than one on a command line.

Both are read, and the flag wins. The factory parses it and hands the provider
the object, so a provider that was given one never looks at the environment.
An empty object is not nothing, so `--extra-body '{}'` silences a `.env` line for
a single run.

Threaded through every path that builds a provider, which is more than the one
that looks obvious: `run`, `record`, and both kinds of flock. A pilot flock
builds one whole pilot per body and a coordinator flock builds one referee for
all of them, and they are two different call sites, so the test walks all four
routes rather than the one that would have looked like enough.

A bad value stops the command before anything connects, and before the branch
that would have ignored it. Parsing in one place rather than in the three
branches that consume it means a typo is refused the same way whichever provider
was named, which also makes `--provider fake` the cheapest way to find out
whether a shell mangled the quoting. Each door names itself: a flag somebody has
just typed and a line in a `.env` they have forgotten want different answers.

Refs #12

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… for it

A knob nobody can find is a knob nobody has, and this one is harder to find by
reading the source than most: the field it carries belongs to the server, so the
name somebody would search for is never in this repository at all.

The Knobs table gains a row, and the sentence about what is never sent to local
servers now says what can be. The vLLM section gets the case itself, with both
ways to turn Qwen3's thinking off rather than one. The issue said this could not
be fixed on the server side; vLLM's own --default-chat-template-kwargs does
exactly that, once, at serve time. It is the better answer for a server you run
yourself, and no answer at all for one you share, which is what --extra-body is
for. Saying only the second would have been a smaller truth than the reader needs.

Two traps are written down because both were found by running them rather than by
reading. No single spelling of a JSON string survives bash, PowerShell 5.1 and
cmd.exe, so the docs show the shells and then show the .env line that sidesteps
all three. And in that file the quoting is not cosmetic: double quotes around JSON
make python-dotenv drop the line and set nothing, so a run reads as though the
line had never been written.

The browser demo has no such door, and web/README.md keeps the canonical list of
what it does not have, so it is recorded there rather than left to be discovered.
docs/architecture.md's run_start row names the new field, because that table is
what a reader checks a transcript against.

The guard is the house shape: one test naming every file that has to mention it.

Closes #12

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rokbenko
rokbenko merged commit 739ff84 into main Sep 14, 2026
7 checks passed
@rokbenko
rokbenko deleted the feat/extra-body branch September 14, 2026 14:17
rokbenko added a commit that referenced this pull request Sep 14, 2026
Vallhalen's PR #23, merged whole. His commit lands as written; the
corrections are in the commits after this one.

Nothing conflicted. His branch is four commits behind main and touches
the two lines about local model evidence, both of which main had left
alone since #7, so the auto merge took his text onto main's rows without
resurrecting anything stale. The CHANGELOG entry auto merged into
Unreleased, which is where it belongs, so #7's trap did not repeat.

This is the measurement that was asked for on #12 and that #22 could not
make here: every test on main proves the extra_body object reaches the
SDK call, and none of them proves the thinking stops. Eight of eight llm
rows in the thinking-on file carry a thinking field. Zero of five do in
the thinking-off file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

local provider: no way to pass server-specific body params (Qwen3 enable_thinking)

1 participant