Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ META_API_KEY=
# LOCAL_API_KEY=not-needed
# QUACKD_TOOL_CHOICE=auto # auto (default) | required | none (omit the field)
# QUACKD_VISION=0 # 1 to send camera frames to a local vision model
# A JSON object merged into every request body, for a field the server wants and quackd
# never sends: this one turns Qwen3's thinking off on vLLM. Every provider that speaks
# OpenAI's API reads it, and --extra-body beats it. Single quotes or none, never double
# quotes, which python-dotenv drops without setting anything.
# QUACKD_EXTRA_BODY='{"chat_template_kwargs": {"enable_thinking": false}}'

# Optional: override the model for the chosen provider. A cloud provider takes an id from
# quackd's catalogue and refuses anything else before it calls out, so `quackd list-models`
Expand Down
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- **A body field the server wants and quackd never sends: `--extra-body` and
`QUACKD_EXTRA_BODY`.** One JSON object, merged into the top of every request body on every
provider that speaks OpenAI's API, which is nine of the eleven cloud vendors and all five
local presets, and sent on Chat Completions and Responses both, so it keeps working when a
run moves from one to the other mid-flight. The case that asked for it: Qwen3 on vLLM thinks
before it answers unless the request body says `{"chat_template_kwargs": {"enable_thinking":
false}}`, and that switch is a chat template argument rather than a sampling parameter, so on
a server somebody else runs there was nowhere to say it. One reported step spent 150 s and
1717 output tokens deliberating before a decision that was correct anyway. The flag beats the
variable, an empty object sends nothing, and a value that is not one JSON object is refused
before a robot is connected, naming the flag or the variable it came from. Six keys are
refused because they are quackd's to send — `model`, `messages`, `input`, `instructions`,
`tools` and `stream`, the fourth being the system prompt on Responses the way the second is
on Chat Completions — and everything else replaces what quackd would have sent, `tool_choice`
included, because overriding it is the point. `run_start` records the object, since a run
whose model was told not to think reads nothing like one that was. Anthropic and Gemini
ignore it, as they already ignore `--base-url`. It is in `.env.example` with the others,
where single quotes matter: double ones make python-dotenv drop the line without a word. If
you run the server yourself, vLLM's own `--default-chat-template-kwargs` does the same thing
once at serve time, and the docs now name both. Thanks to
[@Vallhalen](https://github.com/Vallhalen) (#12), who measured it and proposed the
passthrough ([docs/local-llms.md](docs/local-llms.md#knobs)).

- **A bring-up checklist and a lookout task for the LeRobot arm, which were the last two
missing.** Every other experimental backend had both; the arm had neither, and this file
has said so since 0.7. [docs/lerobot-hardware-checklist.md](docs/lerobot-hardware-checklist.md)
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -583,7 +583,7 @@ browser test.
| Model | `--model` or `QUACKD_MODEL`, an id from the catalogue. An id a cloud vendor does not list is refused before any call, and the refusal prints the ids that vendor does take. `quackd list-models` prints them all. The defaults are `claude-opus-5`, `gpt-5.6-sol`, `gemini-3.8-flash`, `grok-4.6`, `mistral-medium-3-5`, `deepseek-flash`, `command-a-plus-05-2026`, `qwen3.8-max`, `kimi-k3`, `glm-5.3` and `muse-spark-1.3` |
| Claude reasoning effort | `QUACKD_EFFORT` (`low` to `max`, default `medium`). `QUACKD_ANTHROPIC_FALLBACKS=0` disables server side refusal fallbacks. `QUACKD_THINKING_DISPLAY=omitted` stops Claude returning a summary of its reasoning, and `QUACKD_GEMINI_THOUGHTS=0` does the same for Gemini |
| OpenAI API and effort | `QUACKD_OPENAI_API=responses` opens on the Responses API instead of Chat Completions, and `QUACKD_OPENAI_REASONING_EFFORT` sets the effort on either. Neither is usually needed: quackd already knows which models want Responses, and moves a run there by itself when one says so ([FAQ](docs/faq.md)) |
| Local models | `--provider ollama`, `vllm`, `llamacpp`, `lmstudio` or `local --base-url http://host:port/v1`. No key. `--model` takes any id the server serves, and without it quackd uses the first model the server lists. The catalogue is for cloud vendors only, so nothing here is refused for being unlisted. `--vision` sends frames. `QUACKD_TOOL_CHOICE=auto`, `required` or `none` for picky servers. See [docs/local-llms.md](docs/local-llms.md) |
| Local models | `--provider ollama`, `vllm`, `llamacpp`, `lmstudio` or `local --base-url http://host:port/v1`. No key. `--model` takes any id the server serves, and without it quackd uses the first model the server lists. The catalogue is for cloud vendors only, so nothing here is refused for being unlisted. `--vision` sends frames. `QUACKD_TOOL_CHOICE=auto`, `required` or `none` for picky servers. `--extra-body` or `QUACKD_EXTRA_BODY` merges a JSON object into every request body, which is how Qwen3 is told not to think on vLLM, and it works on every vendor that speaks OpenAI's API. See [docs/local-llms.md](docs/local-llms.md) |
| Robot | `--robot <adapter>:<backend>` or a name from `quackd robot add`, or a `robots:` line in the `.duck`, the flag wins. Default `microduck:sim2d`. `quackd list-adapters` lists the seven that ship, `quackd list-verbs --robot X` what each can do |
| Physics simulator | `--robot microduck:mujoco`, with `quackd[mujoco]`. The model and the policies are fetched once into `~/.quackd/cache`, where `QUACKD_CACHE_DIR` moves them and `QUACKD_MICRODUCK_ASSETS` points at your own `microduck_rl` checkout instead. `QUACKD_MUJOCO_BODY=puppet` runs the kinematic stand-in, which downloads nothing and is the body the tests build. `--live` opens MuJoCo's own viewer |
| Determinism | `--seed N` makes a simulator run repeatable |
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ One JSON object per line: `{"t": seconds, "kind": ..., ...}`.

| Kind | What it records |
|---|---|
| `run_start` | contract, system prompt, tool names, robot manifest, how long connecting took |
| `run_start` | contract, system prompt, tool names, robot manifest, any `extra_body` sent with every request, how long connecting took |
| `observation` | what the model was shown this turn, and how long gathering it took |
| `llm_request` | how many messages went out, how many still carry an image, whether this is the re-prompt |
| `llm` | text, `thinking`, tool_calls, usage (this turn and the run's total), stop_reason, latency, or `error` when the call failed |
Expand Down
6 changes: 4 additions & 2 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,8 @@ is more than one to choose from: Qwen goes to DashScope's *international* endpoi
(`dashscope-intl.aliyuncs.com`), so a key issued on Alibaba's China console will not
authenticate, and Cohere goes to its OpenAI compatibility path rather than its native one.
`--base-url` moves any vendor that speaks OpenAI's API, which is every one of them except
Anthropic and Gemini, where the flag is accepted and ignored.
Anthropic and Gemini, where the flag is accepted and ignored. `--extra-body` adds a field
that vendor wants and quackd never sends, on the same terms.

**Why did my OpenAI run move to a different API mid-flight?** Because some OpenAI models refuse
function tools on `/v1/chat/completions` at every reasoning effort and name `/v1/responses` in
Expand All @@ -137,7 +138,8 @@ OpenAI's Chat Completions API, so `--provider ollama`, `vllm`, `llamacpp`, `lmst
`local --base-url http://host:port/v1` works with no API key. Tool calling must be enabled
on the server (`llama-server --jinja`, `vllm serve --enable-auto-tool-choice
--tool-call-parser …`), vision is off unless you pass `--vision`, and a small model that
writes its tool call as plain JSON is still understood. Details: [local-llms.md](local-llms.md).
writes its tool call as plain JSON is still understood. A field the server wants in the
body goes in with `--extra-body`, which is how Qwen3 is told not to think on vLLM. Details: [local-llms.md](local-llms.md).

## Seeing what happened

Expand Down
45 changes: 43 additions & 2 deletions docs/local-llms.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,33 @@ quackd run find-and-kick --provider vllm --model Qwen/Qwen3-8B
The `--tool-call-parser` value depends on the model family (`hermes` for Qwen and Hermes
models, `llama3_json` for Llama 3.x, `mistral` for Mistral). vLLM's docs list the pairs.

Qwen3 thinks before it answers unless the request says otherwise, and the switch is a chat
template argument rather than a sampling parameter. One reported step of `find-and-kick` spent
150 s and 1717 output tokens on the reasoning before deciding (#12). There are two places to
turn it off. On a server you run yourself, do it once at serve time:

```bash
vllm serve Qwen/Qwen3-8B --enable-auto-tool-choice --tool-call-parser hermes \n --reasoning-parser qwen3 --default-chat-template-kwargs '{"enable_thinking": false}'
```

On a server somebody else runs, or when you want it per run, send it with the request:

```bash
quackd run find-and-kick --provider vllm --model Qwen/Qwen3-8B \n --extra-body '{"chat_template_kwargs": {"enable_thinking": false}}'
```

That flag is a JSON string, and no single spelling of one survives every shell: the line above
is for bash, PowerShell 5.1 wants `'{\"chat_template_kwargs\": {\"enable_thinking\": false}}'`,
and `cmd.exe` wants the whole thing in double quotes with the inner ones escaped. The way round
all of it is a line in `.env`, which every shell leaves alone:

```
QUACKD_EXTRA_BODY='{"chat_template_kwargs": {"enable_thinking": false}}'
```

Single quotes there, or none. Double quotes around JSON make python-dotenv drop the variable
without setting it, and the run then thinks out loud as though you had never written the line.

**LM Studio**

Developer tab → Start Server (default port 1234), load a model that supports tools, then
Expand Down Expand Up @@ -93,8 +120,22 @@ servers reject image parts. The text observation already carries what the camera
| `--api-key` / `LOCAL_API_KEY` | any string | `not-needed` (servers ignore it) |
| `QUACKD_TOOL_CHOICE` | `auto`, `required`, `none` | `auto` (`none` omits the field for servers that reject it) |
| `--vision` / `QUACKD_VISION` | on, off | off |

`parallel_tool_calls` is never sent to local servers, because some reject unknown fields.
| `--extra-body` / `QUACKD_EXTRA_BODY` | one JSON object, merged into the top of every request body | nothing extra is sent |

`parallel_tool_calls` is never sent to local servers, because some reject unknown fields, and
nothing else is added unless `--extra-body` asks for it.

`--extra-body` works on every provider that speaks OpenAI's API, which is nine of the eleven
cloud vendors and all five local presets, and on Chat Completions and Responses alike, so it
keeps working when a run moves from one to the other. The flag beats the variable, and an empty
object sends nothing, which is how a `.env` line is silenced for a single run. Six keys are
refused because they are quackd's to send: `model`, `messages`, `input`, `instructions`,
`tools` and `stream`. The odd one there is `instructions`, which is the system prompt on the
Responses API the way `messages` carries it on Chat Completions. Everything else replaces what
quackd would have sent, `tool_choice` included, because overriding it is the point. That cuts
both ways: `n` or `response_format` will reach the server too, and what the model answers with
afterwards is yours to live with. In a flock the object goes to every member that speaks
OpenAI's API, and there is no per robot value in the registry.

Add physics by asking for both extras and naming the backend:

Expand Down
4 changes: 4 additions & 0 deletions quackd/agent/loop.py
Original file line number Diff line number Diff line change
Expand Up @@ -462,6 +462,10 @@ async def run(self) -> RunResult:
duck_path=self.duck.path,
provider=cfg.provider.name,
model=cfg.provider.model,
# Fields a passthrough added to every request (#12). A run whose model was told not
# to think reads very differently from one that was, and the transcript is the only
# place a reader can tell which they are holding.
extra_body=getattr(cfg.provider, "extra_body", None),
transport=backend_name(cfg.transport),
adapter=adapter_name(cfg.transport),
robot=manifest.model_dump(mode="json") if manifest is not None else None,
Expand Down
41 changes: 38 additions & 3 deletions quackd/agent/providers/factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

import importlib
import os
from typing import Any

from quackd.agent.providers.base import LLMProvider, ProviderError
from quackd.agent.providers.catalogue import CATALOGUE as CATALOGUE
Expand Down Expand Up @@ -136,6 +137,17 @@ def resolve_model(provider: str, model: str | None, *, source: str = "--model")
raise ProviderError(_unknown_model(provider, model, source))


def _extra_body(text: str | None) -> dict[str, Any] | None:
"""`--extra-body` as the dict a provider takes, parsed here so a bad value names the flag
and stops before a key is read or a packet is sent. None hands the provider nothing, and it
reads `QUACKD_EXTRA_BODY` itself: that is how the flag outranks the variable."""
if text is None:
return None
from quackd.agent.providers.openai import parse_extra_body

return parse_extra_body(text, source="--extra-body")


def make_provider(
name: str,
*,
Expand All @@ -145,8 +157,14 @@ def make_provider(
base_url: str | None = None,
api_key: str | None = None,
vision: bool | None = None,
extra_body: str | None = None,
) -> LLMProvider:
name = name.lower()
# Before the branches, so a typo is refused the same way whichever provider was named,
# `fake` included. Anthropic and Gemini ignore the value as they ignore `--base-url`,
# but ignoring a field is not the same as swallowing a mistake, and `--provider fake`
# is then the cheapest way to find out whether a shell mangled the quoting.
body = _extra_body(extra_body)
if name == "fake":
from quackd.agent.providers.fake import FakeProvider

Expand All @@ -163,7 +181,13 @@ def make_provider(
if name == "openai":
from quackd.agent.providers.openai import OpenAIProvider

return OpenAIProvider(model=model, api_key=api_key, base_url=base_url, vision=vision)
return OpenAIProvider(
model=model,
api_key=api_key,
base_url=base_url,
vision=vision,
extra_body=body,
)
if name == "gemini":
from quackd.agent.providers.gemini import GeminiProvider

Expand All @@ -172,11 +196,22 @@ def make_provider(
module = importlib.import_module(f"quackd.agent.providers.{name}")
vendor = getattr(module, OPENAI_COMPATIBLE[name])
provider: LLMProvider = vendor(
model=model, api_key=api_key, base_url=base_url, vision=vision
model=model,
api_key=api_key,
base_url=base_url,
vision=vision,
extra_body=body,
)
return provider
if name in LOCAL_NAMES:
from quackd.agent.providers.local import LocalProvider

return LocalProvider(model, preset=name, base_url=base_url, api_key=api_key, vision=vision)
return LocalProvider(
model,
preset=name,
base_url=base_url,
api_key=api_key,
vision=vision,
extra_body=body,
)
raise ProviderError(f"unknown provider {name!r}; choose one of {', '.join(PROVIDER_NAMES)}")
2 changes: 2 additions & 0 deletions quackd/agent/providers/local.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ def __init__(
api_key: str | None = None,
tool_choice: str | None = None,
vision: bool | None = None,
extra_body: dict[str, Any] | None = None,
) -> None:
if preset not in PRESETS:
raise ProviderError(f"unknown local preset {preset!r}; one of {', '.join(LOCAL_NAMES)}")
Expand All @@ -148,6 +149,7 @@ def __init__(
base_url=url,
tool_choice=choice,
vision=vision,
extra_body=extra_body,
)
self.text_fallbacks = 0

Expand Down
Loading