Qwen3.8 Fixed Chat Template

A minimal, auditable patch of the Qwen3.8 chat template. It changes seven things, each motivated by a failure measured on real hardware, and leaves everything else byte-identical to the upstream template it derives from.

The headline fix: reasoning_effort defaults to medium instead of xhigh, because xhigh has an unbounded failure mode — it can consume the entire token budget inside <think> and return empty content, which every downstream layer reads as the model failing.

---

Credits

This is derivative work. Nearly all of the template is other people's.

Source	What it contributed	License
Qwen team	The original chat template — the thinking contract, the XML tool-call format, vision handling. Effectively all of the structure here.	Apache-2.0
Unsloth	Our direct base. Their redistribution already fixes the developer role, merged system messages, and tool calling (see the trailing comment in the upstream file). We patch their version, not the stock Qwen one, so those fixes are retained.	Apache-2.0
froggeric/Qwen-Fixed-Chat-Templates	Prior art. Independently identified several of the same defects (JSON-string tool arguments, system-message placement, tool-message guards). Reviewing v22 confirmed our findings were real and not local misconfiguration. We take a different approach — see below.	Apache-2.0

Licensed Apache-2.0, matching all three.

---

Why not just use froggeric's template?

It is a good template and considerably more ambitious than this one. Use it if you want its extra features. We diverge on three points:

1. It defaults to xhigh; we default to medium. This is the bug that motivated the whole exercise (measurements below).

2. It resolves unknown effort values upward to xhigh; we resolve down to medium.

{#- froggeric v22 #}
{%- else %}
    {%- set _reasoning_effort = 'xhigh' %}

vLLM's API advertises Literal["none","minimal","low","medium","high","xhigh","max"] and passes the value straight through. Under that fallback a client asking for minimal — an explicit request for the least reasoning — silently receives the most expensive setting, and with it the empty-content failure. A wrong-direction default is worse than a loud error. We map the full vocabulary monotonically instead.

3. It changes agent dynamics; we deliberately do not. froggeric v22 adds tool-argument and tool-response truncation, two-tier error escalation, and retry guards. Those are useful, but they alter how an agent loop behaves — which breaks comparability if you are benchmarking models against each other. This template changes prompt correctness only.

The practical difference: 230 lines here versus 330 there, against a 183-line base. Every change is greppable as a {#- FIX: ... #} comment.

---

The seven fixes

1. Default effort is medium, not xhigh ← the important one

reasoning_effort is implemented as nothing but an injected sentence. There is no special token, no sampling change, and critically no budget enforcement. Rendering the same conversation at each level shows the entire mechanism:

--- reasoning_effort=medium      (123 chars rendered)
+++ reasoning_effort=xhigh       (332 chars rendered)
@@ -1,0 +2,2 @@
+Reasoning effort is set to xhigh. Please think carefully through the task, validate key
+assumptions, consider plausible alternatives, and prioritize correctness, ...

So nothing can make it stop. Measured on unsloth/Qwen3.8-27B-NVFP4 (vLLM 0.27.1, DGX Spark GB10), asking for a thread-safe generic LRU cache in Go at stock defaults:

	xhigh (upstream default)	medium (this template's default)
finish_reason	length	stop
completion tokens	26,000 (the entire budget)	10,544
reasoning	97,708 chars	20,038 chars
content	0 chars	14,845 chars

The code produced at medium compiles and passes an independent test suite under -race, including eviction ordering. The capability was never in question — xhigh just never reaches the answer. A benchmark harness scored that empty response as vet=FAIL own=FAIL neutral=FAIL: a false failure caused entirely by the default.

medium is also the only level that injects no instruction, making it the least opinionated choice for a caller who expressed no preference. xhigh remains available — opt in.

2. The full serving vocabulary, mapped monotonically — with a real high

Upstream accepts only xhigh|medium|low (with high silently aliased to xhigh) and raises on anything else. Since vLLM forwards its wider Literal straight into the template, a standards-following client sending minimal or max gets HTTP 400 against a healthy server.

sent	upstream	here
none	(via vLLM) thinking off	thinking off
minimal	HTTP 400	low
low	low	low
medium	medium	medium
high	xhigh	high (own level)
xhigh	xhigh	xhigh
max	HTTP 400	xhigh
anything else	HTTP 400	medium (never upward)

Upstream aliases high to xhigh, so a client asking for high receives the one level with an unbounded failure mode — the same defect as resolving minimal upward, in the other direction. Here the ladder is monotonic and distinct: low < medium < high < xhigh.

high asks for verification of the assumptions and edge cases that can change the answer, and ties reasoning length to task difficulty. It deliberately avoids xhigh's open-ended framing, and it terminates on a condition the model can actually evaluate ("as soon as it is settled") rather than on a token limit it cannot observe.

3. xhigh gets a terminating condition

The upstream xhigh instruction asks the model to "consider plausible alternatives" with no stopping criterion. We append one:

Conclude your reasoning and produce the final answer well before the response limit.

An earlier revision also said "always emit the final answer, even if your analysis is incomplete." That was removed: it invites premature conclusion at the setting people choose precisely for quality, and its effect on xhigh answer quality was never measured. Adding an unmeasured instruction to the quality-max setting is the wrong trade.

This costs nothing, but be clear about what it is: a nudge, not a mechanism. The model has no reliable knowledge of how many tokens it has emitted or what max_tokens is, so the instruction asks it to respect a limit it cannot observe. xhigh running to 26,000 tokens shows how little that class of instruction is worth on its own. It does not make xhigh safe as a default, which is why the default changed too.

We also curb verbosity at the source. The observed failure at xhigh is not thinking too hard — it is restating and re-deriving the same ground, so the instruction now says:

Keep each step concise — depth means covering what matters, not restating or re-deriving it.

This targets the actual failure mode and is cheaper than dropping to a lower effort level, which would also remove the analysis the caller asked for. Unmeasured, like every prompt-level change here — see Limitations.

If your engine supports it, use a real budget instead. vLLM 0.27+ exposes thinking_token_budget as a top-level request field. It is enforced by the sampler (v1/sample/thinking_budget_state.py): it counts tokens since <think> opened and, at exhaustion, masks and bumps the logits to force the </think> tokens — so the model concludes and writes the answer, rather than being cut off mid-thought. Measured on the same Go task, at xhigh:

	no budget	thinking_token_budget: 1500
reasoning	97,708 chars	5,562 chars (budget respected)
content	0 chars	18,512 chars of working Go

{"model": "...", "thinking_token_budget": 1500, "messages": [...]}

This is strictly better than the prompt nudge and better than lowering the effort level, because it bounds reasoning without changing how the model reasons. Requires --reasoning-parser qwen3 so the engine knows the delimiters. Where it is unavailable (llama.cpp, MLX, older vLLM), the medium default plus the terminating sentence are what you have.

4. Tool arguments may be a JSON string

Upstream raises and fails the whole request if tool_calls[].function.arguments is a serialized string rather than an object. Several agent clients send exactly that. We accept both. Dict arguments render unchanged.

5. A conversation starting with a tool message

Upstream guards on loop.previtem, which is undefined for the first message, so a conversation whose first entry is a tool result never opens <|im_start|>user and renders malformed. Fixed with a loop.first guard.

6. System messages after the first block

Upstream raises System message must be at the beginning. Frameworks that inject a system message mid-conversation therefore fail outright. We render it in place instead.

---

Usage

# vLLM
vllm serve <model> --chat-template /path/to/chat_template.jinja

# llama.cpp
llama-server -m <model.gguf> --chat-template-file /path/to/chat_template.jinja --jinja

Or point tokenizer_config.json / chat_template.jinja at it in a local checkpoint directory.

Server-side default without a template change. If you would rather not swap the template, vLLM 0.27+ can pin the effort level directly — this fixes only defect 1, and none of the others:

--default-chat-template-kwargs '{"reasoning_effort":"medium"}'

Tests

python3 test_template.py chat_template.jinja

18 assertions covering all six fixes plus regressions (dict tool arguments, leading system message merging, the XML tool contract, and the <think> generation prompt). All pass.

Known limitations

• Fix 3 is a nudge, not a mechanism (see above). A template renders prompts; it cannot enforce a token budget. Where the engine supports thinking_token_budget, use that instead.
• The effect of fix 3's wording on xhigh answer quality is unmeasured. That covers both the terminating sentence and the concision clause. Both should be A/B'd on a reasoning benchmark before anyone relies on xhigh + this template for quality-critical work.
• The new high level is unmeasured. Its routing is tested (it no longer aliases to xhigh), but whether its instruction actually lands between medium and xhigh in reasoning length and answer quality has not been benchmarked. Treat the ladder as intended-monotonic, not verified-monotonic.
• Fixes 1, 2 and 4-6 are behavioural and covered by tests. Fix 3 — and the high instruction — are the ones whose effect rests on the model's cooperation.

Scope

• Targets Qwen3.8. The same defects likely exist in the 3.5/3.6 templates, but this has only been tested against Qwen3.8-27B.
• Verified against unsloth/Qwen3.8-27B-NVFP4 on vLLM 0.27.1. Not yet exercised on llama.cpp, MLX or LM Studio.
• Not a feature template. If you want tool truncation, retry escalation or JSON tool-call output, use froggeric's.

Provenance of the measurements

Every number above comes from a single-GB10 DGX Spark running unsloth/Qwen3.8-27B-NVFP4 on stock pip install vllm==0.27.1, at the checkpoint's own generation_config.json sampling (temp 1.0 / top_p 0.95 / top_k 20), 2026-08-15. Longer write-up: dgx-spark-setup-guide, Appendix G.