She's small. She thinks out loud. Just strip her and get the job done.
A tiny reverse proxy that strips Gemma 4's thinking habits and makes her output client-appropriate. What happens in <|channel>thought stays in reasoning_content.
Gemma 4 has a thinking problem. She always thinks out loud — even when you ask her not to. With llama-server's --reasoning-format none (the only option that doesn't leave content empty), every response arrives fully exposed:
{
"content": "<|channel>thought\nOk let me think about this very carefully...\n2+2... carry the nothing... equals 4!\n<channel|>4"
}Your clients see everything. Your chatbot shows the inner monologue. Your agent tries to parse channel tags. Your users see way more than they signed up for.
Put strip-gemma between your client and llama-server. She handles the wardrobe change:
Client --> strip-gemma :8081 --> llama-server :8080
|
"keep it classy"
Now your responses come out properly dressed:
{
"content": "4",
"reasoning_content": "Ok let me think about this very carefully...\n2+2... carry the nothing... equals 4!"
}Clean content for the audience. Thinking tucked away in reasoning_content for those who want a peek behind the curtain.
curl -fsSL https://github.com/ArtemisAI/strip-gemma/releases/latest/download/strip-gemma-linux-amd64 -o strip-gemma
chmod +x strip-gemma
./strip-gemma -upstream http://localhost:8080docker run --network host ghcr.io/artemisai/strip-gemma -upstream http://localhost:8080go install github.com/ArtemisAI/strip-gemma@lateststrip-gemma [flags]
Flags:
-listen string Address to listen on (default ":8081")
-upstream string Upstream llama-server URL (default "http://localhost:8080")
-mode string 'split' (default) or 'strip'
-version Print version and exit
| Mode | What the client sees | What happens to thinking | Vibe |
|---|---|---|---|
split (default) |
Clean answer | Preserved in reasoning_content |
Tasteful |
strip |
Clean answer | Gone forever | Full censorship |
Gemma 4 wraps her thoughts in channel tags:
<|channel>thought
[internal monologue about life, the universe, and 2+2]
<channel|>[the actual answer]
Non-streaming: strip-gemma reads the full response, separates thinking from answer, and rewrites the JSON. One and done.
Streaming: A three-state machine processes SSE chunks token by token:
- State 0 — Buffering: accumulates tokens looking for the opening tag
- State 1 — Thinking: emits tokens as
reasoning_content(split mode) or suppresses them (strip mode) - State 2 — Presenting: passes answer tokens through as
contentwith zero added latency
No buffering delay once thinking ends. Answer tokens stream the instant <channel|> is seen.
| Scenario | Behavior |
|---|---|
Thinking cut off by max_tokens |
Thinking emitted, content empty |
| No thinking tags at all | Passed through unchanged |
Empty thinking (thought\n<channel|>) |
Content extracted cleanly |
Non-chat endpoints (/health, /metrics) |
Passed through, not touched |
sudo cp strip-gemma /usr/local/bin/
sudo tee /etc/systemd/system/strip-gemma.service << 'EOF'
[Unit]
Description=strip-gemma thinking proxy
After=llama-server.service
Requires=llama-server.service
[Service]
Type=simple
ExecStart=/usr/local/bin/strip-gemma -listen :8081 -upstream http://localhost:8080
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl enable --now strip-gemmaThen point your load balancer at :8081 instead of :8080. She'll take it from there.
Because llama-server's deepseek mode puts everything into reasoning_content and leaves content completely empty. The model's thoughts and answer get tangled together in one field. Known issue with Gemma 4's chat template.
--reasoning-format none + strip-gemma = the only combo that actually works.
| GPU Setup | Quant | Speed | Proxy overhead |
|---|---|---|---|
| RTX 3090 + RTX 3060 | Q6_K | ~19 t/s | < 1ms |
| 2x RTX 3060 | Q4_K_M | ~16.5 t/s | < 1ms |
5.6MB static binary. ~2MB RAM. Zero dependencies. She's low maintenance.
Q: Does this work with models other than Gemma 4?
A: If your model uses <|channel>thought...<channel|> tags, yes. Otherwise she'll just pass everything through untouched.
Q: Does this disable thinking? A: No. Gemma still thinks. strip-gemma just makes sure the audience doesn't see the rehearsal.
Q: Is this name appropriate? A: We're stripping tags from Gemma's output. What were you thinking?
- ArtemisAI/gemma-4-eval — Gemma 4 benchmarks, llama.cpp vs Ollama comparison
- llama.cpp Gemma 4 issues — 35+ open upstream issues
- Google Gemma 4 Model Card
MIT - do whatever you want with her (the code).