Skip to content

Upstream: contribute Gemma 4 reasoning format to llama.cpp #1

Description

@ArtemisAI

Context

strip-gemma exists because llama-server has no working --reasoning-format option for Gemma 4:

  • --reasoning-format deepseek → content field empty, everything in reasoning_content
  • --reasoning-format none → raw <|channel>thought\n...<channel|> tags in content
  • --reasoning-format auto → doesn't recognize Gemma 4's channel tags

Proposed upstream fix

Teach --reasoning-format auto in llama-server to recognize Gemma 4's <|channel>thought\n...<channel|> tag pattern and split it into reasoning_content vs content, the same way it already handles DeepSeek's <think>...</think> tags.

Target files

  • common/chat.cpp — reasoning format parsing/extraction
  • common/common.henum common_reasoning_format (note: comment says "do not extend unless you absolutely have to" — prefer making auto smarter over adding a new enum value)

Related upstream issues

Affected communities (same problem, different stacks)

Acceptance criteria

  • --reasoning-format auto correctly splits Gemma 4 channel tags in non-streaming responses
  • --reasoning-format auto correctly splits Gemma 4 channel tags in streaming (SSE) responses
  • Truncated thinking (no closing tag) handled gracefully
  • No regressions for DeepSeek or other model reasoning formats
  • PR submitted to ggml-org/llama.cpp

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions