Skip to content

fix(provider): consolidate leading system messages to prevent Jinja crash on compaction - #12

Open
adi-IL wants to merge 1 commit into
s0ld13rr:mainfrom
adi-IL:fix/jinja-compaction-system-message
Open

fix(provider): consolidate leading system messages to prevent Jinja crash on compaction#12
adi-IL wants to merge 1 commit into
s0ld13rr:mainfrom
adi-IL:fix/jinja-compaction-system-message

Conversation

@adi-IL

@adi-IL adi-IL commented Aug 25, 2026

Copy link
Copy Markdown

Fixes #11

What was wrong

When running with OpenAI-compatible endpoints or local models (Ollama, vLLM, Qwen, M.O.G.-SEC, Mistral) and reaching auto-compaction or turns with volatile engagement state, the session failed permanently with an HTTP 500 Jinja template error:

Error: Jinja
Exception: System message must be at the beginning.

Root Cause

  1. In packages/opencode/src/session/llm/request.ts, prepare appends volatile engagement context as a secondary system entry (system.push(input.volatileSystem)). When messages is constructed via ...system.map((x) => ({ role: "system", content: x })), multiple discrete { role: "system" } messages are emitted at the head of the message array.
  2. Standard Jinja chat templates (e.g., Qwen, Llama, Mistral) iterate over messages and enforce that role == 'system' is only valid on loop.first (messages[0]). When the loop evaluates messages[1] (role == 'system'), not loop.first evaluates to true, raising System message must be at the beginning.
  3. Similarly, if any chronological system update appeared later in the conversation history, it also triggered this template assertion.

What changed

  1. System Message Normalization (packages/opencode/src/provider/transform.ts):

    • In normalizeMessages, all contiguous leading role: "system" messages are consolidated into a single leading role: "system" message (joining contents with \n\n) for OpenAI-compatible, Ollama, and non-Anthropic providers.
    • For Anthropic models (@ai-sdk/anthropic, @ai-sdk/google-vertex/anthropic), distinct leading system blocks are preserved so prompt caching continues to hit.
    • Any non-leading role: "system" messages appearing after user/assistant turns in history are lowered into user-compatible <system-update>\n...\n</system-update> blocks, preserving chronological context without violating chat template role ordering.
  2. Middleware Transform Support (packages/opencode/src/session/llm.ts):

    • Updated transformParams in wrapLanguageModel to run ProviderTransform.message for both stream and generate request types.
  3. Unit Tests (packages/opencode/test/provider/transform.test.ts):

    • Added unit tests for:
      • Consolidating multiple leading system messages for OpenAI-compatible models.
      • Lowering non-leading system messages to <system-update> blocks.
      • Preserving separate leading system blocks on Anthropic for prompt caching.
      • Single leading system messages and messages without system prompts.

Verification

  • bun test test/provider/transform.test.ts: 295 passed (all new tests passing).
  • bun test test/session/compaction.test.ts: 52 passed.
  • bun turbo typecheck: 19/19 packages passed cleanly with zero type errors.

…rash on compaction

Consolidate multiple contiguous leading system messages into a single system
message for OpenAI-compatible, Ollama, and local model providers, and lower
non-leading system messages in conversation history into user-compatible
<system-update> blocks.

Prevents HTTP 500 Jinja template errors ("System message must be at the
beginning.") when auto-compaction and per-turn engagement state are lowered
across OpenAI-compatible chat completion endpoints (such as Qwen, Ollama,
vLLM, and M.O.G.-SEC models), while preserving distinct leading system
blocks on Anthropic for prompt caching.

Fixes s0ld13rr#11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Jinja crash after auto-compaction: System message must be at the beginning

1 participant