Add reasoning controls and preserve reasoning across chat turns

[jennyf19]

Add reasoning controls and preserve reasoning across chat turns

Summary

Adds typed chat_template_kwargs to Chat Completions and preserves prior assistant reasoning for qualified model templates, including Qwen 3.8.

This combines two prompt-identity requirements:

1. Template controls such as enable_thinking, preserve_thinking, and reasoning_effort must be applied consistently to every render and retained-state decision.
2. Templates that explicitly support historical reasoning_content must receive the reasoning already stored in Foundry Local's typed conversation transcript.

Changes

Chat template kwargs

• Parses typed chat_template_kwargs values without flattening booleans, numbers, arrays, or nested objects.
• Supports session defaults, request overrides, JSON payload precedence, and explicit {} clearing.
• Applies or clears kwargs under the shared tokenizer render lock.
• Threads effective kwargs through Generator, Engine, media, and continued-turn prompt rendering.
• Treats effective kwargs as retained-generator identity and rebuilds when they change.
• Supports stable GenAI packages and explicit capability selection for local GenAI builds.

Preserved reasoning

• Retains incoming Chat Completions reasoning_content as reasoning-typed assistant transcript content.
• Preserves reasoning across stored and stateless Responses replay.
• Adds a behavioral template capability probe instead of branching on a model name alone.
• Qualifies preservation only when the loaded template:
  • renders distinct historical reasoning by default;
  • keeps visible assistant content distinct; and
  • omits historical reasoning when preserve_thinking=false.
• Emits assistant reasoning_content into canonical message JSON only for qualified templates.
• Lets the model template own <think> markers and preserve_thinking behavior.
• Uses the same capability, transcript, and effective kwargs for Generator and Engine prompt paths.
• Keeps the existing conservative omission behavior for unqualified templates.

Why

Qwen 3.8 templates read prior assistant reasoning from message.reasoning_content. Foundry Local already classifies and stores generated reasoning separately, but previously omitted it when rebuilding canonical message JSON. Stateless Chat Completions also parsed incoming reasoning_content without retaining the bytes.

As a result, reconstructed prompts could contain empty historical thinking blocks and diverge from the conversation the model actually generated. This change makes the canonical prompt represent the typed transcript for templates that explicitly support preserved reasoning, while exact token-prefix comparison remains the final authority for retained-state reuse.

Compatibility

• No reasoning is inserted into visible content.
• Unqualified templates do not receive reasoning_content.
• Existing tool-call, raw-envelope, positional-result, rollback, cancellation, and exact-prefix behavior remains in place.
• Qwen 3.8 support uses the existing GenAI qwen3_5_text model type and is qualified by rendered template behavior.

Validation

• Added focused tests for Chat Completions reasoning ingestion and ordering.
• Added policy-driven message-projection tests.
• Added positive and fail-closed preserved-reasoning capability-probe tests.
• Updated warm/cold Responses replay tests for typed reasoning retention.
• Updated stateless Responses reasoning replay tests.
• Standalone 18-case chat-template kwargs CMake capability matrix passes.
• git diff --check passes.
• Editor diagnostics report no errors in changed source and test files.

[vercel]

Someone is attempting to deploy a commit to the MSFT-AIP Team on Vercel.

A member of the Team first needs to authorize it.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

Stable and release builds remain pinned to a GenAI version that lacks the required tokenizer-option support, and rendering/cache behavior lacks coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds typed chat-template kwargs to OpenAI-compatible chat completions and propagates them through prompt rendering.

Changes:

• Preserves typed and nested JSON kwargs during conversion.
• Applies kwargs across text, media, and continued-turn rendering.
• Invalidates cached generators when kwargs change.

File summaries

File	Description
.pipelines/foundry-local-packaging.yml	Updates non-release GenAI dependency.
sdk_v2/cpp/src/contracts/chat_completions.h	Adds the request field.
sdk_v2/cpp/src/contracts/chat_completions.cc	Deserializes and validates kwargs.
sdk_v2/cpp/src/contracts/chat_completions_converter.cc	Serializes kwargs into request options.
sdk_v2/cpp/src/inferencing/generative/preprocessor.h	Declares option-aware template rendering.
sdk_v2/cpp/src/inferencing/generative/preprocessor.cc	Configures tokenizer template kwargs.
sdk_v2/cpp/src/inferencing/generative/toolcalling/tool_call_context.h	Carries serialized kwargs.
sdk_v2/cpp/src/inferencing/generative/chat/chat_template.h	Extends prompt-building interface.
sdk_v2/cpp/src/inferencing/generative/chat/chat_template.cc	Forwards kwargs during rendering.
sdk_v2/cpp/src/inferencing/generative/chat/onnx_chat_generator.h	Extends continued-turn API.
sdk_v2/cpp/src/inferencing/generative/chat/onnx_chat_generator.cc	Routes kwargs through text and media prompts.
sdk_v2/cpp/src/inferencing/generative/chat/chat_session.cc	Tracks changes and invalidates cached generators.
sdk_v2/cpp/test/internal_api/chat_completions_test.cc	Tests parsing and validation.
sdk_v2/cpp/test/internal_api/chat_completions_converter_test.cc	Tests typed-value conversion.

Review details

• Files reviewed: 14/14 changed files
• Comments generated: 3
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jennyf19]

Baiju Meswani (@baijumeswani) The published GenAI package is now wired into the canonical/release pins, and all three Copilot review findings are addressed and resolved on 9613638c. Fork validation built the full C++ test target against 0.16.0-dev1001407373 and passed the 68 self-contained affected tests: https://github.com/jennyf19/foundry-local/actions/runs/34281760470

Could you please review and authorize the official pipeline with /azp run so the new Qwen2.5/Qwen3.5 model-backed tests execute against the repository test-model cache?

[Baiju Meswani (baijumeswani)]

I found one cache/rollback correctness issue that should be addressed before approval; details are inline.

[Baiju Meswani (baijumeswani)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.