Skip to content

Use tokenizer options for chat template kwargs - #2474

Merged
Baiju Meswani (baijumeswani) merged 1 commit into
microsoft:mainfrom
jennyf19:feature/chat-template-kwargs-update-options
Aug 27, 2026
Merged

Baiju Meswani (baijumeswani) merged 1 commit into
microsoft:mainfrom
jennyf19:feature/chat-template-kwargs-update-options

Conversation

@jennyf19

@jennyf19 jennyf19 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Use tokenizer options for chat template kwargs

Summary

  • route typed chat template kwargs through the existing tokenizer UpdateOptions API
  • remove the parallel GenAI ApplyChatTemplateWithOptions C and C++ surface
  • document chat_template_kwargs as a serialized JSON-object tokenizer option
  • update the chat template C API test to configure kwargs through UpdateOptions
  • pin ONNX Runtime Extensions to Route chat template kwargs through tokenizer options onnxruntime-extensions#1108 merge commit
    7192ff774619fb37358783714ec996a438bfdb05

Why

This is the cleanup discussed after #2421 merged. The existing tokenizer options
API is the forward-compatible surface for extending tokenizer behavior without
adding new C API signatures.

microsoft/onnxruntime-extensions#1108 now makes legacy ApplyChatTemplate
consume validated, typed chat_template_kwargs, while preserving core request
values such as messages, tools, and add_generation_prompt.

The GenAI-specific ApplyChatTemplateWithOptions API was introduced by #2421
but has not shipped in a release. Removing it here avoids establishing a
redundant public surface before release.

Validation

Fork-only native validation passed against the merged Extensions commit:

https://github.com/jennyf19/onnxruntime-genai/actions/runs/33018340933

The run built the native C++ tests with guidance enabled and passed:

CAPITests.ChatTemplate

No production Foundry package pin is changed by this PR. Foundry will consume
the final shape after a GenAI .Foundry package containing this change is
published.

@jennyf19
jennyf19 requested a review from a team as a code owner August 27, 2026 00:10
Copilot AI lite review requested due to automatic review settings August 27, 2026 00:10
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR consolidates chat-template keyword arguments (chat_template_kwargs) onto the existing tokenizer UpdateOptions surface, removing the parallel GenAI-specific ApplyChatTemplateWithOptions API while documenting the new option and updating C API tests accordingly. It also updates the pinned ONNX Runtime Extensions commit to pick up the upstream validated/typed kwargs support.

Changes:

  • Route typed chat_template_kwargs through OgaUpdateTokenizerOptions / Tokenizer::UpdateOptions and use the legacy ApplyChatTemplate call path.
  • Remove the C/C++ ApplyChatTemplateWithOptions API surface from GenAI headers and C API implementation.
  • Update documentation and tests, and pin onnxruntime_extensions to the specified merge commit.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
test/c_api_tests.cpp Updates the chat template test to set kwargs via UpdateOptions and call ApplyChatTemplate.
src/ort_genai.h Removes the C++ convenience wrapper ApplyChatTemplateWithOptions.
src/ort_genai_c.h Documents chat_template_kwargs as a tokenizer option and removes the C API ApplyChatTemplateWithOptions declaration.
src/ort_genai_c.cpp Removes the OgaTokenizerApplyChatTemplateWithOptions C API implementation.
src/models/model.h Removes Tokenizer::ApplyChatTemplateWithOptions declaration.
src/models/model.cpp Removes Tokenizer::ApplyChatTemplateWithOptions implementation (now relies on OrtxApplyChatTemplate + options).
cmake/deps.txt Pins onnxruntime_extensions to commit 7192ff774619fb37358783714ec996a438bfdb05.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread test/c_api_tests.cpp
Comment thread src/ort_genai_c.h
Route typed chat template kwargs through the existing tokenizer UpdateOptions API, remove the unreleased parallel GenAI API surface, update coverage, and pin the merged Extensions implementation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3783556e-822f-4790-a42b-faee9fb57e6b
@jennyf19
jennyf19 force-pushed the feature/chat-template-kwargs-update-options branch from 9852e92 to e44fcd9 Compare August 27, 2026 00:46
@baijumeswani

Copy link
Copy Markdown
Collaborator

/azp run macOS arm64 Build

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@baijumeswani
Baiju Meswani (baijumeswani) merged commit b82601e into microsoft:main Aug 27, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants