Use tokenizer options for chat template kwargs - #2474
Merged
Baiju Meswani (baijumeswani) merged 1 commit intoAug 27, 2026
Merged
Baiju Meswani (baijumeswani) merged 1 commit into
Baiju Meswani (baijumeswani) merged 1 commit into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR consolidates chat-template keyword arguments (chat_template_kwargs) onto the existing tokenizer UpdateOptions surface, removing the parallel GenAI-specific ApplyChatTemplateWithOptions API while documenting the new option and updating C API tests accordingly. It also updates the pinned ONNX Runtime Extensions commit to pick up the upstream validated/typed kwargs support.
Changes:
- Route typed
chat_template_kwargsthroughOgaUpdateTokenizerOptions/Tokenizer::UpdateOptionsand use the legacyApplyChatTemplatecall path. - Remove the C/C++
ApplyChatTemplateWithOptionsAPI surface from GenAI headers and C API implementation. - Update documentation and tests, and pin
onnxruntime_extensionsto the specified merge commit.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/c_api_tests.cpp | Updates the chat template test to set kwargs via UpdateOptions and call ApplyChatTemplate. |
| src/ort_genai.h | Removes the C++ convenience wrapper ApplyChatTemplateWithOptions. |
| src/ort_genai_c.h | Documents chat_template_kwargs as a tokenizer option and removes the C API ApplyChatTemplateWithOptions declaration. |
| src/ort_genai_c.cpp | Removes the OgaTokenizerApplyChatTemplateWithOptions C API implementation. |
| src/models/model.h | Removes Tokenizer::ApplyChatTemplateWithOptions declaration. |
| src/models/model.cpp | Removes Tokenizer::ApplyChatTemplateWithOptions implementation (now relies on OrtxApplyChatTemplate + options). |
| cmake/deps.txt | Pins onnxruntime_extensions to commit 7192ff774619fb37358783714ec996a438bfdb05. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Route typed chat template kwargs through the existing tokenizer UpdateOptions API, remove the unreleased parallel GenAI API surface, update coverage, and pin the merged Extensions implementation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3783556e-822f-4790-a42b-faee9fb57e6b
jennyf19
force-pushed
the
feature/chat-template-kwargs-update-options
branch
from
August 27, 2026 00:46
9852e92 to
e44fcd9
Compare
kunal-vaishnavi
approved these changes
Aug 27, 2026
Collaborator
|
/azp run macOS arm64 Build |
Baiju Meswani (baijumeswani)
enabled auto-merge (squash)
August 27, 2026 18:59
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Baiju Meswani (baijumeswani)
approved these changes
Aug 27, 2026
Baiju Meswani (baijumeswani)
merged commit Aug 27, 2026
b82601e
into
microsoft:main
16 of 17 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use tokenizer options for chat template kwargs
Summary
UpdateOptionsAPIApplyChatTemplateWithOptionsC and C++ surfacechat_template_kwargsas a serialized JSON-object tokenizer optionUpdateOptions7192ff774619fb37358783714ec996a438bfdb05Why
This is the cleanup discussed after #2421 merged. The existing tokenizer options
API is the forward-compatible surface for extending tokenizer behavior without
adding new C API signatures.
microsoft/onnxruntime-extensions#1108 now makes legacy
ApplyChatTemplateconsume validated, typed
chat_template_kwargs, while preserving core requestvalues such as
messages,tools, andadd_generation_prompt.The GenAI-specific
ApplyChatTemplateWithOptionsAPI was introduced by #2421but has not shipped in a release. Removing it here avoids establishing a
redundant public surface before release.
Validation
Fork-only native validation passed against the merged Extensions commit:
https://github.com/jennyf19/onnxruntime-genai/actions/runs/33018340933
The run built the native C++ tests with guidance enabled and passed:
No production Foundry package pin is changed by this PR. Foundry will consume
the final shape after a GenAI
.Foundrypackage containing this change ispublished.