Skip to content

feat: add chat template - #1706

Open
amd-youchen wants to merge 2 commits into
ROCm:mainfrom
amd-youchen:feat-support-chat-template
Open

feat: add chat template#1706
amd-youchen wants to merge 2 commits into
ROCm:mainfrom
amd-youchen:feat-support-chat-template

Conversation

@amd-youchen

Copy link
Copy Markdown
Contributor

Motivation

support custom chat template as passed in arguments

Technical Details

python -m atom.entrypoints.openai_server \
  --model /mnt/md0/models/DeepSeek-V4-Flash-FP8 \
  --kv_cache_dtype fp8 \
  --gpu-memory-utilization 0.83 \
  --served-model-name DeepSeek-V4-Flash-FP8 \
  --chat-template "{% for message in messages %}{% if message['role'] == 'system' %}{{ message['content'] + '\n\n' }}{% elif message['role'] == 'user' %}{{ 'User: ' + message['content'] + '\n\n' }}{% elif message['role'] == 'assistant' %}{{ 'Assistant: ' + message['content'] }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ 'Assistant:' }}{% endif %}" \
  -tp 8

@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Run after the PR is approved and Pre Checkin passes
  • ✅ Run immediately when an approval review is submitted
  • ✅ Can be requested before approval with labels
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 1706 --add-label <label>

@zufayu
zufayu requested a review from yitingw1 July 28, 2026 01:44
yitingw1
yitingw1 previously approved these changes Jul 28, 2026
@amd-youchen
amd-youchen force-pushed the feat-support-chat-template branch 3 times, most recently from 9538f91 to ce7d07b Compare July 28, 2026 02:34
@amd-youchen
amd-youchen force-pushed the feat-support-chat-template branch from ce7d07b to 4b674d0 Compare July 28, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants