Skip to content

[agent_loop, tool] fix: support hermes-format tool calls on gpt-oss tokenizer models #6481

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dafu-wu wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[agent_loop, tool] fix: support hermes-format tool calls on gpt-oss tokenizer models #6481

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion verl/experimental/agent_loop/tool_agent_loop.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,15 @@ def __init__(self, *args, tools: Optional[ToolListWrap] = None, **kwargs):
self.tool_parser = ToolParser.get_tool_parser(self.rollout_config.multi_turn.format, self.tokenizer)
self.tool_parser_name = self.rollout_config.multi_turn.format

# Detect gpt-oss tokenizer by presence of <|channel|> special token.
# When detected, tool responses must be manually formatted (bypassing apply_chat_template)
# regardless of the tool parser, since the gpt-oss jinja template does not support
# standard role:tool messages.
_channel_token_id = self.tokenizer.convert_tokens_to_ids("<|channel|>")
self._is_gpt_oss_tokenizer = (
_channel_token_id is not None and _channel_token_id != self.tokenizer.unk_token_id
)

self.prompt_length = self.rollout_config.prompt_length
self.response_length = self.rollout_config.response_length

Expand Down Expand Up @@ -350,7 +359,7 @@ async def _handle_processing_tools_state(self, agent_data: AgentData) -> AgentSt

agent_data.messages.extend(add_messages)

if self.tool_parser_name == "gpt-oss":
if self.tool_parser_name == "gpt-oss" or self._is_gpt_oss_tokenizer:
logger.info("manually format tool responses for gpt-oss")
tool_response_text = build_gpt_oss_tool_response_text(add_messages, tool_call_names)
Comment thread
dafu-wu marked this conversation as resolved.
response_ids = await self.loop.run_in_executor(
Expand Down