Skip to content

Fix parallel tool call input bugs#166

Merged
Ken Jiang (knjiang) merged 5 commits intomainfrom
03-31-add_paralell_tool_calls_test_case
Apr 1, 2026
Merged

Fix parallel tool call input bugs#166
Ken Jiang (knjiang) merged 5 commits intomainfrom
03-31-add_paralell_tool_calls_test_case

Conversation

@knjiang
Copy link
Copy Markdown
Contributor

@knjiang Ken Jiang (knjiang) commented Mar 31, 2026

This PR can be reviewed commit by commit. https://linear.app/braintrustdata/issue/BT-4573/fix-parallel-tools-google-adk-bug-with-anthropic-openai

failures look like:

{
  "error": "400 An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: toolu_nyc",
  "name": "Error"
}

First commit:

  1. ran make capture which does two things:
    a. snapshots the LLM request/responses for the parallelToolCallsRequest
    b. runs transform_request logic by swapping out models for a different provider to "simulate" LLM transforms.
  2. All LLM "passthrough" captures passed and found 2 bugs related to transform logic
  "anthropic_to_chat-completions": {
    "parallelToolCallsRequest": "Bug: parallel tool results in a single Message::Tool are collapsed to one role:tool message; OpenAI rejects with missing tool_call_id"
  },
  "chat-completions_to_google": {
    "parallelToolCallsRequest": "Bug: role:tool messages lack a function name, so functionResponse.name is empty when converting to Google format"
  }
  1. document bugs in transform_errors.json which lets CI pass. the CI fails when there are undocumented bugs.

Second commit:

  1. remove bugs from transform_errors.json
  2. have AI fix the two bugs in transform_errors.json while maintaining lingua best practices
  3. once i was happy with the library-level changes in crates/lingua, I ran make regenerate-failed-transforms
    a. make regenerate-failed-transforms only runs the capture script on transforms that have materially changed based on the snapshots in https://github.com/braintrustdata/lingua/blob/main/payloads/scripts/transforms/__snapshots__/transforms.test.ts.snap
  4. bugs fixed.

@knjiang Ken Jiang (knjiang) changed the title add paralell tool calls test case Fix parallel tool call input bugs Mar 31, 2026
Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@knjiang Ken Jiang (knjiang) marked this pull request as ready for review March 31, 2026 20:44
@knjiang Ken Jiang (knjiang) force-pushed the 03-31-add_paralell_tool_calls_test_case branch from bcd28fc to 30a5404 Compare March 31, 2026 20:57
@knjiang Ken Jiang (knjiang) merged commit 537e368 into main Apr 1, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants