Repository navigation
🧵 fix: Harmony Headers in Text Attachments Break OpenAI Models on Bedrock - #17101
Conversation
…ls on Bedrock A pasted text/plain attachment is sent inline as file context, so a harmony header such as `<|channel|>analysis` inside it reaches every later turn. OpenAI models on Bedrock answer that request with an event stream that closes before any event, which crashed the turn in @librechat/agents 4.0.4 with `Cannot use 'in' operator to search for 'tool_calls' in undefined`. Drive LibreChat's own file-context formatting through the SDK's Bedrock model with a recorded response and assert the header reaches an OpenAI model neutralized and other model families unchanged.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e18eb11c49
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| expect(text).toContain(`# "pasted-text.txt"\nNotes on the format:\n${NEUTRALIZED}`); | ||
| expect(text).not.toContain('<|'); |
There was a problem hiding this comment.
The checked-in package-lock.json still installs @librechat/agents 4.0.4, while the neutralization asserted here is only implemented by the unreleased agents#611 change. Consequently, every locked install running the normal packages/api Jest suite sends the original HEADER and fails this assertion; production also remains unfixed. Update packages/api/package.json and the lockfile to an SDK release containing that change before adding this test.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed, and intended: this PR stays a draft until the @librechat/agents release containing LibreChat-AI/agents#611 is published, and the packages/api/api dependency and lockfile bump lands here before it is marked ready. The spec is checked in first so it demonstrates the failure on 4.0.4 (it fails with the raw header) and passes against the SDK branch build.
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Pull Request
Summary
Agents running an OpenAI model on Bedrock (
global.openai.gpt-6.1-sol) fail withAn error occurred while processing the request: Cannot use 'in' operator to search for 'tool_calls' in undefinedonce the conversation holds a text attachment whose content includes an OpenAI harmony header such as<|channel|>analysis(a pasted model transcript, or notes on the format). LibreChat sendstext/plainattachments inline as file context, so every later turn resends the text. Bedrock answers such a request with200 OKand an event stream that closes before a single event, and@librechat/agents4.0.4 turned the empty stream into{ messages: [undefined] }and crashed collecting tool calls. Images carry no text, other Bedrock model families are unaffected, and a fork made with "Branch without files" works.The fix belongs to the SDK's Bedrock converter, which is the only layer that sees every text the model receives (system prompt, attachments, tool results, fetched pages) and knows the target model: LibreChat-AI/agents#611 neutralizes harmony control tokens for OpenAI models on Bedrock (
<|channel|>becomes<｜channel｜>, which the model still reads), rejects a Bedrock stream that ends without any event with an explanatory error, and reports a missing attempt message as an empty response instead of aTypeError.This PR adds the LibreChat side of the regression coverage: a spec that drives LibreChat's own file-context formatting (
extractFileContext,prependFileContext,formatAgentMessages) through the SDK's Bedrock model with a recorded OpenAI-on-Bedrock response, and asserts the header reaches an OpenAI model neutralized and other model families unchanged. It fails on 4.0.4 and passes against the SDK branch; the@librechat/agentsbump lands here once the release containing LibreChat-AI/agents#611 is published.Depends on LibreChat-AI/agents#611
How it works
Production stack, reproduced in the demo pod with 4.0.4, real Bedrock and a synthetic attachment:
Request path for the attachment:
Type of change
Testing
Live characterization against
global.openai.gpt-6.1-sol(us-west-2, synthetic text): any user text containing<|channel|>analysis(also with spaces or zero-width characters inside the token) returns zero events on every run;<|channel|>final,<|start|>user…, single tokens, and the fullwidth-bar form respond normally. A Bedrockdocumentblock for this model is rejected outright (This model doesn't support documents.), so text attachments only reach it as file context. With the SDK branch, the same attached-file request completes 3/3 throughattemptInvokeagainst real Bedrock.Tested environments/configuration:
global.openai.gpt-6.1-sol, us-west-2Automated tests:
packages/api/src/endpoints/bedrock/harmony.spec.ts: fails on@librechat/agents4.0.4, passes with the 🧵 fix: Neutralize Harmony Control Tokens for OpenAI Models on Bedrock agents#611 build in place.npx tsc --noEmitinpackages/api.Screenshots / recordings
No user-facing change
Risk / compatibility
Text sent to OpenAI models on Bedrock shows harmony tokens with fullwidth bars; the model still reads them (asked which channel a neutralized example uses, it answers
analysis). Other providers and Bedrock model families are untouched.Until the bump ships, a conversation that already holds such an attachment can continue via "Branch without files", or by switching the agent to a non-OpenAI Bedrock model.
Checklist