Tool calls are correctly received but never executed for custom OpenAI-compatible endpoint (Bifrost gateway → vLLM)

[Feuerhamster]

Summary

When using a custom OpenAI-compatible endpoint (a self-hosted Bifrost gateway in front of vLLM, serving GLM-5.2 and gpt-oss-120b), LibreChat correctly sends tools in the request and the upstream model correctly responds with a valid, complete tool_calls payload (verified via a logging man-in-the-middle proxy). However, LibreChat never executes the tool call and never makes the expected second LLM call with the tool result — it simply ends the turn immediately, resulting in an empty/near-empty final assistant message (completionTokens: 4, no visible content).

This reproduces identically for:

• web_search (built-in tool)
• an MCP server's tools (memos)
• multiple models behind the same custom endpoint (GLM-5.2, gpt-oss-120b, Gemma)
• both stream: true and non-streaming title-generation calls
• LibreChat v0.8.7 (latest) and v0.8.6 (downgrade tested, same result)

Environment

• LibreChat version: v0.8.7 (also reproduced on v0.8.6)
• Deployment: Docker Compose, self-hosted
• Endpoint type: endpoints.custom (OpenAI-compatible), pointing at a self-hosted Bifrost gateway (multi-provider LLM proxy), which forwards to a vLLM cluster
• Models tested: vllm/release/glm-5-2, vllm/release/gpt-oss-120b, vllm/release/gemma-4-31b-it
• Web search config: searchProvider: searxng (self-hosted SearXNG) and searchProvider: tavily (both tested, same result) + self-hosted Firecrawl-compatible scraper
• MCP: one streamable-http MCP server (memos) also exhibits the same "tool never executes" symptom

Steps to Reproduce

1. Configure a custom endpoint per the docs, pointing at any OpenAI-compatible backend that supports tool calling (verified working directly via curl, see evidence below).
2. Enable the built-in Web Search feature (webSearch config, any provider).
3. Send a message that requires current information, e.g. "What were yesterday's Formula 1 results?"
4. Observe: the assistant message ends up empty or with a couple of throwaway tokens. No search is performed. No error is shown to the user.

Expected Behavior

1. LLM responds with tool_calls (confirmed happening correctly upstream).
2. LibreChat executes the corresponding tool (web_search, or the MCP tool).
3. LibreChat sends a second request to the LLM with the tool result appended as a role: "tool" message.
4. The LLM produces the final, content-bearing answer.

Actual Behavior

Only step 1 happens. LibreChat's own debug logs show a single [agents:graph] Invoking LLM → LLM call complete → Emitting FINAL event sequence per turn, with completionTokens: 4 and no second LLM invocation and no visible tool execution logs. The chat then simply shows an empty response.

Evidence

To rule out any misconfiguration or upstream (gateway/model) fault, we inserted a transparent logging reverse proxy between LibreChat and the Bifrost gateway (baseURL pointed at the proxy, which forwarded 1:1 to the real gateway and logged both request and response bodies, including full SSE stream reconstruction).

Request received from LibreChat (tools array present, well-formed, single web_search function):

{
  "model": "vllm/release/glm-5-2",
  "stream": true,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "web_search",
        "description": "...",
        "parameters": {
          "type": "object",
          "properties": {
            "query": { "type": "string", "description": "..." },
            "date": { "type": "string", "enum": ["h","d","w","m","y"] },
            "country": { "type": "string", "description": "..." },
            "images": { "type": "boolean" },
            "videos": { "type": "boolean" },
            "news": { "type": "boolean" }
          },
          "required": ["query"]
        }
      }
    }
  ],
  "messages": [ /* system + user message */ ]
}

Response received from the upstream model (reconstructed from the SSE stream, reasoning deltas omitted for brevity): a fully-formed tool call and a correct finish reason:

{
  "choices": [
    {
      "index": 0,
      "finish_reason": "tool_calls",
      "delta": {}
    }
  ],
  "usage": { "prompt_tokens": 1287, "completion_tokens": 99, "total_tokens": 1386 }
}

The accumulated tool_calls delta stream assembles into a fully valid call:

{
  "id": "call_...",
  "type": "function",
  "function": {
    "name": "web_search",
    "arguments": "{\"query\": \"Formel 1 Ergebnisse 24. Juli 2026\", \"date\": \"d\", \"country\": \"de\"}"
  }
}

This was independently reproduced multiple times, for both GLM-5.2 and gpt-oss-120b, with and without a "reasoning"/"reasoning_details" field present in intermediate chunks (we tested stripping reasoning/reasoning_details from every chunk via the proxy to rule out a reasoning-field-name mismatch — no change in behavior).

Meanwhile, LibreChat's own debug log for the same turn shows no second LLM call and no tool-execution log line at all:

debug: [agents:graph] Invoking LLM { messageCount: 1, provider: "openAI", ... }
debug: [agents:graph] LLM call complete (1.01s) { messageCount: 1, ... }
debug: [spendTokens] ... completionTokens: 4
debug: [ResumableAgentController] Emitting FINAL event { wasAbortedBeforeComplete: false, ... }

No error is logged. wasAbortedBeforeComplete is false. The turn is simply treated as complete after the first (tool-call) response, without executing the tool or looping back to the model.

Additional data point (MCP)

The same pattern occurs with an MCP server (memos, streamable-http, 19 tools listed successfully via /api/mcp/tools). Asking the agent to use an MCP tool produces a short assistant message announcing intent ("Ich rufe deine aktuellen Notizen ab...") but the tool is never actually invoked, and the log shows:

debug: [getMCPTools] No tools found for server memos

immediately before the (tool-less) LLM call — even though the same server correctly lists 19 tools via the REST API moments earlier in the same session.

What we ruled out

• Gateway/model correctness: verified via direct curl (non-streaming and streaming) against the same Bifrost endpoint/model — both GLM-5.2 and gpt-oss-120b return well-formed tool_calls with finish_reason: "tool_calls" immediately, no retries needed.
• Search/scraper provider misconfiguration: reproduced identically with searchProvider: searxng and searchProvider: tavily; reproduced identically with different scraper backends.
• reasoning vs reasoning_content field naming (cf. [Bug]: Reasoning content not rendered for custom OpenAI-compatible endpoints (vLLM, SGLang) — hardcoded to deprecated reasoning_content field #12775, [Bug]: v0.8.7-rc1 stops rendering reasoning and breaks tool calling for a custom OpenAI-compatible endpoint (works on v0.8.6, backend verified correct) #13844): tested customParams.reasoningKey, and separately tested a proxy that strips reasoning/reasoning_details entirely from every streamed chunk before it reaches LibreChat — no change in behavior.
• LibreChat version regression: reproduced identically on both v0.8.7 and v0.8.6.
• Model-specific issue: reproduced identically across three different models (GLM-5.2, gpt-oss-120b, Gemma) behind the same custom endpoint.

Suspected Area

Given the evidence, the issue appears to be in the agent orchestration logic that decides whether to execute a tool call and loop back to the LLM for a custom (endpointType: "custom") OpenAI-compatible endpoint specifically — possibly related to how the tool_calls finish reason or accumulated tool call arguments are detected for this endpoint type in @librechat/agents, as opposed to native openAI/azureOpenAI endpoint types (untested by us, we don't have those configured).

[Dunkoro]

We reproduced a closely related failure on LibreChat v0.8.7 with a saved Agent using a custom OpenAI-compatible endpoint:

• The agent successfully invokes its MCP tool.
• The initial completion ends with tool_calls and no assistant content.
• No follow-up model completion is requested, so the user receives an empty response.
• The same backend completes the identical tool-result continuation correctly in both streaming and non-streaming OpenAI Chat Completions probes.

Our temporary workaround is to use LibreChat native provider openAI and route it through an internal OpenAI-compatible reverse proxy via OPENAI_REVERSE_PROXY. That preserves our gateway routing and attribution, but it is global configuration rather than a saved-agent choice.

Could LibreChat either fix the custom endpoint Agent tool loop, or make the native OpenAI transport/reverse-proxy route configurable per endpoint/agent? This would let self-hosters retain OpenAI-compatible gateways without patching code or migrating agents between provider types.

[cfs603]

Confirming the same failure pattern on a different backend; Ollama (not vLLM/Bifrost), models qwen3:8b and qwen3:14b, via the same endpoints.custom OpenAI-compatible connection type described here.

Same signature: the model correctly reasons about which tool to call (visible in the Thoughts/reasoning block), but the turn ends with empty final text and no tool call ever executes; no error surfaced in server logs, the browser console, or the raw SSE stream.

Setup:

• LibreChat v0.8.8-rc1
• endpoints.custom entry pointing at a local Ollama server (http://host.docker.internal:11434/v1/)
• MCP tool server: ha-mcp (Home Assistant), streamable-HTTP, tested with both the full tool catalog (78 tools) and ha-mcp's search-based discovery mode (11 tools, incl. ha_search_tools/ha_call_read_tool/etc.)

What I ruled out while isolating this:

• Setting addParams: { think: false } on the custom endpoint in librechat.yaml (confirmed loaded via server startup logs): no change
• Sending the model-level /no_think directive (Qwen3-specific reasoning suppression): no change
• Tool count/selection: tested with both the full catalog and a reduced/curated tool set: no change
• Model size: tested qwen3:8b and qwen3:14b: identical failure on both

What works, as a control:

• The identical MCP tool server and identical Home Assistant backend execute tool calls correctly when driven by Claude (native Anthropic endpoint, not custom) through this same LibreChat instance
• The identical Ollama model + identical Home Assistant MCP backend also work correctly through Home Assistant's own native Ollama conversation-agent integration (a separate, non-LibreChat client)

Both controls point away from the model, the MCP server, and the backend host, and toward the custom OpenAI-compatible endpoint's tool-execution path specifically: consistent with the orchestration-logic theory already raised in this issue.

Example of the raw SSE stream tail for a failed turn:

```
event: message
data: {"final":true, ... "responseMessage":{ ... "text":"","content":[{"type":"think","think":"...reasoning ending in a decision to call a tool, never executed..."}], ... }}
```

Happy to provide the full request/response trace or test a patch if useful.

[vgmartinez]

Disclosure: TrustGate / NeuralTrust DevRel.

This pattern (OpenAI-compatible gateway → vLLM) often breaks tool calls in one of three places:

1. Streaming vs non-streaming. Some gateways forward tool_calls deltas incorrectly on SSE; LibreChat then "receives" tools in the UI but never gets a completed tool_calls array to execute. Try the same model with streaming off through Bifrost as a bisect.
2. tools / tool_choice passthrough. Confirm Bifrost is not stripping or renaming fields on the way to vLLM (especially tool_calls[].function.arguments as a string vs object).
3. Model actually supports tools. A few vLLM-served checkpoints advertise chat but ignore tools silently; the gateway still returns 200.

If Bifrost logs show a full assistant message with tool_calls and LibreChat still does not execute, the bug is client-side parsing. If Bifrost's upstream response has empty tool_calls, fix the gateway/model path first.

We run an OSS Agent Gateway in front of OpenAI-compatible backends + MCP (Go, Apache-2.0): https://github.com/NeuralTrust/TrustGate — happy to compare notes on how LibreChat expects tool payloads shaped if useful.

[RockingRolli]

Same symptom here on v0.8.8-rc3 with LocalAI (llama-cpp backend, gemma-4-31b-it and other models), so this isn't specific to Bifrost/vLLM. I traced it to the root cause and have a one-line fix confirmed working.

Root cause

Backends that stream reasoning emit delta.reasoning chunks before the first chunk that carries role: "assistant". @langchain/openai (1.5.8, pinned by @librechat/agents 3.9.0) converts each delta in dist/converters/completions.js:

const role = delta.role ?? defaultRole; // undefined for the first chunks

With role === undefined it falls through to new ChatMessageChunk(...) instead of AIMessageChunk. The stream is folded left-to-right, and ChatMessageChunk.concat() in @langchain/core only merges content/additional_kwargs/response_metadata — it has no tool_call_chunks. So the tool call that arrives later (on a proper AIMessageChunk) is silently dropped from the aggregate. The final message has tool_calls: [], hasToolCalls in Graph.mjs is false, and the run ends right after the LLM call — which is why the logs show one Invoking LLM, ~4–6 completion tokens, and no tool execution. The UI still briefly shows the tool-call placeholder because the per-chunk stream events are AIMessageChunks.

Repro (LocalAI)

Same request, stream: true, with tools → first chunk:
data: {"choices":[{"index":0,"delta":{"content":null,"reasoning":"The"}}]} ← no role
Without tools → first chunk:
data: {"choices":[{"index":0,"delta":{"role":"assistant","content":null}}]} ← role present, works
Non-streaming with tools works, since the full response carries message.role. Setting disableStreaming: true on the modelSpec is therefore a config-only workaround.

Fix (confirmed)

Defaulting the role to assistant — which is what the Python LangChain client already does — fixes it, with streaming, web search and file search working again:

const role = delta.role ?? defaultRole ?? "assistant";

(patched in node_modules/@langchain/openai/dist/converters/completions.{js,cjs}). Arguably the backends should send role on the first chunk too, but every OpenAI-compatible server that separates reasoning seems to get this wrong, so defaulting in the converter is the robust fix.