[BOT ISSUE] OpenAI chat completions streaming drops `logprobs` and `refusal` from aggregated output

[braintrust-bot]

Summary

The aggregateChatCompletionChunks function in the OpenAI instrumentation plugin silently discards two fields from streaming chat completion chunks:

1. logprobs — per-token log probabilities on each choice are ignored; the aggregated output hardcodes logprobs: null.
2. refusal — the delta.refusal field (model safety refusal text) is never concatenated; the aggregated message omits it entirely.

Users who request logprobs: true or receive refusal responses in streaming mode will see incomplete span output compared to non-streaming calls.

What instrumentation is missing

In js/src/instrumentation/plugins/openai-plugin.ts, the aggregateChatCompletionChunks function (lines 389–466):

logprobs (line 461):
The iteration over chunks (lines 403–447) never reads chunk.choices[0].logprobs. The final output hardcodes:

logprobs: null,  // line 461

OpenAI streaming chunks include choices[0].logprobs with per-token content arrays containing token, logprob, bytes, and top_logprobs. All of this data is discarded.

refusal (not present at all):
The delta processing (lines 416–446) handles delta.role, delta.content, delta.finish_reason, and delta.tool_calls, but there is no handling of delta.refusal. A grep for "refusal" across the entire plugins directory returns zero matches. The aggregated message object (lines 456–459) only includes role, content, and tool_calls.

Braintrust docs status

not_found — The Braintrust docs at https://www.braintrust.dev/docs/instrument/wrap-providers state "Streaming responses are fully supported" but do not specifically mention logprobs or refusal handling.

Upstream reference

• OpenAI Chat Completions streaming chunk schema: each choices[].logprobs contains a content array of {token, logprob, bytes, top_logprobs} objects when logprobs: true is requested.
• OpenAI Chat Completions delta.refusal: string field present when the model refuses a request, documented alongside delta.content in the choice delta object.
• OpenAI API reference: https://platform.openai.com/docs/api-reference/chat/create
• Equivalent Python SDK gaps: [BOT ISSUE] OpenAI: chat completions streaming discards logprobs from chunks braintrust-sdk-python#180 (logprobs), [BOT ISSUE] OpenAI: chat completions streaming drops refusal delta text from span output braintrust-sdk-python#181 (refusal)

Local files inspected

• js/src/instrumentation/plugins/openai-plugin.ts — lines 389–466: aggregateChatCompletionChunks function
• js/src/instrumentation/plugins/openai-channels.ts — channel definitions
• js/src/vendor-sdk-types/openai-common.ts — type definitions for streaming chunks