fix(http): stop advertising zstd for streamed responses

[akashkokare2910]

TLDR

Problem this solves:

• amazon_nova streaming fails 100% of the time, non-streaming works
• httpx cannot decode zstd delivered one complete frame per read
• litellm advertises zstd only because zstandard is transitively installed

How it solves it:

• Drops zstd from the default Accept-Encoding litellm sends
• Keeps every other codec the install can actually decode
• Adds LITELLM_ACCEPT_ENCODING as an escape hatch

Relevant issues

Fixes #35589

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Local gates all pass (details under Changes), and CI is green on the current head (df9359cb04: 76 successful, 1 skipped, 0 failed). The remaining unchecked box is Greptile: its 4/5 score was given on the first commit, before the Brotli fix it asked for and before the upstream merge, so it does not yet apply to this code. I've re-requested a review and will tick it once it re-scores.

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live end-to-end against the real amazon_nova provider (api.nova.amazon.com, model nova-micro-v1, real paid calls — no mocks).

BEFORE — at upstream 491eda319c, where the default Accept-Encoding is gzip, deflate, br, zstd. Failed 5/5 streaming attempts:

$ litellm.completion(model="amazon_nova/nova-micro-v1", stream=True, ...)

litellm.MidStreamFallbackError: litellm.APIConnectionError: Amazon_novaException -
cannot use a decompressobj multiple times
└─ httpx.DecodingError: cannot use a decompressobj multiple times
   └─ zstandard.backend_c.ZstdError: cannot use a decompressobj multiple times

non-streaming on the same account/model/prompt: OK

AFTER — validated by sending gzip, deflate, br, same account, model and prompt. On the Brotli-capable machine used for this capture, that string is exactly what the implementation now derives, so the live run reproduces the resulting default rather than a hand-picked value:

Accept-Encoding sent: 'gzip, deflate, br'
model=nova-micro-v1 tools=True max_tokens=200

  chunks received : 79
  content length  : 350
  finish_reason   : 'tool_calls'
  tool_calls      : [(0, 'tooluse_wgPXUvU0mWyTg7NRII8L25', 'get_current_weather',
                      '{"location":"Pune"}'),
                     (1, 'tooluse_foHLGu5EcztxSLgCh6txSL', 'get_exchange_rate',
                      '{"base_currency":"INR","quote_currency":"USD"}')]

  multiple chunks    : True
  non-empty payload  : True
  finish_reason set  : True
  >> WORKS PROPERLY  : True

Encoding matrix on the same live provider:

Accept-Encoding	Result
gzip, deflate, br, zstd (current default)	failed 5/5, zero usable chunks
gzip	13 chunks, correct content, finish_reason=stop
br	13 chunks, correct content, finish_reason=stop
gzip, deflate, br (this PR's derived default on a Brotli-capable install, + tools)	79 chunks, content and both parallel tool calls preserved

Proof commits: before = 491eda319c (upstream base), after = b49dd32d5f (first commit on this PR). The follow-up commit e8f58b1ed1 changed only how the header value is derived, not the value sent on a Brotli-capable install, and df9359cb04 is a merge of upstream that adapts this branch to lint rules added since it was opened. Neither changes the header. Re-verified at the current head: _supported_accept_encodings() → ('gzip', 'deflate', 'br').

This change is transport-level and endpoint-agnostic — it alters only the request header litellm sends, so it applies identically to /v1/chat/completions, /v1/responses and /v1/messages. The live proof above is on /v1/chat/completions because amazon_nova is the provider that surfaces the defect.

Type

🐛 Bug Fix

Changes

Root cause. Amazon Nova returns content-encoding: zstd for streaming SSE and emits the body as independent zstd frames — a 785-byte capture contained 5 frame magics (28b52ffd) at offsets [0, 210, 405, 584, 762]. httpx.ZStandardDecoder.decode() resets its decompressor only when unused_data is non-empty within the same call, so a read that consumes exactly one whole frame leaves eof=True with no leftover bytes and the next read reuses a finished decompressobj:

frame 0: decoded ok (eof=True, unused=0)
frame 1: FAILED -> ZstdError: cannot use a decompressobj multiple times

The same bytes fed in arbitrary 64-byte chunks decode fine, which is why this only appears over a real network and not in replay-based tests.

litellm advertises zstd only because zstandard is installed, and that is commonly transitive (langsmith requires zstandard>=0.23.0) — so this is the default state of an ordinary install, not an opt-in.

The durable fix is upstream and already exists as encode/httpx#3697 (open since 2025-10-27, tracked in encode/httpx#3538). I verified locally that its if self.decompressor.eof: ...reset logic fixes exactly this reproduction. It has not shipped, so this PR is interim mitigation.

The change. get_default_headers() in litellm/llms/custom_httpx/http_handler.py now derives Accept-Encoding from the decoders the installation actually has, and removes only zstd.

httpx already maintains that capability set: httpx._decoders.SUPPORTED_DECODERS pops br when neither brotli nor brotlicffi is importable, and pops zstd when zstandard is absent — it is the same registry httpx joins into its own Accept-Encoding in _client.py. Deriving from it keeps litellm in step with what the stack can decode, so br is advertised only when Brotli decoding is available. An earlier revision of this PR hardcoded gzip, deflate, br, which would have advertised Brotli on installs with no Brotli decoder; that is fixed in e8f58b1ed1.

That registry is a private httpx symbol, so this is not risk-free: if it is renamed or moved, the import fails. Construction is therefore wrapped, falling back to the codecs the standard library always provides (gzip, deflate) rather than raising, and there is a test that deletes the symbol to prove the fallback path works.

LITELLM_ACCEPT_ENCODING still overrides the entire value (mirroring the existing LITELLM_USER_AGENT pattern), per-request headers still win over the client default, and both the sync (HTTPHandler) and async (AsyncHTTPHandler) client factories consume the same derived default, so one change covers both.

It does not touch httpx internals, and does not affect Bedrock Nova (bedrock/amazon.nova-*), which dispatches through a different provider path.

Scope note. I raised shared-vs-provider-specific in #35589. This PR takes the shared default because the incompatibility is between litellm's shared httpx transport and valid multi-frame zstd streaming, not something specific to Amazon Nova — any provider streaming multi-frame zstd hits it. A provider-specific pin would additionally require _complete_amazon_nova to forward caller headers=/client= (it currently drops both), making it a larger diff that leaves the transport-level failure reachable elsewhere. Happy to switch to the narrower version if maintainers prefer it.

Tests (offline, no credentials, no network). Capability combinations:

• Brotli decoder available → br advertised
• Brotli decoder unavailable → br not advertised
• zstd decoder installed → zstd still excluded
• zstd decoder absent → unchanged result
• neither optional decoder → gzip, deflate
• httpx registry unreadable → falls back to gzip, deflate

Plus: LITELLM_ACCEPT_ENCODING override, per-request Accept-Encoding override, sync and async client request headers, the LITELLM_USER_AGENT branch still emitting Accept-Encoding, and a deterministic multi-frame decoder characterization test. That characterization test is self-retiring — once httpx decodes frame-aligned chunks correctly it skips instead of failing, so an httpx bump won't break CI, and the skip reason tells the reader the pin can then be reconsidered.

Local verification at df9359cb04:

• pytest tests/test_litellm/llms/custom_httpx/ → 235 passed
• make format-check → clean (2169 files)
• make lint → all gates pass (ruff, strict-rule budgets, type-discipline, basedpyright, circular imports, from litellm import * import safety)
• Full tests/test_litellm run compared against a clean-tree baseline run with identical flags: no failures attributable to this change (the suite has pre-existing flakiness under -n 4; every branch-only failure passed in isolation)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

All committers have signed the CLA.

[greptile-apps]

Greptile Summary

The PR replaces LiteLLM’s implicit httpx Accept-Encoding default with a capability-aware value that excludes zstd while preserving supported codecs and explicit overrides.

• Adds shared constants for excluded and fallback content encodings.
• Derives the default header from httpx’s installed decoder registry.
• Covers Brotli availability, zstd exclusion, fallback behavior, environment and per-request overrides, and sync/async clients.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the prior Brotli capability issue is addressed by deriving the header from the installed decoder registry, with fallback and sync/async coverage.

Important Files Changed

Filename	Overview
litellm/constants.py	Adds centralized zstd exclusion and standard-library fallback encoding constants.
litellm/llms/custom_httpx/http_handler.py	Builds a capability-aware Accept-Encoding header that resolves the previously reported Brotli-without-decoder issue and excludes zstd.
tests/test_litellm/llms/custom_httpx/test_http_handler.py	Adds focused offline coverage for decoder capabilities, overrides, fallback behavior, client propagation, and the motivating zstd decoder behavior.

_{Reviews (2): Last reviewed commit: "Merge upstream/litellm_internal_staging ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing akashkokare2910:fix/httpx-zstd-streaming-decode (df9359c) with litellm_internal_staging (4b7adab)}

[akashkokare2910]

@greptileai