Skip to content

Bound gzip decompression in ingest to prevent decompression-bomb DoS - #3

Merged
VasiliyRad merged 1 commit into
PioneerSquareLabs:mainfrom
VasiliyRad:bound-gzip-decompression
Jul 24, 2026
Merged

Bound gzip decompression in ingest to prevent decompression-bomb DoS#3
VasiliyRad merged 1 commit into
PioneerSquareLabs:mainfrom
VasiliyRad:bound-gzip-decompression

Conversation

@VasiliyRad

Copy link
Copy Markdown
Collaborator

Summary

  • /v1/ingest decompressed Content-Encoding: gzip bodies with a single gzip.decompress() call, which fully materializes the output before any size check runs. A holder of a valid ingest token could send a small, highly-compressible payload (~1000:1 ratio is trivial with gzip) that expands to gigabytes in memory, past MG_MAX_BODY_BYTES, before the existing post-decompression size check ever fires — a cheap way to OOM the server for every integration sharing that token.
  • Replaced it with _gunzip_bounded, which decompresses via zlib.decompressobj(...).decompress(data, max_length=...) and aborts (413) as soon as running output exceeds the configured limit, so peak memory stays close to the limit regardless of the true decompressed size.
  • Preserves existing behavior for all legitimate traffic: the decompressed-size cap was already MG_MAX_BODY_BYTES before this change (just enforced too late) — this PR does not introduce a new limit, only enforces the existing one earlier.

Test plan

  • Added server/tests/test_gunzip_bounded.py: happy-path round-trip, exact-boundary (limit vs. limit+1), a payload spanning multiple internal decompression rounds (regression guard for an unconsumed_tail handling bug caught during review of my own first draft), a real ~1000:1 decompression bomb rejected without full expansion, and malformed-gzip handling.
  • Verified red→green locally: new tests fail against the pre-fix code, pass after the fix.
  • Full existing suite (26/26, including the pre-existing test_gzip_ingest end-to-end round-trip) passes locally against Postgres 14 — no regressions to legitimate gzip ingest traffic.

🤖 Generated with Claude Code

/v1/ingest decompressed Content-Encoding: gzip bodies with a single
gzip.decompress() call, which fully materializes the output before any
size check runs. A holder of a valid ingest token could send a small,
highly-compressible payload (~1000:1 ratio is trivial with gzip) that
expands to gigabytes in memory before the existing post-decompression
size check ever fires.

Replace it with _gunzip_bounded, which decompresses via
zlib.decompressobj(...).decompress(data, max_length=...) and aborts
(413) as soon as running output exceeds the configured limit, so peak
memory stays close to the limit regardless of the true decompressed
size.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@VasiliyRad

Copy link
Copy Markdown
Collaborator Author

@calebjohn24, @jalexray, can you take a look? Can you add me as a contributor to the repo?

@VasiliyRad
VasiliyRad merged commit 44d74b7 into PioneerSquareLabs:main Jul 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant