Add compression stats and ratio threshold for ZSTD dict

[romange]

Summary

• Expose QList compression stats as Prometheus metrics (list_compression_attempts, list_compressed_bytes)
• Skip storing incompressible blobs with a 30% savings ratio threshold in CompressNodeWithDict
• Fix malloc_size_ tracking in CompressWithZstdDict by computing per-node deltas instead of a second loop

Follow-up to #6967.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR exposes QList compression statistics via Prometheus, adds a savings-ratio threshold to avoid storing poorly-compressible blobs with the ZSTD dictionary path, and adjusts malloc_size_ tracking during bulk ZSTD dict compression.

Changes:

• Add Prometheus metrics for QList compression attempts and compressed/raw byte counts.
• Add a “30% savings required” ratio gate to reject low-benefit ZSTD dict compression results.
• Switch CompressWithZstdDict from a full malloc_size_ recomputation loop to per-node delta updates.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
src/server/server_family.cc	Exposes QList compression stats as Prometheus metrics.
src/core/qlist.cc	Adds ratio-based rejection for ZSTD dict compression and changes how malloc_size_ is updated during bulk compression.
src/core/qlist_test.cc	Adds tests covering incompressible-data rejection and stats invariants.

[augmentcode]

🤖 Augment PR Summary

Summary: Improves QList compression observability and tightens ZSTD-dictionary compression behavior.

Changes:

• Expose QList compression attempt/failure counters and current compressed/raw byte gauges via Prometheus.
• Add a 30% savings ratio guard in CompressNodeWithDict to avoid storing near-incompressible blobs.
• Update bulk ZSTD-dict compression to adjust malloc_size_ using per-node deltas instead of a full recompute loop.
• Add unit tests covering incompressible inputs and stat tracking invariants.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.

[romange]

Fixed some bugs as well.

Results, based on https://github.com/romange/python_queue_benchmark I forked and improved to have also non-python frameworks.

→ Celery: 197 MB → 18 MB (10.7x reduction)
→ Laravel: 106 MB → 17 MB (6.1x reduction)
→ Sidekiq: 34 MB → 6.8 MB (5x reduction)

TBD: streams - they are being used by some of the frameworks like bullmq and others.