Compress message bodies before encrypt / before glossia-encode to shrink the encoded body

[asherp]

Summary

Glossia expands an encoded body by ~6.7× (measured, default dialect) — it's the dominant size cost. Compressing the plaintext before glossia inflates it shrinks whatever glossia then expands. There are two insertion points, same idea:

1. Compress-before-encrypt — for encrypted messages: the manifest body blob (§11.2) and the pairwise NIP-44 body. Compress → AES/NIP-encrypt → glossia.
2. Compress-before-encode — for signed plaintext: we glossia-encode the plaintext directly so the exact bytes are preserved for signature validation. Compressing first shrinks that body too.

Compression is the only lever that helps without sacrificing glossia's transport-survival / graceful-degradation properties (unlike moving content to MIME parts, cf. the externalized-keyring proposal in spec §11.2.1).

Measured impact

Glossia inflation is ~constant (~6.7× prose, ~7× high-entropy), so the glossia body shrinks by roughly the compression ratio.

sample	plaintext	glossia(raw)	DEFLATE	glossia(deflate)	body saving
short reply	203 B	1,363 ch	160 B	1,168 ch	−14%
~1 KB email	996 B	6,707 ch	563 B	4,000 ch	−40%
quoted thread	~4 KB	~26,800 ch*	~620 B	4,463 ch	−83%

*raw ~4 KB prose currently fails to encode (see "detection cliff" below).

It does not fully offset glossia: breaking even with the original plaintext size needs compression ≤ ~0.15 (an 85% reduction), which only redundant content like quoted threads reach (~1.1× there). For standalone prose the body stays ~4–6× the plaintext — but ~40% off a typical email is still a real win for an app where the glossia body is the payload.

Bonus: sidesteps the glossia detection cliff

Compression turns low-entropy prose into high-entropy bytes, which the dialect detector scores higher. Today the round-trip self-check (glossia_encode_bytes_with → detect_dialect_best, gated at hit_rate ≥ 0.3) caps reliably-encodable bodies at ~1 KB for signed plaintext and ~4 KB for encrypted (the underlying decay is filed upstream in glossia). Compressing both shrinks the body and raises entropy, clearing the cliff. (Independent of the upstream glossia fix — either helps.)

Design

• Algorithm: raw DEFLATE (no gzip header) to minimize overhead; zstd/brotli optional later. flate2 is already in the dependency tree.
• Length gate: only compress above a threshold (~512 B). Tiny bodies don't benefit and a header can inflate them (short sample only compressed 21%).
• Marker / versioning:
  • Encrypted: add a compression field to the capnp Manifest/EncryptedBlob (or a 1-byte algo prefix on the pre-encryption plaintext) and bump the manifest version. For the no-manifest NIP-44 body, a 1-byte algo prefix on the plaintext before encrypt.
  • Signed plaintext: frame the canonical signed bytes with a small magic + algo header so the decoder inflates before display; the signature is computed over the framed (compressed) canonical bytes, and the receiver inflates after verifying.
• Agreement H: the document hash (§11.3.1) must be computed consistently over the framed/compressed canonical bytes on both sides.

Security caveat

Compress-then-encrypt has the CRIME/BREACH length-leak property, but it's only exploitable with an adaptive chosen-plaintext oracle, which store-and-forward email does not hand an attacker; NIP-44's own length-padding blunts it further. Low risk here, but a conscious decision — document it.

Scope / touch points

• manifest.rs — body blob in build_capnp_manifest / parse_manifest
• agreement.rs / email.rs — pairwise NIP-44 body encode path
• glossia_encode_signed_body / glossia_encode_bytes_with — signed-plaintext path
• schema/nostr_mail.capnp — compression marker + version bump

Suggested first step

Prototype raw-DEFLATE on the manifest body blob and the signed-plaintext path, length-gated and version-flagged, then re-run the before/after measurement on real sent emails (these numbers are synthetic samples) to get distribution-level savings.

🤖 Filed via Claude Code while investigating encoded-body size.