API & Shared: Research binary json for query endpoint

[ivanslabbert]

Look at MessagePack or other binary json alternatives to improve response sizes on the query endpoint

[MrDirkelz]

Research: binary JSON (MessagePack / CBOR) for POST /query

Bottom line: it won't reduce response sizes here, and would likely be a slight regression. /query responses are already compressed — @fastify/compress with ["br", "gzip", "deflate"] is registered in api/src/main.ts, and the @HttpCode(200) on the controller exists specifically so a downstream reverse proxy can gzip too. Binary formats save space over uncompressed JSON; over the compression already on this endpoint, that saving collapses.

The measurement

Benchmarked a realistic 100-doc sync batch built from the repo's actual content seeding docs, with synthetic fts trigram arrays added (real sync payloads carry these; the seeding docs don't):

Format	Raw	+ gzip	+ brotli
JSON	584 KB	220 KB	183 KB
MessagePack	456 KB (−22%)	213 KB (−3.3%)	187 KB (+2.2%)
CBOR	456 KB (−22%)	212 KB (−3.7%)	186 KB (+2.0%)

The ~22% raw shrink is the number proposals cite, and it's real — but it comes almost entirely from eliminating repeated JSON keys (_id, type, parentId, …) across many docs, which is exactly what a compressor's dictionary already removes for near-free. So:

• Under gzip: binary is ~3% smaller — marginal.
• Under brotli: binary is ~2% larger — a regression. And Luminary lists br first, so modern browsers negotiate Brotli in production.

Whichever compressor production actually uses (Fastify or the proxy), binary lands within ±3% and loses under brotli. The synthetic fts used random trigrams, which is a worst case for compression — real trigram distributions are skewed and compress better, shrinking the JSON+brotli baseline further and leaving even less room for a binary format to win.

Why it's the wrong tool for the stated goal

• Size (the actual ask) sees ≈0 to negative benefit, per above.
• Parse speed is a different axis, and not a clear win either: libraries like cbor-x/msgpackr are engineered to rival native JSON.parse, but don't reliably beat it — so speed doesn't rescue the size case.
• Costs are real: a decoder dependency added to the shared bundle (ironic on a PWA where bundle size matters), reworking http.ts's res.json() path into arrayBuffer() + decode with Accept/Content-Type negotiation on both ends, an extra server-side re-encode step (CouchDB hands the API JSON already), and losing human-readable / curl-able responses for debugging.

Levers that would actually reduce response size

1. Field projection (Mango fields) — the genuine lever. text (tiptap HTML) and fts dominate the payload. Caveat: the bulk sync path (shared/src/api/sync/syncBatch.ts) legitimately needs both fts (offline search) and text, so there's little room there. The room is on the one-shot HybridQuery / queryRemote remote reads — worth auditing whether any over-fetch text/fts they never use.
2. Check the Brotli quality setting — @fastify/compress defaults Brotli to a conservative quality; bumping BROTLI_PARAM_QUALITY is a far cheaper, lower-risk size win than changing serialization. If the reverse proxy is the real compressor, the same check applies to its config. (Worth verifying which one actually compresses in prod, and its quality level — not confirmed here.)

Recommendation

Don't adopt MessagePack/CBOR for /query. Keep JSON + Brotli. If response size is a live pain point, chase field projection on the one-shot query reads and confirm the Brotli quality setting first.

[MrDirkelz]

Research: binary JSON (MessagePack / CBOR) for POST /query

Bottom line: it won't reduce response sizes here, and would likely be a slight regression. /query responses are already compressed — @fastify/compress with ["br", "gzip", "deflate"] is registered in api/src/main.ts, and the @HttpCode(200) on the controller exists specifically so a downstream reverse proxy can gzip too. Binary formats save space over uncompressed JSON; over the compression already on this endpoint, that saving collapses.

The measurement

Benchmarked a realistic 100-doc sync batch built from the repo's actual content seeding docs, with synthetic fts trigram arrays added (real sync payloads carry these; the seeding docs don't):

Format Raw + gzip + brotli
JSON 584 KB 220 KB 183 KB
MessagePack 456 KB (−22%) 213 KB (−3.3%) 187 KB (+2.2%)
CBOR 456 KB (−22%) 212 KB (−3.7%) 186 KB (+2.0%)
The ~22% raw shrink is the number proposals cite, and it's real — but it comes almost entirely from eliminating repeated JSON keys (_id, type, parentId, …) across many docs, which is exactly what a compressor's dictionary already removes for near-free. So:

• Under gzip: binary is ~3% smaller — marginal.
• Under brotli: binary is ~2% larger — a regression. And Luminary lists br first, so modern browsers negotiate Brotli in production.

Whichever compressor production actually uses (Fastify or the proxy), binary lands within ±3% and loses under brotli. The synthetic fts used random trigrams, which is a worst case for compression — real trigram distributions are skewed and compress better, shrinking the JSON+brotli baseline further and leaving even less room for a binary format to win.

Why it's the wrong tool for the stated goal

• Size (the actual ask) sees ≈0 to negative benefit, per above.
• Parse speed is a different axis, and not a clear win either: libraries like cbor-x/msgpackr are engineered to rival native JSON.parse, but don't reliably beat it — so speed doesn't rescue the size case.
• Costs are real: a decoder dependency added to the shared bundle (ironic on a PWA where bundle size matters), reworking http.ts's res.json() path into arrayBuffer() + decode with Accept/Content-Type negotiation on both ends, an extra server-side re-encode step (CouchDB hands the API JSON already), and losing human-readable / curl-able responses for debugging.

Levers that would actually reduce response size

1. Field projection (Mango fields) — the genuine lever. text (tiptap HTML) and fts dominate the payload. Caveat: the bulk sync path (shared/src/api/sync/syncBatch.ts) legitimately needs both fts (offline search) and text, so there's little room there. The room is on the one-shot HybridQuery / queryRemote remote reads — worth auditing whether any over-fetch text/fts they never use.
2. Check the Brotli quality setting — @fastify/compress defaults Brotli to a conservative quality; bumping BROTLI_PARAM_QUALITY is a far cheaper, lower-risk size win than changing serialization. If the reverse proxy is the real compressor, the same check applies to its config. (Worth verifying which one actually compresses in prod, and its quality level — not confirmed here.)

Recommendation

Don't adopt MessagePack/CBOR for /query. Keep JSON + Brotli. If response size is a live pain point, chase field projection on the one-shot query reads and confirm the Brotli quality setting first.

I agree with this claude assessment. I have done the field projection with the token caching ( #1692 )as well for the accessMap and it is actually a nice "hack" to keep performance up.