feat(stack-encrypt,eql): store a JSON document through a data plan as JsonSearch

.changeset/eql-encrypted-json-target.md

@@ -0,0 +1,5 @@
+---
+"@cipherstash/eql": minor
+---
+
+**The `json_tree` family is in the EQL catalog as EQL v4's searchable JSON document, and a data plan can produce `public.eql_v4_json_search` by name: the target `JsonSearch`.** A language SDK names it in a plan field with the `json` type, gives it the JSON index's options (`{"name": "JsonSearch", "options": {"sealed_root": true, "ordered": ["$.age"]}}`, or the bare name for the defaults), and hands each document over as UTF-8 JSON text; the `eql-bindings` crate's by-name dispatch (`encryption::targets::encrypt_with`, `decrypt`) seals it through the same JSON index layout typed Rust uses and opens it back to its text. The text is parsed by Stack Encrypt's rules, before any key is requested: an object that names a member twice, an integer literal outside the 64-bit range, and text that is not JSON are refused (`TargetError::Json`, with a code and never a byte of the text). The catalog lists the family once (`eql_domains::DOCUMENT_FAMILIES`), so the target table, `v4::all()` / `v4::all_query()` and the Go types (`eql.JSONSearch`, `JSONSearchQuery`, `JSONSearchLeafQuery`, `JSONEntry`) are generated from it. The types take the names EQL v3 gives its searchable JSON document: `Json` (`public.eql_v4_json`) stays the storage-only type, `JsonSearch` is the searchable document, queried with `JsonSearchQuery` (`eql_v4.query_json`) and `JsonSearchLeafQuery` (`eql_v4.query_json_leaf`), and `JsonEntry` (`public.eql_v4_json_entry`) is one node as `->` and the reads return it. The JSON tree SQL domains are renamed to match: `public.eql_v4_json_tree` is now `public.eql_v4_json_search`, `public.eql_v4_json_tree_entry` is `public.eql_v4_json_entry`, `eql_v4.query_json_tree` is `eql_v4.query_json` and `eql_v4.query_json_tree_leaf` is `eql_v4.query_json_leaf`; the `eql_v4.json_tree_*` functions keep their names. The v4 bundle no longer carries the SteVec domains: in v4, `json_tree` implements `json_search`. The SteVec SQL (`src/v3/json/`) and the generated `json_entry_<T>` cross operators are confined to EQL v3 with `-- ONLY EQL VERSION: 3`, and the v4 target table drops its unproducible `SteVecDocument` row. The v3 bundle gains only those directive comment lines.

.changeset/eql-json-tree-domain.md

@@ -2,6 +2,6 @@
 "@cipherstash/eql": minor
 ---
 
-**EQL v4 gains the `json_tree` domain family: `public.eql_v4_json_tree`, an encrypted JSON document stored as one entry per node, with `@>` and `<@` that agree with PostgreSQL's own `jsonb @>` across array elements.** The family is new beside `public.eql_v4_json` and is not a port of the v3 SteVec surface: it is what Stack Encrypt's JSON index writes. A document's entries each carry a path selector `s` and a value selector `vs` (containment is a set comparison over the pair), the node's own ciphertext `c` (its label, kind and scalar value; a document's ciphertext is linear in its size, where a SteVec document repeats every subtree at every level), and a token chain (`p`, `w`, `o`) that lets core PostgreSQL walk the subtree under a node from one client-derived token: `eql_v4.json_tree_subtree(doc, token)`, `json_tree_subtree_at(doc, token, selector)`, `json_tree_children` and `json_tree_child_count`. By default the root is sealed, so no query can hand the server a token for the whole column. Containment projects a stored document into an element-nested shape from its group tags (`n`, `g`), `eql_v4.json_tree_query(doc)`, and compares it with the needle (`eql_v4.query_json_tree`) by native `jsonb @>`, so each needle array element must fit inside one stored element, which closes the cross-element false positives of selector-set containment (#1231) for this family; a GIN index on `(eql_v4.json_tree_query(col)::jsonb) jsonb_path_ops` serves it. `->` by selector returns an entry that opens on its own (`public.eql_v4_json_tree_entry`, with the header and ordinal grafted), and `eql_v4.query_json_tree_leaf` orders an extracted number or string leaf with `<`, `<=`, `>`, `>=`. Every other native jsonb operator on the column is blocked, as on the other EQL domains. EQL v4 only (`-- ONLY EQL VERSION: 4`); the v3 bundle is unchanged. The `eql-bindings` crate carries the hand-written `v4::json_tree` types with Stack Encrypt's `EncryptFrom` / `DecryptInto` for the document and the entry.
+**EQL v4 gains the `json_tree` domain family: `public.eql_v4_json_search`, an encrypted JSON document stored as one entry per node, with `@>` and `<@` that agree with PostgreSQL's own `jsonb @>` across array elements.** The family is new beside `public.eql_v4_json` and is not a port of the v3 SteVec surface: it is what Stack Encrypt's JSON index writes. A document's entries each carry a path selector `s` and a value selector `vs` (containment is a set comparison over the pair), the node's own ciphertext `c` (its label, kind and scalar value; a document's ciphertext is linear in its size, where a SteVec document repeats every subtree at every level), and a token chain (`p`, `w`, `o`) that lets core PostgreSQL walk the subtree under a node from one client-derived token: `eql_v4.json_tree_subtree(doc, token)`, `json_tree_subtree_at(doc, token, selector)`, `json_tree_children` and `json_tree_child_count`. By default the root is sealed, so no query can hand the server a token for the whole column. Containment projects a stored document into an element-nested shape from its group tags (`n`, `g`), `eql_v4.json_tree_query(doc)`, and compares it with the needle (`eql_v4.query_json`) by native `jsonb @>`, so each needle array element must fit inside one stored element, which closes the cross-element false positives of selector-set containment (#1231) for this family; a GIN index on `(eql_v4.json_tree_query(col)::jsonb) jsonb_path_ops` serves it. `->` by selector returns an entry that opens on its own (`public.eql_v4_json_entry`, with the header and ordinal grafted), and `eql_v4.query_json_leaf` orders an extracted number or string leaf with `<`, `<=`, `>`, `>=`. Every other native jsonb operator on the column is blocked, as on the other EQL domains. EQL v4 only (`-- ONLY EQL VERSION: 4`); the v3 bundle's SQL is unchanged. The `eql-bindings` crate carries the hand-written `v4::json_tree` types with Stack Encrypt's `EncryptFrom` / `DecryptInto` for the document and the entry.
 
 Why. Stack Encrypt had no JSON index: the operators (`Contains`, `ContainedBy`, `AtPath`) were declared and answered by nothing, and the v4 JSON wire format was unconstrained by anything shipped. This is Phase 1 and 2 of the `json_tree` index's phased plan (#1249), implemented against `docs/plans/2026-10-17-json-tree-index.md`, whose "Implementation notes" section records where the code deviates from the design.

.changeset/eql-json-tree-walk-and-projection.md

@@ -4,7 +4,7 @@
 
 **The `json_tree` family's read path no longer pays for the document's size per query.** Three changes to the EQL v4 `json_tree` SQL, each measured (cipherstash/benches, `report/JSON_TREE_COMPARISON.md` at 57abcf9) on a 10,000-row corpus of nested documents (about 220 nodes each) where the shipped surface was one to four orders of magnitude behind native `jsonb`:
 
-- **A stored document carries its own containment projection, `q`.** The client writes the document's `{s, vs}` pairs nested by array element beside `sv`, and `eql_v4.json_tree_query(doc)` now reads that field instead of rebuilding the projection from the group tags on every call. The GIN index recheck, which runs the projection once per candidate row, was the whole cost of `@>` on a nested document: 75 ms for ten rows, 16 s for a two-thousand-row result set, 44 s to build the index on 10,000 rows. With the projection stored the same queries measure in the low milliseconds and the index builds in seconds. `json_tree_query` returns `jsonb` rather than the needle domain, since the domain's CHECK would run per recheck; the document's own CHECK validates `q` when the row is written. It is a plpgsql function costed at 1000 rather than an inlinable SQL one: inlined to `value -> 'q'` the planner costs the filter as one operator, and a column of large documents is TOASTed out of the heap, so a sequential scan over it looked nearly free and was chosen over the GIN index while in fact decompressing every row (1.5 s for a one-row match on 10,000 rows; 0.5 to 0.7 ms through the index). The index expression is `(eql_v4.json_tree_query(col)) jsonb_path_ops`; the `::jsonb` cast earlier guidance wrote is harmless and no longer needed. **Wire change:** `q` is required in `public.eql_v4_json_tree`, and a document written by the earlier build of this unreleased family is refused by the domain CHECK.
+- **A stored document carries its own containment projection, `q`.** The client writes the document's `{s, vs}` pairs nested by array element beside `sv`, and `eql_v4.json_tree_query(doc)` now reads that field instead of rebuilding the projection from the group tags on every call. The GIN index recheck, which runs the projection once per candidate row, was the whole cost of `@>` on a nested document: 75 ms for ten rows, 16 s for a two-thousand-row result set, 44 s to build the index on 10,000 rows. With the projection stored the same queries measure in the low milliseconds and the index builds in seconds. `json_tree_query` returns `jsonb` rather than the needle domain, since the domain's CHECK would run per recheck; the document's own CHECK validates `q` when the row is written. It is a plpgsql function costed at 1000 rather than an inlinable SQL one: inlined to `value -> 'q'` the planner costs the filter as one operator, and a column of large documents is TOASTed out of the heap, so a sequential scan over it looked nearly free and was chosen over the GIN index while in fact decompressing every row (1.5 s for a one-row match on 10,000 rows; 0.5 to 0.7 ms through the index). The index expression is `(eql_v4.json_tree_query(col)) jsonb_path_ops`; the `::jsonb` cast earlier guidance wrote is harmless and no longer needed. **Wire change:** `q` is required in `public.eql_v4_json_search`, and a document written by the earlier build of this unreleased family is refused by the domain CHECK.
 - **The token-chain walk reads the document once.** `json_tree_subtree`, `json_tree_subtree_at`, `json_tree_children` and `json_tree_child_count` computed each parent tag inside a scan of `sv`, once per (node, ordinal, entry); a 244-entry subtree took ten seconds. The walk now builds one map of the document's entries keyed by parent tag and looks each computed tag up in it, so a level costs its fan-out in SHA-256 evaluations and the document is never scanned per node. The same subtree walks out in 23 ms; `children` and `child_count` in 1.6 ms. The walk is also bounded by the document's size, so a stored document whose unauthenticated chain fields claim a huge ordinal, or loop back to an ancestor, can no longer keep a query running until `statement_timeout`.
 - **A walk has a budget, and past it returns the sealed root.** `json_tree_subtree(doc, token, budget DEFAULT 64)` and `json_tree_subtree_at(doc, token, selector, budget DEFAULT 64)` stop once the subtree exceeds `budget` entries and return the root entry alone, with `d = 0`, when the document's root is sealed: the root's ciphertext is the whole document at about the plaintext's size, where the walk returns some four hundred characters per entry and pays the fan-out in hashes per node, so past a dozen or so entries the whole document is cheaper on both sides. The client opens the root and takes the walked path over it, and gets what the walked rows open to. The rows are opened through the key that keyed the walk, which carries the path it was derived from and knows from its gates which walk the rows came from: `key.open(rows)` in `stack-encrypt`, `JsonTreeSubtree::from_key(&key, rows)` in `eql-bindings`. Rows cannot be opened without it, so a document that grows past the budget does not break a caller. A budget of `0` never falls back; a token that names no node still yields no row; a document whose root is not sealed is always walked out.
 

.github/workflows/fuzz.yml

@@ -79,6 +79,7 @@ jobs:
           - { task: "fuzz:check-record", slug: check-record }
           - { task: "fuzz:plan-build", slug: plan-build }
           - { task: "fuzz:json-wire", slug: json-wire }
+          - { task: "fuzz:json-text", slug: json-text }
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
         with:
@@ -128,6 +129,7 @@ jobs:
           - { task: "fuzz:check-record", slug: check-record, dir: packages/stack-encrypt, target: check_record }
           - { task: "fuzz:plan-build",   slug: plan-build,   dir: packages/stack-encrypt, target: plan_build }
           - { task: "fuzz:json-wire",    slug: json-wire,    dir: packages/stack-encrypt, target: json_wire_decode }
+          - { task: "fuzz:json-text",    slug: json-text,    dir: packages/stack-encrypt, target: json_text }
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
         with:

AGENTS.md

@@ -691,6 +691,14 @@ monorepo, which is where the silent failures are.
   `jsonb @>` over the plaintexts, is `packages/eql/tests/encryption/tests/json_tree.rs`,
   run by `mise run test:encryption:postgres`. The design and its
   implementation notes are `docs/plans/2026-10-17-json-tree-index.md`.
+  In v4 it carries v3's names for the searchable JSON document: the target
+  `JsonSearch` stores `public.eql_v4_json_search`, queried with
+  `eql_v4.query_json` and `eql_v4.query_json_leaf`, with entries
+  `public.eql_v4_json_entry`; `json_tree` names the implementation only (the
+  SQL directory, the Rust structs, the SQL functions). The v3 SteVec SQL
+  that owns those names in v3 (`src/v3/json/` and the generated
+  `json_entry_<T>` cross files) is `-- ONLY EQL VERSION: 3`, so no bundle
+  carries both.
   A change to the derivations on one side (a PAE domain, the header layout,
   the projection shape) is a change to the other: the Postgres suite is
   what holds them together.

docs/fuzzing.md

@@ -28,6 +28,7 @@ Current targets:
 | `fuzz:term-decode`   | `stack-encrypt` | `term_decode`       | the SEM term decoders (`EqualityTerm`, `MatchTerms`, `OreTerm`, `OpeTerm` `from_bytes`) |
 | `fuzz:check-record`  | `stack-encrypt` | `check_record`      | `dynamic::record::check_record` over an `Arbitrary`-derived plan and tree, against a model of the record rules (structure-aware) |
 | `fuzz:plan-build`    | `stack-encrypt` | `plan_build`        | `plan::FieldsBuilder::build` over `Arbitrary`-derived untrusted names (context, fields, identities); a built field's label must render and parse back, and passthrough names are never refused (structure-aware) |
+| `fuzz:json-text`     | `stack-encrypt` | `json_text`         | `json::parse_text`, the boundary parser for a language SDK's JSON document, against serde_json: it reads what serde_json reads to the same value, refuses beyond it only repeated members and out-of-range integers, and what it reads writes back to itself |
 | `fuzz:json-wire`     | `stack-encrypt` | `json_wire_decode`  | the JSON index's wire decoders (`JsonDocument`, `JsonNode`, `JsonSelectorSet`, `JsonLeafQuery` from JSON text; `JsonHeader::from_bytes`), over base64 tags and base85 ciphertexts; an accepted document or node must round-trip through its wire form |
 
 Each target is a few lines — `libfuzzer-sys` hands a `&str` (or `&[u8]`