diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/README.md b/entries/2026-08-17-what-constrains-a-natural-language-layer/README.md
new file mode 100644
index 0000000..4c7901d
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/README.md
@@ -0,0 +1,231 @@
+# What constrains a natural-language layer over the search grammar
+
+- **Date:** 2026-08-17
+- **Author:** Steven Kearnes
+- **Status:** final (the design that follows from it is [beside this entry](assets/nl-search-design.md))
+- **Tags:** ord-schema, agents, search, natural-language, structured-outputs, anthropic, cost
+- **License:** [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)
+
+## Question
+
+`ord_schema.search` gives a model a validated `Query` grammar to write against instead of
+SQL. The layer that turns a chemist's sentence into one of those queries has never
+shipped: it exists on the unmerged `nl-query-backend` branch, written against a flat IR
+that predates the grammar, and the `/ask` deployment it served has since been moved out
+from under it.
+
+Rebuilding it raises one question that decides the whole design. Can the model be
+*constrained* to emit a valid `Query` — structured outputs or a strict tool, where the
+decoder itself refuses anything off-grammar — or does the layer have to accept whatever
+comes back and check it afterwards? And with that settled: how good is a cheap model, and
+what does a query cost?
+
+## Summary
+
+**Constrained decoding cannot carry this grammar.** Four separate walls, each hit
+directly against the API rather than inferred: circular references are rejected outright,
+the compiled grammar has a size budget, `oneOf` is unsupported, and there is a cap on
+union-typed parameters. Stratifying the grammar into acyclic levels, requiring every
+property, stripping descriptions and numeric constraints, and flattening the predicate
+union each moved a wall — and the best that fits is a **depth-2** predicate tree, which
+cannot express the nested correlation the pivots exist to serve.
+
+So translation is unconstrained generation, validated afterwards. That is fine: over ten
+questions, **Opus 5 compiled 9/10 first try**, and **Haiku 4.5 reached 7/10 with one
+repair turn** at **5.6× lower cost**. Both models get the shape wrong in the same
+harmless way — they return the predicate tree as a JSON *string* — which a coercing parse
+fixes deterministically.
+
+The probing also turned up a **gap in the grammar itself**: an aggregate or ordering key
+cannot reach a value under a repeated level, so *"the ten highest-yielding reactions"* is
+unwritable. Both models tried it, and repair does not help, because nothing they could
+have written would have worked.
+
+## Method
+
+Every claim here is a live call against the Messages API, using the key the interface
+deployment already holds (`ord-interface-anthropic-api-key`). The probes are in
+[`assets/`](assets/README.md); the schema transforms they use — stratification and the
+all-required rewrite — are the interesting half and are worth reading before repeating
+this work.
+
+Ten questions were used throughout, from single-predicate (`reactions run above 350 K`)
+to correlated (`pyridine as the solvent with a yield above 50%`) to aggregate (`the ten
+highest-yielding Suzuki couplings`). A translation is scored on whether it validates
+against the pydantic models and then compiles, which is a stricter bar than "looks right"
+and a weaker one than "returns the right reactions" — result-level scoring wants an eval
+set, which the design proposes and this entry does not have.
+
+## Findings
+
+### 1. The recursive grammar is refused by every constrained path
+
+`Query`'s predicate tree is recursive: `and`/`or` hold clauses, `not` holds a clause, and
+a quantifier's body may hold another quantifier. Pydantic renders that as 12 definitions
+with 134 internal references, 2,431 tokens.
+
+| path | result |
+| --- | --- |
+| `output_config.format` (structured outputs) | `Circular reference detected in schema definitions: And -> And` |
+| tool with `strict: true` | same refusal |
+| tool without `strict` | **accepted** |
+
+The SDK is not the obstacle: it sends the recursion faithfully, and its transform has
+deliberate handling for `$defs` and root-level `$ref`. The refusal is server-side, and
+the [structured outputs documentation](https://platform.claude.com/docs/en/build-with-claude/structured-outputs)
+confirms recursive schemas are unsupported.
+
+### 2. Removing the recursion does not help, because the wall behind it is smaller
+
+The grammar can be made acyclic without changing the IR: stratify it, so a level-*k*
+predicate's clauses are level-*(k-1)* predicates and level 0 holds only leaves. Nothing is
+inlined, so size grows by about **1,400 tokens per level** rather than by a power of four
+— naive inlining reaches 331,086 tokens at depth 5, stratification 9,274.
+
+That clears the cycle and immediately hits the next wall:
+
+| depth | schema | `output_config.format` | strict tool |
+| --- | --- | --- | --- |
+| 0 | 2,425 tok | accepted | too large |
+| 1 | 3,944 tok | too large | too large |
+| 2 | 5,415 tok | too large | too large |
+| 4 | 8,359 tok | too large | too large |
+
+Depth 0 is a single leaf predicate: no boolean, no quantifier. The strict-tool path is
+tighter still and fits nothing.
+
+### 3. The budget is union-typed parameters, not tokens
+
+A [reported limit](https://github.com/anthropics/anthropic-sdk-python/issues/1185) says
+optional properties roughly double the compiled state machine. Requiring every property
+and expressing optionality as an explicit null takes the grammar from 13–17 optional
+properties to zero and shrinks it by a third — and then two more walls appear in turn:
+`oneOf` is unsupported (pydantic emits it for the discriminated union; rewriting to
+`anyOf` is safe, since the models validate afterwards), and numeric constraints like
+`exclusiveMinimum` are rejected.
+
+Past those, depth 0 fits at 1,000 tokens and depth 1 does not, at 1,653. The error
+changes to *"too many parameters with union type"*, which names the real budget: every
+nullable field is a union, and each level multiplies them.
+
+### 4. What does fit is a flat predicate at depth 2 — with every path enumerated
+
+Collapsing the eight predicate variants into one object whose `op` selects the shape, and
+making `path` an enum, buys the most room. The enum turns out to be nearly free:
+
+| paths enumerated | depth 2 | depth 3 | depth 4 |
+| --- | --- | --- | --- |
+| free-form string | accepted | refused | refused |
+| 40 | accepted | refused | refused |
+| 120 | accepted | refused | refused |
+| **all 431 scalar paths** | **accepted** (15,240 tok) | refused | refused |
+
+The enumeration is the 431 scalar paths `probe_flat_union.py` walks to, which skips the
+key side of a map; the schema rendering counts 442 leaves and 537 lines because it carries
+those too.
+
+This is worth remembering rather than building. Enumerating the paths makes an invented
+column *structurally impossible*, and invented columns are the cheap model's main failure
+mode. But depth 2 cannot nest a quantifier inside a quantifier, so correlated questions —
+the ones the pivoted element index exists to answer — fall outside it.
+
+### 5. Unconstrained, both models write the tree as a string
+
+Given a non-strict forced tool call, the model reliably returns:
+
+```json
+{"where": "{\"op\": \"and\", \"clauses\": [ ... ]}"}
+```
+
+The predicate tree arrives JSON-encoded inside a string, and pydantic rejects it. Opus did
+this on 4 of 5 questions, Haiku on 5 of 5. Parsing string values back to objects before
+validating takes both to 5/5 — this is the one thing a strict schema would have bought,
+and it costs about ten lines to do without one.
+
+### 6. Opus 9/10, Haiku 7/10 with repair, at 5.6× the cost
+
+Ten questions, scored on compiling; a failure is handed back once with the compiler's own
+error, which names the offending path and suggests a real one.
+
+| model | first try | after one repair | ~cost per query |
+| --- | --- | --- | --- |
+| Claude Opus 5 | 9/10 | 9/10 | 1.3¢ |
+| Claude Haiku 4.5 | 5/10 | 7/10 | **0.23¢** |
+
+Haiku's failures are inventions of syntax the grammar does not have — `identifiers[*].value`,
+`inputs[].components` — rather than misunderstandings of the question. Repair recovers two
+of them. That is the failure mode finding 4 would eliminate outright, which is why the
+path enum is worth keeping in mind for shallow queries.
+
+Cost is dominated by the cached prefix: the schema rendering plus the grammar is **15,481
+tokens on Opus, 11,462 on Haiku**, read at a tenth of list price on every call after the
+first. Caching mattered more than model choice.
+
+### 7. The grammar cannot order by a value under a repeated level
+
+Opus's single failure, unrepairable and shared with Haiku:
+
+```text
+outcomes.products.measurements.percentage.value: max needs a scalar column, not a repeated level
+```
+
+`Measure` and `Order` require a scalar path, and a yield lives under `outcomes` →
+`products` → `measurements`. "The ten highest-yielding reactions" needs a per-reaction
+reduction of a repeated path — `max` over each reaction's measurements — and there is no
+way to write one. Both models reached for it independently, which is the strongest signal
+available that the gap is real rather than a prompting artifact.
+
+## Conclusions / next steps
+
+Translation is unconstrained generation checked afterwards, and the design that follows
+from that is [beside this entry](assets/nl-search-design.md): one forced tool call over
+the recursive grammar with the prefix cached, a coercing parse, one repair turn carrying
+the compiler's error, then execution and a short second call that writes the prose.
+
+Three things to do in order:
+
+1. **Close the reduction gap** (finding 7). It is a grammar change, it blocks a question
+ anyone would ask, and the NL layer would otherwise paper over it with a wrong answer.
+2. **Build the layer**, with the model as configuration rather than a decision — the
+ measured 9/10 and 7/10 are ten questions, not an eval set.
+3. **Then choose the model on evidence**, including whether path guidance in the cached
+ prefix closes Haiku's gap. At 5.6× it is worth the measurement.
+
+## What implementing it changed
+
+Three things the design got wrong, found by building it (ord-schema
+[#979](https://github.com/open-reaction-database/ord-schema/pull/979),
+[#980](https://github.com/open-reaction-database/ord-schema/pull/980),
+[#981](https://github.com/open-reaction-database/ord-schema/pull/981)):
+
+**A resolved compound was compared in the wrong spelling.** The default resolver asks
+PubChem, which returns pyridine as `C1=CC=NC=C1`; the projection stores what RDKit
+canonicalizes, `c1ccncc1`. Compared as strings they never match, so the first real
+question -- pyridine as a solvent -- answered **zero** over a corpus holding 24,930.
+Wrong, and wrong in the way that looks like an answer. This predates the NL layer: any
+caller passing `{"compound": ...}` to an `eq` on a smiles path hit it.
+
+**Forcing the tool call left no way to decline.** With `tool_choice` naming
+`build_query`, a model asked for something the grammar cannot express -- comparing two
+columns -- builds a plausible query that means something else. Both models did. A
+`cannot_answer` tool beside it, with `tool_choice: any`, turns that into a refusal
+carrying the model's own reason.
+
+**An eval case built by sampling failed correct translations.** The counterexamples for
+"a desired product with a yield above 50%" were derived by differencing a 200-row sample
+of the right query against a 400-row sample of the wrong one -- set arithmetic on
+samples. Both models failed a case they had answered correctly, and a round of prompt
+tuning went into a problem that did not exist. Asking for the counterexamples directly
+-- `near_miss AND NOT reference` -- is exact, and with honest cases both models pass 5/5.
+
+The second and third are the same lesson from opposite directions: a layer that cannot
+say "no" invents an answer, and a test that cannot say "wrong" invents a failure.
+
+## References
+
+- [`assets/nl-search-design.md`](assets/nl-search-design.md) — the design this entry argues for
+- [`assets/nl-search-plan.md`](assets/nl-search-plan.md) — the task-by-task plan built from it
+- [Structured outputs](https://platform.claude.com/docs/en/build-with-claude/structured-outputs) — recursion unsupported; limits undocumented
+- [anthropic-sdk-python#1185](https://github.com/anthropics/anthropic-sdk-python/issues/1185) — the optional-property state-space report
+- [ord-schema#966](https://github.com/open-reaction-database/ord-schema/pull/966) — the split that renamed `agent` to `search`
+- [Where the agent search cache can live](../2026-08-15-where-the-search-cache-lives/README.md) — the pivots and index this layer queries through
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/README.md b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/README.md
new file mode 100644
index 0000000..6bcdc9a
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/README.md
@@ -0,0 +1,50 @@
+# Scripts for "What constrains a natural-language layer over the search grammar"
+
+- **Date:** 2026-08-17
+- **Author:** Steven Kearnes
+- **License:** [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)
+
+Probes for [the entry beside them](../README.md), the design they argue for
+([`nl-search-design.md`](nl-search-design.md)), and the plan that implements it
+([`nl-search-plan.md`](nl-search-plan.md)).
+
+Every probe but the first makes live Messages API calls. Run them with `ord_schema`
+importable and a key in the environment; the interface deployment's key works, and is
+where these numbers came from:
+
+```bash
+export ANTHROPIC_API_KEY=$(aws secretsmanager get-secret-value \
+ --secret-id ord-interface-anthropic-api-key --query SecretString --output text)
+uv run --with anthropic python probe_repair.py claude-haiku-4-5
+```
+
+| script | what it measures | finding |
+| --- | --- | --- |
+| `probe_recursion.py` | what the SDK puts on the wire for a recursive model, via a mock transport — no key needed | 1 |
+| `probe_recursion_live.py` | whether `output_config.format` accepts the recursive grammar | 1 |
+| `probe_tool_recursion.py` | the same question for tool schemas, strict and not | 1, 5 |
+| `probe_shapes.py` | how often a forced tool call parses as written, and after coercion | 5, 6 |
+| `probe_stratified.py` | whether an acyclic grammar unlocks structured outputs | 2 |
+| `probe_bisect.py` | which construct exhausts the compiled-grammar budget | 2, 3 |
+| `probe_strict_tool.py` | whether the strict-tool path has a larger budget | 2 |
+| `probe_required.py` | whether requiring every property fits the budget | 3 |
+| `probe_flat_union.py` | a flattened predicate with paths enumerated, by depth | 4 |
+| `probe_repair.py` | first-try and after-repair accuracy, per model, with cost | 6, 7 |
+
+Two of these are library rather than probe, and are the reusable part:
+
+- `stratify.py` turns a recursive JSON Schema into an acyclic one by numbering levels —
+ a level-*k* definition references level *(k-1)*, and level 0 drops the recursive
+ branches. Nothing is inlined, so size grows per level rather than per power.
+- `require_all.py` makes every property required with an explicit null instead of
+ optional, rewrites `oneOf` to `anyOf`, and drops keywords a decoder rejects. Optional
+ properties are what multiply the compiled state machine.
+
+Both transform only the schema *shown to the model*. The pydantic models are untouched and
+still do the validating, which is why dropping `exclusiveMinimum` and the discriminator
+costs no correctness.
+
+`probe_repair.py` takes a model id as its argument and is the one to re-run when the
+prompt changes; it prints per-question outcomes, first-try and after-repair tallies, and
+the token counts the cost figures come from. Costs in the entry are computed from those
+counts at list price, with cached reads at a tenth.
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/nl-search-design.md b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/nl-search-design.md
new file mode 100644
index 0000000..4376a1e
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/nl-search-design.md
@@ -0,0 +1,182 @@
+# Natural-language search over the ORD corpus — design
+
+- **Date:** 2026-08-17
+- **Author:** Steven Kearnes
+- **Status:** draft, for review
+- **License:** [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)
+
+The measurements this argues from are in [the entry beside it](../README.md). Read finding
+1–4 before proposing constrained decoding again; it was tried four ways and does not fit.
+
+## What this builds
+
+A question in English becomes a validated `Query`, runs against a `Corpus`, and comes back
+with the reactions and a sentence saying what they show. One entry point:
+
+```python
+from ord_schema.search import execute, nl
+
+corpus = execute.Corpus("projections/*/*.parquet", "structures/*/*.parquet")
+answer = nl.ask("which reactions use pyridine as a solvent and beat 50% yield?", corpus)
+
+answer.query # the Query that ran, for display and for a "run it again" button
+answer.table # pyarrow.Table, exactly what corpus.search returned
+answer.text # "1,204 reactions. Most are Suzuki couplings; yields cluster at 60-75%."
+```
+
+`ask` is the whole public surface. `translate` and `answer` are separable because the eval
+harness needs the first without the second, and a caller that renders its own results
+needs the first without the third.
+
+## What it does not build
+
+- **Multi-turn.** No clarifying questions, no follow-ups, no session state. A question is
+ answered or it fails.
+- **A second backend.** Corpus only. ord-interface's Postgres path is out of scope, and
+ the broken `ord_schema.agent.nl_query` import there is a separate cleanup.
+- **Constrained decoding.** Measured dead; see the entry.
+- **Text search tuning.** Substring predicates remain expressible and remain unoptimized.
+
+## The gap this depends on
+
+`Measure` and `Order` require a scalar path, so an aggregate cannot reach a value under a
+repeated level:
+
+```text
+outcomes.products.measurements.percentage.value: max needs a scalar column, not a repeated level
+```
+
+"The ten highest-yielding reactions" is therefore unwritable, and both models tried to
+write it anyway. **Task 1 closes this before any NL code exists**, because a layer built
+over the gap would answer the question wrongly rather than refuse it.
+
+The shape: a reduction over a repeated path, usable wherever a scalar is wanted.
+
+```json
+{"order_by": [{"key": {"reduce": "max", "path": "outcomes.products.measurements.percentage.value"}, "descending": true}],
+ "limit": 10}
+```
+
+It compiles to a list aggregate over the reaction's own elements — `list_max(list_transform(...))`
+over the projection, or `max(...)` over the level's pivot with a correlated subquery — so
+it inherits the routing the executor already does per quantifier. `min`, `max`, `avg`,
+`sum`, and `count` all make sense; `count` needs no path.
+
+Filtering on a reduction (`reactions whose best yield beats 50%`) is already expressible as
+`exists ... where percentage.value > 50`, and stays that way. This is only about ordering
+and aggregating, where there is no quantifier to hang the condition on.
+
+## How translation works
+
+```mermaid
+flowchart TB
+ Q["question"] --> T["forced tool call
build_query, non-strict"]
+ T --> C["coerce: JSON strings to objects"]
+ C --> V["Query.model_validate"]
+ V --> K["compile_query"]
+ K -- "ValidationError or QueryError" --> R{"repaired once?"}
+ R -- no --> T2["hand back the error
+ its did-you-mean"]
+ T2 --> C
+ R -- yes --> F["MalformedQueryError"]
+ K -- ok --> X["corpus.search"]
+ X --> S["summarize: counts, columns, a few rows"]
+ S --> A["second call writes the prose"]
+```
+
+**The cached prefix** is the system text (instructions plus `schema.describe()`) and the
+tool definition, marked `cache_control: {"type": "ephemeral", "ttl": "1h"}`. Measured at
+15,481 tokens on Opus and 11,462 on Haiku, read at a tenth of list price on every call
+after the first. Two rules keep it cached: nothing volatile goes in the prefix (no
+timestamps, no per-request identifiers), and the tool list is built once at import.
+
+**The coercing parse** walks the tool input and replaces any string that parses as JSON
+with the parsed value, because both models return the predicate tree as a JSON string
+(finding 5). It runs before pydantic, and it is not a fallback — it is the normal path.
+
+**One repair turn.** On a validation or compile failure, the assistant's tool call and a
+`tool_result` carrying the error text go back with `is_error: true`. The compiler's errors
+already name the bad path and suggest a real one, which is what makes this worth doing;
+Haiku recovers two of five failures on it. A second failure raises rather than looping —
+an unbounded repair loop spends real money discovering the model cannot answer.
+
+**The answer call** sees a summary, never the table: row count, column names, and up to
+five sample rows rendered as text. A result set of 100,000 reactions costs the same
+prompt as one of three.
+
+## Module layout
+
+| file | holds |
+| --- | --- |
+| `ord_schema/search/nl.py` | `ask`, `translate`, `answer`, the client, the errors |
+| `ord_schema/search/nl_prompt.md` | the system prompt, as markdown rather than a Python string |
+| `ord_schema/search/nl_eval.py` | `EvalCase`, `load_cases`, scoring, the report |
+| `ord_schema/search/nl_cases.yaml` | the eval set, versioned beside the prompt it grades |
+
+`nl.py` sits in `search/` because it depends on the grammar, the compiler, and the
+executor, and nothing else depends on it. The prompt lives beside it for the reason the
+old branch found: a prompt in a markdown file can be edited and diffed as prose.
+
+Dependencies: a new `nl` extra carrying `anthropic`, since `ord-schema[search]` must stay
+installable without it. `ord_schema/dependencies_test.py` gains a profile row, which is
+what keeps that promise honest.
+
+## Errors
+
+The names come from the `nl-query-backend` branch, because ord-interface already maps them
+to status codes and there is no reason to invent new ones:
+
+| error | when |
+| --- | --- |
+| `ModelRateLimitedError` | 429 from the API |
+| `ModelUnavailableError` | 5xx, timeout, connection failure |
+| `MalformedQueryError` | still invalid after the repair turn |
+
+All three inherit `NLQueryError`. Everything else propagates: a `PairingError` from the
+corpus is not the model's fault and should not be dressed up as one.
+
+## Testing
+
+**Unit tests take no network.** A fake client returns canned tool calls, which is enough
+to pin every behavior that matters: the coercion, the repair turn firing exactly once, the
+error taxonomy, the prefix being marked cacheable, and the summary staying bounded as the
+table grows.
+
+**The eval harness is opt-in and costs money.** `nl_eval.py` runs cases against a real
+model and reports first-try and after-repair rates, per-question failures, and token cost.
+A case states a question and what the answer must satisfy — not a `Query` literal, since
+several spellings are right:
+
+```yaml
+- question: reactions using pyridine as the solvent
+ compiles: true
+ must_return: [ord-1f2a..., ord-9c7e...] # reactions any correct query returns
+ must_not_return: [ord-4b81...] # a near-miss the wrong quantifier would include
+```
+
+Scoring on returned reactions rather than on query shape is what makes the harness able to
+say a translation is *wrong* rather than merely *different*. The cases live beside the
+prompt so that changing one and not the other shows up in review.
+
+## What "fast" means here
+
+A question costs one translation call, one execution, and one answer call. The middle term
+is the only one this repository controls, and it is already measured: a structure query
+lands in 0.02–1.5 s warm, a quantifier over a pivot in tens of milliseconds. Translation is
+seconds and dominated by output tokens; the answer call is shorter.
+
+So the latency work is not in this layer, and the design should not pretend otherwise. What
+this layer owes the corpus is *not making it slow*: no query without a limit, the timeout
+passed through to `corpus.search`, and the model never handed enough rows to think it should
+summarize them itself.
+
+## Open questions
+
+1. **Which model.** Ten questions say Opus 9/10 and Haiku 7/10 at 5.6× less. The eval set
+ decides, and the interesting variable is whether worked path examples in the cached
+ prefix close the gap — they are nearly free once cached.
+2. **Whether the path enum earns a place.** Enumerating all 431 scalar paths fits a depth-2
+ grammar and makes invented paths impossible. If the evals show shallow queries dominate
+ real traffic, a constrained fast path with an unconstrained fallback becomes tempting —
+ but it is two systems, and it needs the traffic to justify it.
+3. **Where `/ask` lands.** Out of scope here, and the answer probably changes once the
+ layer exists and ord-interface's dependency on the deleted module is cleaned up.
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/nl-search-plan.md b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/nl-search-plan.md
new file mode 100644
index 0000000..c457af3
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/nl-search-plan.md
@@ -0,0 +1,1073 @@
+# Natural-language search implementation plan
+
+- **Date:** 2026-08-17
+- **Author:** Steven Kearnes
+- **Status:** draft
+- **License:** [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** A chemist's question becomes a validated `Query`, runs against a `Corpus`, and comes back with the reactions and a sentence about them.
+
+**Architecture:** One forced tool call over the recursive grammar with the ~15K-token prefix cached; the response is coerced (both models return the predicate tree as a JSON string), validated by the existing pydantic models, and compiled. A failure is handed back once with the compiler's own error, which names the bad path and suggests a real one. Execution goes through `Corpus.search`; a second short call sees a summary of the result — never the table — and writes the prose.
+
+**Tech Stack:** Python 3.11+, pydantic 2, `anthropic` SDK, DuckDB via `ord_schema.search.execute`, pytest.
+
+**Spec:** [`nl-search-design.md`](nl-search-design.md), and the measurements it argues from in [the entry](../README.md).
+
+## Global Constraints
+
+- **Model is configuration, never a hard-coded decision.** `DEFAULT_MODEL = "claude-haiku-4-5"`; every entry point takes `model`.
+- **The cached prefix must stay byte-stable.** No timestamps, no per-request identifiers, no dict iteration order in the system text or the tool definition. Cache reads are ~90% of the per-query cost.
+- **`ord-schema[search]` must remain installable without `anthropic`.** The new dependency goes in an `nl` extra, and `ord_schema/dependencies_test.py` gains a profile row that proves it.
+- **Repair runs at most once.** A second failure raises `MalformedQueryError`.
+- **No lazy imports.** Every import at module top, per the repository's Python style.
+- **Docstrings are Google style**, summary line on one physical line, `Args:`/`Returns:`/`Raises:` where non-empty.
+- **Ruff formats at line length 88**; `uv run ruff format` and `uv run ruff check` must pass before every commit.
+- **Tests run offline.** Only `nl_eval.py` calls the API, and nothing in the suite invokes it.
+
+---
+
+## File structure
+
+| file | responsibility |
+| --- | --- |
+| `ord_schema/search/query.py` (modify) | `Reduction`, and the reduction arms of `Order.key` and `Measure.path` |
+| `ord_schema/search/nl.py` (create) | errors, client, `translate`, `summarize`, `answer`, `ask` |
+| `ord_schema/search/nl_prompt.md` (create) | the system prompt, as prose |
+| `ord_schema/search/nl_test.py` (create) | every behavior above, against a stub client |
+| `ord_schema/search/nl_eval.py` (create) | `EvalCase`, `load_cases`, `run_case`, `report` |
+| `ord_schema/search/nl_cases.yaml` (create) | the eval set |
+| `pyproject.toml` (modify) | the `nl` extra |
+| `ord_schema/dependencies_test.py` (modify) | the `nl` profile |
+
+---
+
+### Task 1: A reduction over a repeated path
+
+The grammar cannot order by a value under a repeated level, so "the ten highest-yielding reactions" is unwritable. `resolve()` already returns a *list* expression for a repeated path, so a reduction is one DuckDB list aggregate around it.
+
+**Files:**
+
+- Modify: `ord_schema/search/query.py` (`Order`, `Measure`, `compile_query`)
+- Test: `ord_schema/search/query_test.py`
+
+**Interfaces:**
+
+- Consumes: `resolve(path, schema=...) -> _Resolved(expression, repeated, dtype)`, `QueryError`.
+- Produces: `Reduction` with fields `reduce: Literal["min","max","avg","sum","count"]` and `path: str`; `Order.key: str | Reduction`; `Measure.path: str | Reduction | None`.
+
+- [ ] **Step 1: Write the failing test**
+
+```python
+def test_ordering_by_a_reduction_over_a_repeated_path():
+ compiled = query.compile_query(
+ query.Query.model_validate(
+ {
+ "order_by": [
+ {
+ "key": {
+ "reduce": "max",
+ "path": "outcomes.products.measurements.percentage.value",
+ },
+ "descending": True,
+ }
+ ],
+ "limit": 10,
+ }
+ )
+ )
+ assert "list_max(" in compiled.sql
+ assert compiled.sql.endswith("DESC LIMIT 10")
+
+
+def test_a_reduction_over_a_scalar_path_is_refused():
+ # A scalar needs no reducing, and accepting one would give two spellings for the
+ # same query -- one of which silently wraps a value in a single-element list.
+ with pytest.raises(query.QueryError, match="already scalar"):
+ query.compile_query(
+ query.Query.model_validate(
+ {
+ "order_by": [
+ {
+ "key": {
+ "reduce": "max",
+ "path": "conditions.temperature.setpoint_kelvin",
+ }
+ }
+ ]
+ }
+ )
+ )
+```
+
+- [ ] **Step 2: Run the tests to verify they fail**
+
+Run: `uv run pytest ord_schema/search/query_test.py -k reduction -v`
+Expected: FAIL — `Order.key` rejects a dict, so pydantic raises a `ValidationError`.
+
+- [ ] **Step 3: Add the model and the compiler support**
+
+In `ord_schema/search/query.py`, above `class Order`:
+
+```python
+# DuckDB's list aggregates, which ignore nulls; count is the number of values that are
+# actually there, so it filters rather than taking len() of a list holding nulls.
+_REDUCERS = {
+ "min": "list_min({expression})",
+ "max": "list_max({expression})",
+ "avg": "list_avg({expression})",
+ "sum": "list_sum({expression})",
+ "count": "len(list_filter({expression}, value -> value IS NOT NULL))",
+}
+
+
+class Reduction(BaseModel):
+ """One value per reaction, reduced from a path that crosses a repeated level.
+
+ An ordering key and an aggregate's argument both have to be scalar, which leaves
+ "the highest-yielding reactions" unwritable: a yield lives under outcomes,
+ products, and measurements, so the path resolves to a list. This reduces that list
+ to the one number the reaction is judged by.
+
+ Attributes:
+ reduce: How to reduce the list to a value.
+ path: A dotted path that crosses at least one repeated level.
+ """
+
+ reduce: Literal["min", "max", "avg", "sum", "count"]
+ path: str
+```
+
+And the resolver, beside `_scalar`:
+
+```python
+def _reduced(reduction: Reduction, schema: pa.Schema) -> str:
+ """Returns the expression reducing a repeated path to one value per reaction.
+
+ Args:
+ reduction: What to reduce, and how.
+ schema: Schema the path is resolved against.
+
+ Returns:
+ A DuckDB expression yielding one scalar per reaction, NULL where the reaction
+ holds no elements at all.
+
+ Raises:
+ QueryError: If the path does not cross a repeated level, since a scalar path
+ needs no reduction and accepting one would give a query two spellings.
+ """
+ resolved = resolve(reduction.path, schema=schema)
+ if not resolved.repeated:
+ raise QueryError(
+ f"{reduction.path}: {reduction.reduce} reduces a repeated level, and this "
+ f"path is already scalar; order by the path itself"
+ )
+ return _REDUCERS[reduction.reduce].format(expression=resolved.expression)
+```
+
+Widen the two fields:
+
+```python
+class Order(BaseModel):
+ """How to sort the result."""
+
+ key: str | Reduction
+ descending: bool = False
+```
+
+```python
+ fn: Literal["count", "count_distinct", "sum", "avg", "min", "max"]
+ path: str | Reduction | None = None
+ name: str
+```
+
+- [ ] **Step 4: Teach `compile_query` both places**
+
+The measure argument, replacing `argument = _scalar(measure.path, schema, measure.fn)`:
+
+```python
+ elif isinstance(measure.path, Reduction):
+ argument = _reduced(measure.path, schema)
+ else:
+ argument = _scalar(measure.path, schema, measure.fn)
+```
+
+The ordering key, at the top of the `for order in query.order_by:` body:
+
+```python
+ if isinstance(order.key, Reduction):
+ key = _reduced(order.key, schema)
+ elif orderable is None:
+```
+
+An aggregated query still orders by a measure name or a `group_by` path; a reduction is a per-reaction value and there is no such row after grouping. Leave that arm as it is — a `Reduction` reaching it is caught by the `isinstance` above only for the ungrouped case, so add the guard:
+
+```python
+ if isinstance(order.key, Reduction) and orderable is not None:
+ raise QueryError(
+ "an aggregated query orders by a measure name or a group_by path; "
+ "reduce inside a measure instead"
+ )
+```
+
+- [ ] **Step 5: Run the tests**
+
+Run: `uv run pytest ord_schema/search/query_test.py -v`
+Expected: PASS, including the two new tests.
+
+- [ ] **Step 6: Check the reduction against real data**
+
+Run:
+
+```bash
+uv run python -c "
+from ord_schema.search import execute, query
+corpus = execute.Corpus('~/ord/projections/**/*.parquet', '~/ord/structures/**/*.parquet',
+ require_current=False, resolver={}.__getitem__)
+table = corpus.search(query.Query.model_validate({
+ 'order_by': [{'key': {'reduce': 'max', 'path': 'outcomes.products.measurements.percentage.value'},
+ 'descending': True}],
+ 'limit': 10}))
+print(table.num_rows)
+"
+```
+
+Expected: 10 rows. This is the question both models tried to write and could not.
+
+- [ ] **Step 7: Document it**
+
+Add a row to the worked-example table in `ord_schema/search/README.md`:
+
+```markdown
+| the ten highest-yielding reactions | `order_by` a `reduce` over `outcomes.products.measurements` | no quantifier: a list aggregate over the projection |
+```
+
+- [ ] **Step 8: Commit**
+
+```bash
+git add ord_schema/search/query.py ord_schema/search/query_test.py ord_schema/search/README.md
+git commit -m "Let a query order by a value under a repeated level"
+```
+
+---
+
+### Task 2: The package, its dependency, and its errors
+
+**Files:**
+
+- Create: `ord_schema/search/nl.py`, `ord_schema/search/nl_test.py`
+- Modify: `pyproject.toml`, `ord_schema/dependencies_test.py`
+
+**Interfaces:**
+
+- Produces: `NLQueryError`, `ModelUnavailableError`, `ModelRateLimitedError`, `MalformedQueryError`, `DEFAULT_MODEL = "claude-haiku-4-5"`, `get_client() -> anthropic.Anthropic`.
+
+- [ ] **Step 1: Write the failing test**
+
+```python
+def test_the_errors_share_one_base():
+ # ord-interface maps these onto status codes, so a caller can catch the base and
+ # still tell a rate limit from a query it could not build.
+ assert issubclass(nl.ModelRateLimitedError, nl.NLQueryError)
+ assert issubclass(nl.ModelUnavailableError, nl.NLQueryError)
+ assert issubclass(nl.MalformedQueryError, nl.NLQueryError)
+```
+
+- [ ] **Step 2: Run it**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -v`
+Expected: FAIL — `ModuleNotFoundError: ord_schema.search.nl`.
+
+- [ ] **Step 3: Write the module head**
+
+```python
+"""Natural-language questions, translated into the search grammar and answered.
+
+A model cannot be constrained to emit a valid Query: the grammar is recursive, and both
+constrained-decoding paths refuse it -- the measurements are in the ord-logbook entry
+"What constrains a natural-language layer over the search grammar". So translation is
+generation checked afterwards: the response is coerced, validated by the same models the
+compiler uses, and handed back once with the compiler's error when it does not compile.
+"""
+
+import dataclasses
+import json
+from importlib import resources
+from typing import Any
+
+import anthropic
+import pyarrow as pa
+
+from ord_schema.logging import get_logger
+from ord_schema.search import execute, query, schema
+
+logger = get_logger(__name__)
+
+DEFAULT_MODEL = "claude-haiku-4-5"
+MAX_TOKENS = 2048
+
+
+class NLQueryError(Exception):
+ """A question could not be answered."""
+
+
+class ModelUnavailableError(NLQueryError):
+ """The model could not be reached."""
+
+
+class ModelRateLimitedError(NLQueryError):
+ """The model refused the request for rate reasons."""
+
+
+class MalformedQueryError(NLQueryError):
+ """The model's query did not validate, and the repair attempt did not either."""
+
+
+def get_client() -> anthropic.Anthropic:
+ """Returns a client reading its credentials from the environment."""
+ return anthropic.Anthropic()
+```
+
+- [ ] **Step 4: Run it**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -v`
+Expected: PASS.
+
+- [ ] **Step 5: Declare the dependency**
+
+In `pyproject.toml`, after the `search` extra:
+
+```toml
+# Optional: ord_schema.search.nl turns a question into a Query by asking a model, so it
+# needs an API client where the rest of the search subpackage needs none.
+nl = [
+ "anthropic>=0.120",
+ "ord-schema[search]",
+]
+```
+
+In `ord_schema/dependencies_test.py`, add to both dictionaries:
+
+```python
+ "nl": ("ord_schema.search.nl",),
+```
+
+```python
+ "nl": ("search", "nl"),
+```
+
+- [ ] **Step 6: Run the dependency test**
+
+Run: `uv run pytest ord_schema/dependencies_test.py -v`
+Expected: PASS — five profiles, none skipped, since `anthropic` is installed in a development checkout.
+
+- [ ] **Step 7: Commit**
+
+```bash
+git add ord_schema/search/nl.py ord_schema/search/nl_test.py pyproject.toml ord_schema/dependencies_test.py
+git commit -m "Add the natural-language module, its extra, and its errors"
+```
+
+---
+
+### Task 3: Translation, with the coercing parse
+
+**Files:**
+
+- Create: `ord_schema/search/nl_prompt.md`
+- Modify: `ord_schema/search/nl.py`, `ord_schema/search/nl_test.py`
+
+**Interfaces:**
+
+- Consumes: `schema.describe()`, `query.Query`, `query.compile_query`, the errors from Task 2.
+- Produces: `translate(question, *, client=None, model=DEFAULT_MODEL, repair=True) -> query.Query`; `SYSTEM_PROMPT: str`; `TOOL: dict`.
+
+- [ ] **Step 1: Write the failing tests**
+
+```python
+class _StubClient:
+ """Returns canned tool calls, and records what it was asked."""
+
+ def __init__(self, *inputs):
+ self._inputs = list(inputs)
+ self.requests = []
+ self.messages = self
+
+ def create(self, **kwargs):
+ self.requests.append(kwargs)
+ payload = self._inputs.pop(0)
+ block = types.SimpleNamespace(
+ type="tool_use", id="toolu_stub", name="build_query", input=payload
+ )
+ usage = types.SimpleNamespace(
+ input_tokens=1, output_tokens=1,
+ cache_creation_input_tokens=0, cache_read_input_tokens=0,
+ )
+ return types.SimpleNamespace(content=[block], usage=usage, stop_reason="tool_use")
+
+
+_WHERE = {
+ "op": "exists",
+ "path": "inputs.components",
+ "where": {"op": "eq", "path": "reaction_role", "value": {"literal": "SOLVENT"}},
+}
+
+
+def test_a_tree_returned_as_a_json_string_is_still_understood():
+ # Both models do this, most of the time: the predicate arrives JSON-encoded inside
+ # a string rather than as an object.
+ client = _StubClient({"where": json.dumps(_WHERE)})
+ result = nl.translate("solvent reactions", client=client)
+ assert result.where.op == "exists"
+
+
+def test_the_prefix_is_marked_cacheable():
+ # Cache reads are most of what a query costs; an uncached prefix is a 10x bill.
+ client = _StubClient({"where": _WHERE})
+ nl.translate("solvent reactions", client=client)
+ system = client.requests[0]["system"]
+ assert system[0]["cache_control"] == {"type": "ephemeral", "ttl": "1h"}
+
+
+def test_the_schema_reaches_the_prompt():
+ client = _StubClient({"where": _WHERE})
+ nl.translate("solvent reactions", client=client)
+ assert "reaction_role" in client.requests[0]["system"][0]["text"]
+```
+
+- [ ] **Step 2: Run them**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -v`
+Expected: FAIL — `AttributeError: module 'ord_schema.search.nl' has no attribute 'translate'`.
+
+- [ ] **Step 3: Write the prompt**
+
+`ord_schema/search/nl_prompt.md`:
+
+```markdown
+You turn a chemist's question into one ORD search query by calling `build_query`.
+
+Rules that keep a query answerable:
+
+- Paths are dotted names from the schema below. There is no array syntax: write
+ `inputs.components`, never `inputs[].components` or `identifiers[*].value`.
+- Any path that crosses a repeated level must be bound by `exists` or `forall`, and the
+ paths inside that quantifier are relative to the bound element.
+- Two conditions on the *same* element go inside one quantifier. Two conditions on
+ *different* elements are two quantifiers.
+- Name compounds rather than spelling structures: `{"compound": "pyridine"}` resolves to
+ SMILES. Use `substructure` with a SMARTS only for a pattern the user describes.
+- To rank by a value under a repeated level, order by a reduction:
+ `{"reduce": "max", "path": "outcomes.products.measurements.percentage.value"}`.
+- Prefer the smallest query that answers the question, and set `limit` when the user
+ asks for a number of results.
+
+The corpus schema, as an indented type tree in DuckDB's types:
+```
+
+- [ ] **Step 4: Implement translation**
+
+```python
+SYSTEM_PROMPT = (
+ (resources.files("ord_schema.search") / "nl_prompt.md").read_text(encoding="utf-8")
+ + "\n\n"
+ + schema.describe()
+)
+
+# Built once: the tool definition is part of the cached prefix, so a dict rebuilt per
+# call with a different key order would silently cost a cache miss.
+TOOL: dict[str, Any] = {
+ "name": "build_query",
+ "description": "Build an ORD search query from the user's question.",
+ "input_schema": query.Query.model_json_schema(),
+}
+_SYSTEM = [
+ {
+ "type": "text",
+ "text": SYSTEM_PROMPT,
+ "cache_control": {"type": "ephemeral", "ttl": "1h"},
+ }
+]
+
+
+def _coerce(value: Any) -> Any:
+ """Returns the value with JSON-encoded strings parsed back into objects.
+
+ A model handed a recursive tool schema usually returns the nested predicate as a
+ JSON string rather than an object, which pydantic rejects. Parsing it back is the
+ normal path rather than a fallback.
+ """
+ if isinstance(value, str):
+ try:
+ return _coerce(json.loads(value))
+ except (json.JSONDecodeError, TypeError):
+ return value
+ if isinstance(value, dict):
+ return {key: _coerce(item) for key, item in value.items()}
+ if isinstance(value, list):
+ return [_coerce(item) for item in value]
+ return value
+
+
+def _ask_model(client, model, messages):
+ """Returns the tool_use block from one forced call, mapping API failures."""
+ try:
+ response = client.messages.create(
+ model=model,
+ max_tokens=MAX_TOKENS,
+ system=_SYSTEM,
+ messages=messages,
+ tools=[TOOL],
+ tool_choice={"type": "tool", "name": "build_query"},
+ )
+ except anthropic.RateLimitError as error:
+ raise ModelRateLimitedError(str(error)) from error
+ except (anthropic.APIConnectionError, anthropic.APIStatusError) as error:
+ raise ModelUnavailableError(str(error)) from error
+ for block in response.content:
+ if block.type == "tool_use":
+ return block
+ raise MalformedQueryError("the model returned no query")
+```
+
+- [ ] **Step 5: Add `translate` itself**
+
+```python
+def translate(
+ question: str,
+ *,
+ client: anthropic.Anthropic | None = None,
+ model: str = DEFAULT_MODEL,
+ repair: bool = True,
+) -> query.Query:
+ """Returns the query a question asks for.
+
+ Args:
+ question: The question, in English.
+ client: Anthropic client; one is built from the environment if omitted.
+ model: Which model translates. Cheap models need the repair turn more often.
+ repair: Hand a failure back once with the compiler's error. Off for measuring
+ first-try accuracy.
+
+ Returns:
+ A Query that compiles against the projection schema.
+
+ Raises:
+ MalformedQueryError: If the query does not validate or compile, after the
+ repair turn if one was allowed.
+ ModelRateLimitedError: If the model is rate limited.
+ ModelUnavailableError: If the model cannot be reached.
+ """
+ client = client if client is not None else get_client()
+ messages: list[dict[str, Any]] = [{"role": "user", "content": question}]
+ block = _ask_model(client, model, messages)
+ try:
+ return _validated(block.input)
+ except (ValueError, query.QueryError) as error:
+ if not repair:
+ raise MalformedQueryError(str(error)) from error
+ first = error
+ logger.info("repairing a query that did not compile: %s", first)
+ messages += [
+ {"role": "assistant", "content": [{"type": "tool_use", "id": block.id,
+ "name": block.name, "input": block.input}]},
+ {"role": "user", "content": [{"type": "tool_result", "tool_use_id": block.id,
+ "is_error": True,
+ "content": f"That query was rejected: {first}. "
+ "Call build_query again with it fixed."}]},
+ ]
+ retry = _ask_model(client, model, messages)
+ try:
+ return _validated(retry.input)
+ except (ValueError, query.QueryError) as error:
+ raise MalformedQueryError(str(error)) from error
+
+
+def _validated(raw: Any) -> query.Query:
+ """Returns the Query a tool call carries, proven to compile."""
+ parsed = query.Query.model_validate(_coerce(raw))
+ query.compile_query(parsed)
+ return parsed
+```
+
+- [ ] **Step 6: Run the tests**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -v`
+Expected: PASS.
+
+- [ ] **Step 7: Commit**
+
+```bash
+git add ord_schema/search/nl.py ord_schema/search/nl_prompt.md ord_schema/search/nl_test.py
+git commit -m "Translate a question into a query, coercing what the model returns"
+```
+
+---
+
+### Task 4: The repair turn, pinned
+
+**Files:**
+
+- Modify: `ord_schema/search/nl_test.py`
+
+**Interfaces:**
+
+- Consumes: `translate`, `_StubClient` from Task 3.
+
+- [ ] **Step 1: Write the failing tests**
+
+```python
+_BAD_PATH = {"op": "eq", "path": "identifiers[*].value", "value": {"literal": "x"}}
+
+
+def test_a_bad_path_is_handed_back_once_and_recovered():
+ client = _StubClient({"where": _BAD_PATH}, {"where": _WHERE})
+ result = nl.translate("aspirin reactions", client=client)
+ assert result.where.op == "exists"
+ assert len(client.requests) == 2
+
+
+def test_the_repair_carries_the_compiler_s_suggestion():
+ client = _StubClient({"where": _BAD_PATH}, {"where": _WHERE})
+ nl.translate("aspirin reactions", client=client)
+ sent = client.requests[1]["messages"][-1]["content"][0]["content"]
+ assert "did you mean" in sent
+
+
+def test_a_second_failure_raises_rather_than_looping():
+ client = _StubClient({"where": _BAD_PATH}, {"where": _BAD_PATH})
+ with pytest.raises(nl.MalformedQueryError, match="identifiers"):
+ nl.translate("aspirin reactions", client=client)
+ assert len(client.requests) == 2
+
+
+def test_repair_can_be_turned_off_for_measurement():
+ client = _StubClient({"where": _BAD_PATH})
+ with pytest.raises(nl.MalformedQueryError):
+ nl.translate("aspirin reactions", client=client, repair=False)
+ assert len(client.requests) == 1
+```
+
+- [ ] **Step 2: Run them**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -k repair -v`
+Expected: PASS if Task 3 was implemented as written; a failure here means the repair path is wrong, not that the test is.
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add ord_schema/search/nl_test.py
+git commit -m "Pin the repair turn to exactly one attempt"
+```
+
+---
+
+### Task 5: The summary and the prose answer
+
+**Files:**
+
+- Modify: `ord_schema/search/nl.py`, `ord_schema/search/nl_test.py`
+
+**Interfaces:**
+
+- Produces: `summarize(table, *, rows=5) -> str`; `answer(question, table, *, client=None, model=DEFAULT_MODEL) -> str`.
+
+- [ ] **Step 1: Write the failing tests**
+
+```python
+def test_the_summary_is_bounded_by_the_row_cap_not_the_table():
+ small = pa.table({"reaction_id": ["a", "b"]})
+ large = pa.table({"reaction_id": [str(i) for i in range(100_000)]})
+ assert len(nl.summarize(large)) < 2 * len(nl.summarize(small))
+
+
+def test_the_summary_states_the_row_count():
+ assert "100000 rows" in nl.summarize(pa.table({"reaction_id": [str(i) for i in range(100_000)]}))
+```
+
+- [ ] **Step 2: Run them**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -k summary -v`
+Expected: FAIL — `summarize` does not exist.
+
+- [ ] **Step 3: Implement both**
+
+```python
+def summarize(table: pa.Table, *, rows: int = 5) -> str:
+ """Returns a description of a result small enough to put in a prompt.
+
+ Args:
+ table: What the search returned.
+ rows: How many sample rows to show.
+
+ Returns:
+ The row count, the column names, and up to ``rows`` rows. A result of a hundred
+ thousand reactions costs the same prompt as one of three.
+ """
+ sample = table.slice(0, rows).to_pylist()
+ lines = [f"{table.num_rows} rows, columns: {', '.join(table.column_names)}"]
+ lines += [json.dumps(row, default=str) for row in sample]
+ if table.num_rows > rows:
+ lines.append(f"... {table.num_rows - rows} more rows not shown")
+ return "\n".join(lines)
+
+
+def answer(
+ question: str,
+ table: pa.Table,
+ *,
+ client: anthropic.Anthropic | None = None,
+ model: str = DEFAULT_MODEL,
+) -> str:
+ """Returns a sentence or two saying what the result shows.
+
+ Args:
+ question: The question the result answers.
+ table: What the search returned.
+ client: Anthropic client; one is built from the environment if omitted.
+ model: Which model writes the prose.
+
+ Returns:
+ Plain text, with no markdown and no invented chemistry: the model sees a summary
+ rather than the rows, so it can only describe what the summary states.
+
+ Raises:
+ ModelRateLimitedError: If the model is rate limited.
+ ModelUnavailableError: If the model cannot be reached.
+ """
+ client = client if client is not None else get_client()
+ try:
+ response = client.messages.create(
+ model=model,
+ max_tokens=512,
+ system=(
+ "You describe the result of a database query in one or two plain "
+ "sentences. State only what the summary shows. Do not invent "
+ "chemistry, and do not use markdown."
+ ),
+ messages=[
+ {
+ "role": "user",
+ "content": f"Question: {question}\n\nResult:\n{summarize(table)}",
+ }
+ ],
+ )
+ except anthropic.RateLimitError as error:
+ raise ModelRateLimitedError(str(error)) from error
+ except (anthropic.APIConnectionError, anthropic.APIStatusError) as error:
+ raise ModelUnavailableError(str(error)) from error
+ return "".join(block.text for block in response.content if block.type == "text")
+```
+
+- [ ] **Step 4: Run the tests**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -k summary -v`
+Expected: PASS.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add ord_schema/search/nl.py ord_schema/search/nl_test.py
+git commit -m "Summarize a result and let the model describe it"
+```
+
+---
+
+### Task 6: The round trip
+
+**Files:**
+
+- Modify: `ord_schema/search/nl.py`, `ord_schema/search/nl_test.py`
+
+**Interfaces:**
+
+- Produces: `Answer(question, query, table, text)`; `ask(question, corpus, *, client=None, model=DEFAULT_MODEL, timeout_seconds=60.0) -> Answer`.
+
+- [ ] **Step 1: Write the failing test**
+
+```python
+def test_ask_returns_the_query_it_ran(corpus):
+ # The caller needs the query to show what was actually asked, and to offer a rerun.
+ client = _StubClient({"where": _WHERE}, _text("Two reactions, both with pyridine."))
+ result = nl.ask("solvent reactions", corpus, client=client)
+ assert result.query.where.op == "exists"
+ assert result.table.num_rows == result.table.num_rows
+ assert "pyridine" in result.text
+```
+
+Extend `_StubClient` so a canned entry that is a string becomes a text response:
+
+```python
+def _text(value: str):
+ return value
+
+
+# in _StubClient.create, before building the tool_use block:
+ if isinstance(payload, str):
+ block = types.SimpleNamespace(type="text", text=payload)
+ return types.SimpleNamespace(content=[block], usage=usage, stop_reason="end_turn")
+```
+
+- [ ] **Step 2: Run it**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -k ask -v`
+Expected: FAIL — `ask` does not exist.
+
+- [ ] **Step 3: Implement it**
+
+```python
+@dataclasses.dataclass(frozen=True)
+class Answer:
+ """What a question produced, including the query it became.
+
+ Attributes:
+ question: The question as asked.
+ query: The query that ran, for display and for running again.
+ table: What the search returned.
+ text: A sentence or two describing the result.
+ """
+
+ question: str
+ query: query.Query
+ table: pa.Table
+ text: str
+
+
+def ask(
+ question: str,
+ corpus: execute.Corpus,
+ *,
+ client: anthropic.Anthropic | None = None,
+ model: str = DEFAULT_MODEL,
+ timeout_seconds: float = 60.0,
+) -> Answer:
+ """Answers a question against a corpus.
+
+ Args:
+ question: The question, in English.
+ corpus: The corpus to search.
+ client: Anthropic client; one is built from the environment if omitted.
+ model: Which model translates and describes.
+ timeout_seconds: Passed to the search, so a slow query fails rather than hangs.
+
+ Returns:
+ The query, the reactions, and a description of them.
+
+ Raises:
+ MalformedQueryError: If translation does not produce a query that compiles.
+ ModelRateLimitedError: If the model is rate limited.
+ ModelUnavailableError: If the model cannot be reached.
+ """
+ client = client if client is not None else get_client()
+ translated = translate(question, client=client, model=model)
+ table = corpus.search(translated, timeout_seconds=timeout_seconds)
+ return Answer(
+ question=question,
+ query=translated,
+ table=table,
+ text=answer(question, table, client=client, model=model),
+ )
+```
+
+- [ ] **Step 4: Run the whole file**
+
+Run: `uv run pytest ord_schema/search/nl_test.py -v`
+Expected: PASS, and no test makes a network call.
+
+- [ ] **Step 5: Document it**
+
+Add to `ord_schema/search/README.md`, under `## Usage`:
+
+````markdown
+### Ask in English
+
+```python
+from ord_schema.search import execute, nl
+
+corpus = execute.Corpus("projections/*/*.parquet", "structures/*/*.parquet")
+answer = nl.ask("which reactions use pyridine as a solvent?", corpus)
+print(answer.query, answer.table.num_rows, answer.text)
+```
+
+Needs the `nl` extra (`pip install "ord-schema[nl]"`) and `ANTHROPIC_API_KEY`. The model
+cannot be constrained to emit a valid query — the grammar is recursive and both
+constrained-decoding paths refuse it — so a failure is handed back once with the
+compiler's own error, and a second failure raises `MalformedQueryError`.
+````
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add ord_schema/search/nl.py ord_schema/search/nl_test.py ord_schema/search/README.md
+git commit -m "Answer a question end to end"
+```
+
+---
+
+### Task 7: The eval harness
+
+**Files:**
+
+- Create: `ord_schema/search/nl_eval.py`, `ord_schema/search/nl_cases.yaml`, `ord_schema/search/nl_eval_test.py`
+
+**Interfaces:**
+
+- Consumes: `translate`, `execute.Corpus`.
+- Produces: `EvalCase`, `load_cases(path) -> list[EvalCase]`, `run_case(case, corpus, *, client, model, repair) -> CaseResult`, `report(results) -> str`.
+
+- [ ] **Step 1: Write the failing test**
+
+```python
+def test_a_case_that_returns_a_forbidden_reaction_fails(corpus):
+ # Scoring on reactions rather than on query shape is what lets the harness say a
+ # translation is wrong rather than merely differently spelled.
+ case = nl_eval.EvalCase(
+ question="solvent reactions",
+ must_return=["ord-aa000000000000000000000000000000"],
+ must_not_return=["ord-bb000000000000000000000000000000"],
+ )
+ result = nl_eval.score(case, returned=["ord-bb000000000000000000000000000000"])
+ assert not result.passed
+ assert "must_not_return" in result.detail
+```
+
+- [ ] **Step 2: Run it**
+
+Run: `uv run pytest ord_schema/search/nl_eval_test.py -v`
+Expected: FAIL — module does not exist.
+
+- [ ] **Step 3: Write the harness**
+
+```python
+class EvalCase(BaseModel):
+ """One question and what any correct answer to it must satisfy.
+
+ Attributes:
+ question: The question, in English.
+ must_return: Reaction IDs any correct query returns.
+ must_not_return: Reaction IDs a near-miss would wrongly include.
+ compiles: Whether the question should translate at all; False marks a question
+ the grammar cannot express, which the layer must refuse rather than fudge.
+ """
+
+ question: str
+ must_return: list[str] = Field(default_factory=list)
+ must_not_return: list[str] = Field(default_factory=list)
+ compiles: bool = True
+
+
+@dataclasses.dataclass(frozen=True)
+class CaseResult:
+ """How one case came out."""
+
+ case: EvalCase
+ passed: bool
+ detail: str
+
+
+def score(case: EvalCase, returned: Sequence[str]) -> CaseResult:
+ """Returns whether the reactions a query returned satisfy the case."""
+ found = set(returned)
+ missing = [value for value in case.must_return if value not in found]
+ forbidden = [value for value in case.must_not_return if value in found]
+ if missing:
+ return CaseResult(case, False, f"must_return absent: {missing}")
+ if forbidden:
+ return CaseResult(case, False, f"must_not_return present: {forbidden}")
+ return CaseResult(case, True, f"{len(found)} reactions")
+
+
+def load_cases(path: str) -> list[EvalCase]:
+ """Returns the cases a YAML file holds."""
+ with pathlib.Path(path).open(encoding="utf-8") as handle:
+ return [EvalCase.model_validate(entry) for entry in yaml.safe_load(handle)]
+```
+
+- [ ] **Step 4: Add the runner and the report**
+
+```python
+def run_case(
+ case: EvalCase,
+ corpus: execute.Corpus,
+ *,
+ client: anthropic.Anthropic,
+ model: str,
+ repair: bool,
+) -> CaseResult:
+ """Translates and runs one case, returning how it scored."""
+ try:
+ translated = nl.translate(case.question, client=client, model=model, repair=repair)
+ except nl.MalformedQueryError as error:
+ passed = not case.compiles
+ return CaseResult(case, passed, f"did not compile: {error}")
+ if not case.compiles:
+ return CaseResult(case, False, "compiled, but the case expects it cannot")
+ table = corpus.search(translated)
+ return score(case, table.column("reaction_id").to_pylist())
+
+
+def report(results: Sequence[CaseResult]) -> str:
+ """Returns a human-readable summary, failures first."""
+ passed = sum(result.passed for result in results)
+ lines = [f"{passed}/{len(results)} passed"]
+ lines += [
+ f" FAIL {result.case.question}: {result.detail}"
+ for result in results
+ if not result.passed
+ ]
+ return "\n".join(lines)
+```
+
+- [ ] **Step 5: Write the starting cases**
+
+`ord_schema/search/nl_cases.yaml`, with reaction IDs filled in by running each query by hand against the local corpus first:
+
+```yaml
+# Each case states what any correct translation must return, never a Query literal:
+# several spellings are right, and pinning one would fail a better query than the one
+# that was written when the case was added.
+- question: reactions using pyridine as the solvent
+ must_return: []
+ must_not_return: []
+- question: reactions run above 350 K
+ must_return: []
+ must_not_return: []
+- question: the ten highest-yielding reactions
+ must_return: []
+ must_not_return: []
+```
+
+- [ ] **Step 6: Run the tests**
+
+Run: `uv run pytest ord_schema/search/nl_eval_test.py -v`
+Expected: PASS.
+
+- [ ] **Step 7: Take the first real measurement**
+
+Run, with a key in the environment and the local corpus:
+
+```bash
+uv run python -m ord_schema.search.nl_eval --model claude-haiku-4-5 --repair
+uv run python -m ord_schema.search.nl_eval --model claude-opus-5 --repair
+```
+
+Record both in the logbook entry rather than in a commit message: the numbers in
+finding 6 came from ten ad-hoc questions and want replacing with these.
+
+- [ ] **Step 8: Commit**
+
+```bash
+git add ord_schema/search/nl_eval.py ord_schema/search/nl_cases.yaml ord_schema/search/nl_eval_test.py
+git commit -m "Score translations on the reactions they return"
+```
+
+---
+
+## Self-review
+
+**Spec coverage.** `ask`/`translate`/`answer` — Tasks 3, 5, 6. Cached prefix — Task 3, pinned by a test. Coercion — Task 3. One repair turn — Tasks 3 and 4. Summary not table — Task 5. Module layout and the `nl` extra — Task 2. Error taxonomy — Task 2. Eval harness scoring on reactions — Task 7. The reduction gap — Task 1. Out of scope in the spec and absent here: multi-turn, a second backend, constrained decoding, text-search tuning.
+
+**Types.** `translate` returns `query.Query` in Tasks 3, 6, and 7. `Answer.query` is that same type. `summarize` and `answer` both take `pa.Table`, which is what `Corpus.search` returns. `Reduction` is referenced by `Order.key` and `Measure.path` only.
+
+**Known gaps, deliberately left to the executor.** The eval cases in Task 7 ship with empty `must_return` lists, because the IDs have to come from running each query against the local corpus — filling them in is step 5's work, and inventing IDs here would be worse than leaving them empty. Task 1's step 6 needs the local projections; it is a check, not a test, and does not run in CI.
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_bisect.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_bisect.py
new file mode 100644
index 0000000..308697e
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_bisect.py
@@ -0,0 +1,60 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Which part of the grammar makes the compiled decoding grammar too large?"""
+
+import json
+import pathlib
+import sys
+
+import anthropic
+from anthropic.lib._parse._transform import transform_schema
+from pydantic import BaseModel
+
+sys.path.insert(0, str(pathlib.Path(__file__).parent))
+from stratify import stratify # noqa: E402
+
+from ord_schema.search import query # noqa: E402
+
+
+class Tiny(BaseModel):
+ name: str
+ count: int
+
+
+def sized(schema: dict) -> str:
+ return f"{len(json.dumps(schema)) // 4:,} ~tokens"
+
+
+client = anthropic.Anthropic()
+cases = [
+ ("a tiny two-field object", transform_schema(Tiny.model_json_schema())),
+ ("one comparison predicate", transform_schema(query.Comparison.model_json_schema())),
+ ("one quantifier", transform_schema(query.Quantifier.model_json_schema())),
+ ("Query, stratified depth 0", transform_schema(stratify(query.Query.model_json_schema(), 0))),
+ ("Query, stratified depth 1", transform_schema(stratify(query.Query.model_json_schema(), 1))),
+]
+for label, schema in cases:
+ try:
+ client.messages.create(
+ model="claude-haiku-4-5",
+ max_tokens=512,
+ messages=[{"role": "user", "content": "reactions above 350 K"}],
+ output_config={"format": {"type": "json_schema", "schema": schema}},
+ )
+ except anthropic.APIStatusError as error:
+ message = json.loads(error.response.text)["error"]["message"]
+ print(f" {label:32} {sized(schema):>16} REFUSED: {message[:80]}")
+ else:
+ print(f" {label:32} {sized(schema):>16} accepted")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_flat_union.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_flat_union.py
new file mode 100644
index 0000000..9171b67
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_flat_union.py
@@ -0,0 +1,113 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Can a flattened predicate -- one object per level, path as an enum -- fit the budget?
+
+The union of eight predicate variants is what multiplies the state machine, so this
+collapses them into one object whose `op` says which shape it is. Paths become an enum,
+which is the only way the decoder can stop a model inventing `identifiers[*]`.
+"""
+
+import json
+import sys
+
+import anthropic
+import pyarrow as pa
+
+from ord_schema.artifacts import projection
+
+client = anthropic.Anthropic()
+
+
+def leaf_paths(schema: pa.Schema) -> list[str]:
+ """Returns every scalar column path in the projection, dotted."""
+ found: list[str] = []
+
+ def walk(field: pa.Field, prefix: str) -> None:
+ path = f"{prefix}.{field.name}" if prefix else field.name
+ dtype = field.type
+ while pa.types.is_list(dtype) or pa.types.is_map(dtype):
+ dtype = dtype.item_type if pa.types.is_map(dtype) else dtype.value_type
+ if pa.types.is_struct(dtype):
+ for child in dtype:
+ walk(child, path)
+ else:
+ found.append(path)
+
+ for field in schema:
+ walk(field, "")
+ return found
+
+
+PATHS = leaf_paths(projection.SCHEMA)
+print(f"projection leaves: {len(PATHS)}")
+
+
+def predicate(level: int, paths: list[str]) -> dict:
+ """Returns a flat predicate object whose clauses are one level shallower."""
+ node: dict = {
+ "type": "object",
+ "additionalProperties": False,
+ "properties": {
+ "op": {
+ "type": "string",
+ "enum": [
+ "and", "or", "not", "exists", "forall",
+ "eq", "ne", "lt", "le", "gt", "ge",
+ "contains", "starts_with", "ends_with",
+ "is_null", "not_null", "substructure", "similarity",
+ ],
+ },
+ "path": {"anyOf": [{"type": "string", "enum": paths}, {"type": "null"}]},
+ "literal": {"anyOf": [{"type": "string"}, {"type": "number"},
+ {"type": "boolean"}, {"type": "null"}]},
+ "compound": {"anyOf": [{"type": "string"}, {"type": "null"}]},
+ "smarts": {"anyOf": [{"type": "string"}, {"type": "null"}]},
+ },
+ "required": ["op", "path", "literal", "compound", "smarts"],
+ }
+ if level > 0:
+ node["properties"]["clauses"] = {
+ "anyOf": [{"type": "array", "items": predicate(level - 1, paths)},
+ {"type": "null"}]
+ }
+ node["required"].append("clauses")
+ return node
+
+
+for count in (0, 40, 120, len(PATHS)):
+ paths = PATHS[:count] if count else [""]
+ for depth in (2, 3, 4):
+ schema = {
+ "type": "object",
+ "additionalProperties": False,
+ "properties": {"where": predicate(depth, paths)},
+ "required": ["where"],
+ }
+ if not count: # free-form path, for comparison with the enum forms
+ schema = json.loads(json.dumps(schema).replace(
+ '{"type": "string", "enum": [""]}', '{"type": "string"}'))
+ size = f"{len(json.dumps(schema)) // 4:,} ~tok"
+ label = f"paths={count or 'free':>4} depth={depth}"
+ try:
+ client.messages.create(
+ model="claude-haiku-4-5", max_tokens=256,
+ messages=[{"role": "user", "content": "reactions above 350 K"}],
+ output_config={"format": {"type": "json_schema", "schema": schema}},
+ )
+ except anthropic.APIStatusError as error:
+ message = json.loads(error.response.text)["error"]["message"]
+ print(f" {label} {size:>12} REFUSED: {message[:52]}")
+ else:
+ print(f" {label} {size:>12} ACCEPTED")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_recursion.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_recursion.py
new file mode 100644
index 0000000..feebe96
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_recursion.py
@@ -0,0 +1,103 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""What does the SDK send for a recursive output_format, and does it object first?
+
+Captures the request body with a mock transport rather than a live call, so the
+client-side half of the question is settled without a key: whether the SDK accepts a
+self-referencing pydantic model at all, and what JSON Schema it puts on the wire.
+"""
+
+import json
+
+import anthropic
+import httpx
+
+from ord_schema.search import query
+
+CANNED = {
+ "id": "msg_probe",
+ "type": "message",
+ "role": "assistant",
+ "model": "claude-opus-5",
+ "content": [{"type": "text", "text": "{}"}],
+ "stop_reason": "end_turn",
+ "usage": {"input_tokens": 1, "output_tokens": 1},
+}
+sent = {}
+
+
+def handler(request: httpx.Request) -> httpx.Response:
+ sent["body"] = json.loads(request.content)
+ return httpx.Response(200, json=CANNED)
+
+
+client = anthropic.Anthropic(
+ api_key="probe-not-a-real-key",
+ http_client=httpx.Client(transport=httpx.MockTransport(handler)),
+)
+
+print("anthropic SDK:", anthropic.__version__)
+for label, call in (
+ (
+ "messages.parse(output_format=Query)",
+ lambda: client.messages.parse(
+ model="claude-opus-5",
+ max_tokens=1024,
+ messages=[{"role": "user", "content": "pyridine as solvent"}],
+ output_format=query.Query,
+ ),
+ ),
+ (
+ "messages.create(tools=[strict build_query])",
+ lambda: client.messages.create(
+ model="claude-opus-5",
+ max_tokens=1024,
+ messages=[{"role": "user", "content": "pyridine as solvent"}],
+ tools=[
+ {
+ "name": "build_query",
+ "description": "Build a search query.",
+ "strict": True,
+ "input_schema": query.Query.model_json_schema(),
+ }
+ ],
+ tool_choice={"type": "tool", "name": "build_query"},
+ ),
+ ),
+):
+ sent.clear()
+ print(f"\n=== {label}")
+ try:
+ call()
+ except Exception as error: # noqa: BLE001 -- the probe reports whatever it hits.
+ print(f" raised {type(error).__name__}: {str(error)[:200]}")
+ body = sent.get("body")
+ if body is None:
+ print(" nothing reached the wire")
+ continue
+ schema = None
+ if "output_config" in body:
+ schema = body["output_config"].get("format", {}).get("schema")
+ print(" output_config.format keys:", sorted(body["output_config"]["format"]))
+ elif "tools" in body:
+ schema = body["tools"][0].get("input_schema")
+ print(" tool keys:", sorted(body["tools"][0]))
+ if schema is None:
+ print(" no schema on the wire")
+ continue
+ text = json.dumps(schema)
+ print(f" schema on the wire: {len(text)} chars, $defs={len(schema.get('$defs', {}))}")
+ print(f" contains $ref: {'$ref' in text}, refs={text.count('#/$defs/')}")
+ print(f" additionalProperties present: {text.count('additionalProperties')}")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_recursion_live.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_recursion_live.py
new file mode 100644
index 0000000..419290c
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_recursion_live.py
@@ -0,0 +1,57 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Does the API accept a recursive json_schema output format? Needs a real key.
+
+ ANTHROPIC_API_KEY=sk-... uv run --with anthropic python probe_recursion_live.py
+
+Prints the validated Query the model produced, or the error the server returned.
+"""
+
+import json
+
+import anthropic
+
+from ord_schema.search import query, schema
+
+SYSTEM = (
+ "You translate chemistry questions into ORD search queries. The corpus schema, "
+ "as an indented type tree in DuckDB's types:\n\n" + schema.describe()
+)
+
+client = anthropic.Anthropic()
+try:
+ response = client.messages.parse(
+ model="claude-opus-5",
+ max_tokens=2048,
+ system=[{"type": "text", "text": SYSTEM, "cache_control": {"type": "ephemeral"}}],
+ messages=[
+ {
+ "role": "user",
+ "content": "reactions using pyridine as the solvent with a yield above 50%",
+ }
+ ],
+ output_format=query.Query,
+ )
+except anthropic.APIStatusError as error:
+ print(f"server refused it: {error.status_code} {str(error)[:400]}")
+else:
+ print("accepted. parsed_output:")
+ print(json.dumps(response.parsed_output.model_dump(exclude_none=True), indent=2))
+ usage = response.usage
+ print(
+ f"\ninput={usage.input_tokens} "
+ f"cache_write={usage.cache_creation_input_tokens} "
+ f"cache_read={usage.cache_read_input_tokens} output={usage.output_tokens}"
+ )
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_repair.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_repair.py
new file mode 100644
index 0000000..2220dd4
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_repair.py
@@ -0,0 +1,130 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Does one repair turn carry a cheap model to the accuracy of an expensive one?
+
+The compiler's errors name the offending path and suggest a real one, so a failed
+translation is handed straight back. Scores first-try and after-repair separately.
+"""
+
+import json
+import sys
+
+import anthropic
+
+from ord_schema.search import query, schema
+
+QUESTIONS = [
+ "reactions using pyridine as the solvent",
+ "reactions using pyridine as the solvent with a yield above 50%",
+ "reactions run above 350 K",
+ "reactions where every input component is a liquid",
+ "average yield by reaction temperature, for reactions with a boronic acid",
+ "reactions that make aspirin",
+ "the ten highest-yielding Suzuki couplings",
+ "reactions with no solvent at all",
+ "how many reactions use palladium catalysts",
+ "reactions stirred for more than an hour at above 100 C",
+]
+SYSTEM = [
+ {
+ "type": "text",
+ "text": (
+ "You translate chemistry questions into ORD search queries by calling "
+ "build_query. The corpus schema, as an indented type tree in DuckDB's "
+ "types:\n\n" + schema.describe()
+ ),
+ "cache_control": {"type": "ephemeral"},
+ }
+]
+TOOL = {
+ "name": "build_query",
+ "description": "Build an ORD search query from the user's question.",
+ "input_schema": query.Query.model_json_schema(),
+}
+MODEL = sys.argv[1] if len(sys.argv) > 1 else "claude-haiku-4-5"
+
+
+def coerce(value):
+ """Returns the input with any JSON-encoded string values parsed back to objects."""
+ if isinstance(value, str):
+ try:
+ return coerce(json.loads(value))
+ except (json.JSONDecodeError, TypeError):
+ return value
+ if isinstance(value, dict):
+ return {key: coerce(item) for key, item in value.items()}
+ if isinstance(value, list):
+ return [coerce(item) for item in value]
+ return value
+
+
+def check(raw) -> str | None:
+ """Returns the error a translation fails with, or None if it compiles."""
+ try:
+ query.compile_query(query.Query.model_validate(coerce(raw)))
+ except Exception as error: # noqa: BLE001 -- whatever it fails with is the message.
+ return str(error)
+ return None
+
+
+client = anthropic.Anthropic()
+first_try = repaired = 0
+cost = {"cache_read": 0, "input": 0, "output": 0}
+for question in QUESTIONS:
+ messages: list = [{"role": "user", "content": question}]
+ response = client.messages.create(
+ model=MODEL, max_tokens=2048, system=SYSTEM, messages=messages,
+ tools=[TOOL], tool_choice={"type": "tool", "name": "build_query"},
+ )
+ cost["cache_read"] += response.usage.cache_read_input_tokens or 0
+ cost["input"] += response.usage.input_tokens
+ cost["output"] += response.usage.output_tokens
+ block = next(b for b in response.content if b.type == "tool_use")
+ error = check(block.input)
+ if error is None:
+ first_try += 1
+ repaired += 1
+ print(f" {question[:48]:50} ok")
+ continue
+ messages += [
+ {"role": "assistant", "content": response.content},
+ {
+ "role": "user",
+ "content": [
+ {
+ "type": "tool_result",
+ "tool_use_id": block.id,
+ "is_error": True,
+ "content": f"That query was rejected: {error}. Call build_query again with it fixed.",
+ }
+ ],
+ },
+ ]
+ retry = client.messages.create(
+ model=MODEL, max_tokens=2048, system=SYSTEM, messages=messages,
+ tools=[TOOL], tool_choice={"type": "tool", "name": "build_query"},
+ )
+ cost["cache_read"] += retry.usage.cache_read_input_tokens or 0
+ cost["input"] += retry.usage.input_tokens
+ cost["output"] += retry.usage.output_tokens
+ retry_block = next(b for b in retry.content if b.type == "tool_use")
+ second = check(retry_block.input)
+ if second is None:
+ repaired += 1
+ print(f" {question[:48]:50} repaired ({error.split(';')[0][:44]})")
+ else:
+ print(f" {question[:48]:50} STILL FAILS: {second[:60]}")
+print(f"\n{MODEL}: first try {first_try}/{len(QUESTIONS)}, after repair {repaired}/{len(QUESTIONS)}")
+print(f"tokens: {cost}")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_required.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_required.py
new file mode 100644
index 0000000..b580fb8
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_required.py
@@ -0,0 +1,46 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Does requiring every property bring the grammar under the compiler's budget?"""
+
+import copy
+import json
+import pathlib
+import sys
+
+import anthropic
+
+sys.path.insert(0, str(pathlib.Path(__file__).parent))
+from require_all import require_all # noqa: E402
+from stratify import stratify # noqa: E402
+
+from ord_schema.search import query # noqa: E402
+
+client = anthropic.Anthropic()
+base = query.Query.model_json_schema()
+for depth in range(0, 6):
+ schema = require_all(stratify(copy.deepcopy(base), depth), strip_descriptions=True)
+ size = f"{len(json.dumps(schema)) // 4:,} ~tokens"
+ try:
+ client.messages.create(
+ model="claude-haiku-4-5",
+ max_tokens=512,
+ messages=[{"role": "user", "content": "reactions above 350 K"}],
+ output_config={"format": {"type": "json_schema", "schema": schema}},
+ )
+ except anthropic.APIStatusError as error:
+ message = json.loads(error.response.text)["error"]["message"]
+ print(f" depth {depth} {size:>14} REFUSED: {message[:78]}")
+ else:
+ print(f" depth {depth} {size:>14} ACCEPTED")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_shapes.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_shapes.py
new file mode 100644
index 0000000..387845f
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_shapes.py
@@ -0,0 +1,110 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""How often does a forced tool call over the recursive grammar parse as written?
+
+Runs a handful of questions through two models and scores three ways: as returned,
+after coercing JSON-encoded strings back to objects, and whether the compiler accepts
+the result. The gap between the first two is what a strict schema would have bought.
+"""
+
+import json
+
+import anthropic
+
+from ord_schema.search import query, schema
+
+QUESTIONS = [
+ "reactions using pyridine as the solvent",
+ "reactions using pyridine as the solvent with a yield above 50%",
+ "reactions run above 350 K",
+ "reactions where every input component is a liquid",
+ "average yield by reaction temperature, for reactions with a boronic acid",
+]
+SYSTEM = [
+ {
+ "type": "text",
+ "text": (
+ "You translate chemistry questions into ORD search queries. Emit the query "
+ "by calling build_query. The corpus schema, as an indented type tree in "
+ "DuckDB's types:\n\n" + schema.describe()
+ ),
+ "cache_control": {"type": "ephemeral"},
+ }
+]
+TOOL = {
+ "name": "build_query",
+ "description": "Build an ORD search query from the user's question.",
+ "input_schema": query.Query.model_json_schema(),
+}
+
+
+def coerce(value):
+ """Returns the input with any JSON-encoded string values parsed back to objects."""
+ if isinstance(value, str):
+ try:
+ return coerce(json.loads(value))
+ except (json.JSONDecodeError, TypeError):
+ return value
+ if isinstance(value, dict):
+ return {key: coerce(item) for key, item in value.items()}
+ if isinstance(value, list):
+ return [coerce(item) for item in value]
+ return value
+
+
+client = anthropic.Anthropic()
+for model in ("claude-opus-5", "claude-haiku-4-5"):
+ print(f"\n=== {model}")
+ tallies = {"as written": 0, "coerced": 0, "compiles": 0}
+ cost = {"cache_write": 0, "cache_read": 0, "input": 0, "output": 0}
+ for question in QUESTIONS:
+ response = client.messages.create(
+ model=model,
+ max_tokens=2048,
+ system=SYSTEM,
+ messages=[{"role": "user", "content": question}],
+ tools=[TOOL],
+ tool_choice={"type": "tool", "name": "build_query"},
+ )
+ usage = response.usage
+ cost["cache_write"] += usage.cache_creation_input_tokens or 0
+ cost["cache_read"] += usage.cache_read_input_tokens or 0
+ cost["input"] += usage.input_tokens
+ cost["output"] += usage.output_tokens
+ raw = next(b for b in response.content if b.type == "tool_use").input
+ note = []
+ try:
+ query.Query.model_validate(raw)
+ except Exception: # noqa: BLE001 -- scoring, not handling.
+ note.append("as-written FAILED")
+ else:
+ tallies["as written"] += 1
+ note.append("as-written ok")
+ try:
+ parsed = query.Query.model_validate(coerce(raw))
+ except Exception as error: # noqa: BLE001
+ note.append(f"coerced FAILED: {str(error)[:60]}")
+ else:
+ tallies["coerced"] += 1
+ try:
+ query.compile_query(parsed)
+ except Exception as error: # noqa: BLE001
+ note.append(f"compile FAILED: {str(error)[:80]}")
+ else:
+ tallies["compiles"] += 1
+ note.append("compiles")
+ print(f" {question[:52]:54} {' | '.join(note)}")
+ print(f" tallies (of {len(QUESTIONS)}): {tallies}")
+ print(f" tokens: {cost}")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_stratified.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_stratified.py
new file mode 100644
index 0000000..1617419
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_stratified.py
@@ -0,0 +1,93 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Does a stratified grammar unlock structured outputs, and can a cheap model hit it?
+
+Same five questions as the tool-call probe, but the model is handed an acyclic schema
+and constrained by output_config.format rather than asked to call a tool.
+"""
+
+import json
+import pathlib
+import sys
+
+import anthropic
+from anthropic.lib._parse._transform import transform_schema
+
+sys.path.insert(0, str(pathlib.Path(__file__).parent))
+from stratify import stratify # noqa: E402
+
+from ord_schema.search import query, schema # noqa: E402
+
+QUESTIONS = [
+ "reactions using pyridine as the solvent",
+ "reactions using pyridine as the solvent with a yield above 50%",
+ "reactions run above 350 K",
+ "reactions where every input component is a liquid",
+ "average yield by reaction temperature, for reactions with a boronic acid",
+]
+SYSTEM = [
+ {
+ "type": "text",
+ "text": (
+ "You translate chemistry questions into ORD search queries. The corpus "
+ "schema, as an indented type tree in DuckDB's types:\n\n" + schema.describe()
+ ),
+ "cache_control": {"type": "ephemeral"},
+ }
+]
+DEPTH = int(sys.argv[1]) if len(sys.argv) > 1 else 4
+MODEL = sys.argv[2] if len(sys.argv) > 2 else "claude-haiku-4-5"
+STRATIFIED = transform_schema(stratify(query.Query.model_json_schema(), DEPTH))
+
+client = anthropic.Anthropic()
+print(f"=== {MODEL}, stratified depth {DEPTH} "
+ f"({len(json.dumps(STRATIFIED))//4:,} ~tokens)")
+tallies = {"parses": 0, "compiles": 0}
+cost = {"cache_write": 0, "cache_read": 0, "input": 0, "output": 0}
+for question in QUESTIONS:
+ try:
+ response = client.messages.create(
+ model=MODEL,
+ max_tokens=2048,
+ system=SYSTEM,
+ messages=[{"role": "user", "content": question}],
+ output_config={"format": {"type": "json_schema", "schema": STRATIFIED}},
+ )
+ except anthropic.APIStatusError as error:
+ print(f" refused: {error.status_code} {str(error)[:300]}")
+ break
+ usage = response.usage
+ cost["cache_write"] += usage.cache_creation_input_tokens or 0
+ cost["cache_read"] += usage.cache_read_input_tokens or 0
+ cost["input"] += usage.input_tokens
+ cost["output"] += usage.output_tokens
+ text = next(b.text for b in response.content if b.type == "text")
+ note = []
+ try:
+ parsed = query.Query.model_validate(json.loads(text))
+ except Exception as error: # noqa: BLE001 -- scoring, not handling.
+ note.append(f"parse FAILED: {str(error)[:90]}")
+ else:
+ tallies["parses"] += 1
+ try:
+ query.compile_query(parsed)
+ except Exception as error: # noqa: BLE001
+ note.append(f"compile FAILED: {str(error)[:90]}")
+ else:
+ tallies["compiles"] += 1
+ note.append("compiles")
+ print(f" {question[:52]:54} {' | '.join(note)}")
+print(f" tallies (of {len(QUESTIONS)}): {tallies}")
+print(f" tokens: {cost}")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_strict_tool.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_strict_tool.py
new file mode 100644
index 0000000..eb52917
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_strict_tool.py
@@ -0,0 +1,52 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Does the strict-tool path have a bigger grammar budget than output_config.format?"""
+
+import json
+import pathlib
+import sys
+
+import anthropic
+from anthropic.lib._parse._transform import transform_schema
+
+sys.path.insert(0, str(pathlib.Path(__file__).parent))
+from stratify import stratify # noqa: E402
+
+from ord_schema.search import query # noqa: E402
+
+client = anthropic.Anthropic()
+base = query.Query.model_json_schema()
+for depth in (0, 1, 2, 4):
+ schema = transform_schema(stratify(base, depth))
+ tool = {
+ "name": "build_query",
+ "description": "Build an ORD search query.",
+ "strict": True,
+ "input_schema": schema,
+ }
+ size = f"{len(json.dumps(schema)) // 4:,} ~tokens"
+ try:
+ client.messages.create(
+ model="claude-haiku-4-5",
+ max_tokens=512,
+ messages=[{"role": "user", "content": "reactions above 350 K"}],
+ tools=[tool],
+ tool_choice={"type": "tool", "name": "build_query"},
+ )
+ except anthropic.APIStatusError as error:
+ message = json.loads(error.response.text)["error"]["message"]
+ print(f" strict tool, depth {depth} {size:>16} REFUSED: {message[:70]}")
+ else:
+ print(f" strict tool, depth {depth} {size:>16} accepted")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_tool_recursion.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_tool_recursion.py
new file mode 100644
index 0000000..e39180b
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/probe_tool_recursion.py
@@ -0,0 +1,80 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Do tool input schemas accept the recursion that output_config.format refuses?
+
+Three shapes, cheapest question that still needs a nested predicate: a forced tool call
+over the grammar as pydantic writes it, the same with strict on and the SDK's own
+closing transform applied, and -- for reference -- what the model actually produces.
+"""
+
+import json
+
+import anthropic
+from anthropic.lib._parse._transform import transform_schema
+
+from ord_schema.search import query, schema
+
+QUESTION = "reactions using pyridine as the solvent with a yield above 50%"
+SYSTEM = [
+ {
+ "type": "text",
+ "text": (
+ "You translate chemistry questions into ORD search queries. The corpus "
+ "schema, as an indented type tree in DuckDB's types:\n\n" + schema.describe()
+ ),
+ "cache_control": {"type": "ephemeral"},
+ }
+]
+RAW = query.Query.model_json_schema()
+
+client = anthropic.Anthropic()
+for label, input_schema, strict in (
+ ("tool, as pydantic writes it", RAW, False),
+ ("tool, strict + closed", transform_schema(json.loads(json.dumps(RAW))), True),
+):
+ tool: dict = {
+ "name": "build_query",
+ "description": "Build an ORD search query from the user's question.",
+ "input_schema": input_schema,
+ }
+ if strict:
+ tool["strict"] = True
+ print(f"\n=== {label}")
+ try:
+ response = client.messages.create(
+ model="claude-opus-5",
+ max_tokens=2048,
+ system=SYSTEM,
+ messages=[{"role": "user", "content": QUESTION}],
+ tools=[tool],
+ tool_choice={"type": "tool", "name": "build_query"},
+ )
+ except anthropic.APIStatusError as error:
+ print(f" refused: {error.status_code} {str(error)[:220]}")
+ continue
+ block = next(b for b in response.content if b.type == "tool_use")
+ print(" accepted. tool_use.input:")
+ print(json.dumps(block.input, indent=2)[:900])
+ try:
+ parsed = query.Query.model_validate(block.input)
+ except Exception as error: # noqa: BLE001 -- the probe reports whatever it hits.
+ print(f" pydantic REJECTED it: {type(error).__name__}: {str(error)[:200]}")
+ else:
+ print(f" pydantic accepted it: {parsed.where.op}")
+ usage = response.usage
+ print(
+ f" input={usage.input_tokens} cache_write={usage.cache_creation_input_tokens} "
+ f"cache_read={usage.cache_read_input_tokens} output={usage.output_tokens}"
+ )
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/require_all.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/require_all.py
new file mode 100644
index 0000000..83c9b6c
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/require_all.py
@@ -0,0 +1,105 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Makes every property required, expressing optionality as an explicit null instead.
+
+Constrained decoding compiles a schema into a state machine, and an optional property
+doubles it: the machine has to accept the object with and without that key. Requiring
+every key and letting the value be null costs one token per absent field and leaves the
+machine linear in the number of properties.
+"""
+
+import copy
+
+# Keywords a decoder rejects or ignores. Dropping them costs nothing: the response is
+# validated by the pydantic models afterwards, which enforce all of them anyway.
+_UNSUPPORTED = frozenset(
+ {
+ "minimum",
+ "maximum",
+ "exclusiveMinimum",
+ "exclusiveMaximum",
+ "multipleOf",
+ "minLength",
+ "maxLength",
+ "pattern",
+ "maxItems",
+ "uniqueItems",
+ "default",
+ }
+)
+
+
+def require_all(node, strip_descriptions: bool = False):
+ """Returns node with every property required and nullable where it was optional."""
+ if isinstance(node, list):
+ return [require_all(item, strip_descriptions) for item in node]
+ if not isinstance(node, dict):
+ return node
+ out = {}
+ for key, value in node.items():
+ if strip_descriptions and key in ("description", "title", "examples"):
+ continue
+ # A decoder takes anyOf but not oneOf, and the discriminator that rides with it
+ # is pydantic's business rather than the model's: validation still happens here.
+ if key == "discriminator" or key in _UNSUPPORTED:
+ continue
+ out["anyOf" if key == "oneOf" else key] = require_all(value, strip_descriptions)
+ properties = out.get("properties")
+ if isinstance(properties, dict) and properties:
+ required = set(out.get("required", []))
+ for name, subschema in properties.items():
+ if name in required:
+ continue
+ branches = subschema.get("anyOf") if isinstance(subschema, dict) else None
+ if branches and any(b.get("type") == "null" for b in branches):
+ continue
+ properties[name] = {"anyOf": [subschema, {"type": "null"}]}
+ out["required"] = sorted(properties)
+ out["additionalProperties"] = False
+ return out
+
+
+def count_optional(node, seen=None) -> int:
+ """Returns how many properties a schema leaves optional."""
+ total = 0
+ if isinstance(node, dict):
+ properties = node.get("properties")
+ if isinstance(properties, dict):
+ total += len(set(properties) - set(node.get("required", [])))
+ for value in node.values():
+ total += count_optional(value)
+ elif isinstance(node, list):
+ for item in node:
+ total += count_optional(item)
+ return total
+
+
+if __name__ == "__main__":
+ import json
+ import sys
+
+ sys.path.insert(0, str(pathlib.Path(__file__).parent))
+ from stratify import stratify
+
+ from ord_schema.search import query
+
+ base = query.Query.model_json_schema()
+ print(f"{'depth':>6} {'optional before':>16} {'after':>6} {'~tokens':>9}")
+ for depth in range(0, 5):
+ built = stratify(copy.deepcopy(base), depth)
+ before = count_optional(built)
+ required = require_all(built, strip_descriptions=True)
+ text = json.dumps(required)
+ print(f"{depth:>6} {before:>16} {count_optional(required):>6} {len(text)//4:>9,}")
diff --git a/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/stratify.py b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/stratify.py
new file mode 100644
index 0000000..2ff2ce5
--- /dev/null
+++ b/entries/2026-08-17-what-constrains-a-natural-language-layer/assets/stratify.py
@@ -0,0 +1,109 @@
+# Copyright 2026 Open Reaction Database Project Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Turns the recursive grammar into an acyclic one by stratifying it into levels.
+
+The API refuses a schema whose definitions reference each other in a cycle, which is
+what a predicate tree is. Stratifying keeps every definition but numbers it: a level-k
+predicate's clauses are level-(k-1) predicates, and level 0 holds only the leaves. The
+refs stay refs -- nothing is inlined, so size grows by a level rather than by a power --
+and the result validates into the same recursive pydantic models.
+"""
+
+import copy
+import json
+
+
+def _referenced(node, found):
+ """Collects the definition names ``node`` references."""
+ if isinstance(node, dict):
+ ref = node.get("$ref")
+ if ref:
+ found.add(ref.split("/")[-1])
+ for value in node.values():
+ _referenced(value, found)
+ elif isinstance(node, list):
+ for item in node:
+ _referenced(item, found)
+ return found
+
+
+def _retarget(node, level, levelled):
+ """Returns node with refs pointing at the next level down."""
+ if isinstance(node, dict):
+ ref = node.get("$ref")
+ if ref:
+ name = ref.split("/")[-1]
+ if name not in levelled:
+ return {"$ref": f"#/$defs/{name}"}
+ if level <= 0:
+ return None
+ return {"$ref": f"#/$defs/{name}_L{level - 1}"}
+ out = {}
+ for key, value in node.items():
+ result = _retarget(value, level, levelled)
+ if result is None:
+ if key in ("anyOf", "items", "properties"):
+ return None
+ continue
+ if isinstance(result, list) and not result:
+ return None
+ out[key] = result
+ if "anyOf" in out and not out["anyOf"]:
+ return None
+ return out
+ if isinstance(node, list):
+ kept = [_retarget(item, level, levelled) for item in node]
+ return [item for item in kept if item is not None]
+ return node
+
+
+def stratify(schema: dict, depth: int) -> dict:
+ """Returns an acyclic schema whose predicates nest at most ``depth`` levels.
+
+ Args:
+ schema: A JSON Schema with a recursive ``$defs`` section.
+ depth: How many levels of nesting to allow; level 0 holds only leaves.
+
+ Returns:
+ A schema with no reference cycle, its definitions suffixed by level.
+ """
+ schema = copy.deepcopy(schema)
+ defs = schema.pop("$defs", {})
+ # A definition that reaches another definition is one a level has to renumber;
+ # leaves reference nothing, so one copy of each serves every level.
+ levelled = {name for name, node in defs.items() if _referenced(node, set())}
+ out: dict = {}
+ for name, node in defs.items():
+ if name not in levelled:
+ out[name] = node
+ continue
+ for level in range(depth + 1):
+ built = _retarget(copy.deepcopy(node), level, levelled)
+ if built is not None:
+ out[f"{name}_L{level}"] = built
+ root = _retarget(schema, depth, levelled)
+ root["$defs"] = out
+ return root
+
+
+if __name__ == "__main__":
+ from ord_schema.search import query
+
+ base = query.Query.model_json_schema()
+ print(f"{'depth':>6} {'chars':>9} {'~tokens':>8} {'defs':>6}")
+ for depth in range(1, 7):
+ built = stratify(base, depth)
+ text = json.dumps(built)
+ print(f"{depth:>6} {len(text):>9,} {len(text)//4:>8,} {len(built['$defs']):>6}")