Synopsis surfaces: LLM-derived summaries as indexed surfaces (document-summary routing fuel) #48

Activity

1. thorwhalen

   added

   enhancementNew feature or requestNew feature or request

   on Jun 12, 2026
2. thorwhalen commented on Jun 13, 2026

   

   thorwhalen

   on Jun 13, 2026

   MemberAuthor

   More actions

   Design decision (confirming before coding).

   Module. New ir/synopsis.py (an operator module beside expand.py/formulate.py/graph.py):

   • with_synopsis(strategy, *, synthesize=None, synthesizer_id=None, synopsis_kind="synopsis") -> IndexingStrategy — wraps any strategy, adds one synopsis surface per artifact.
   • make_llm_synthesizer(*, summarize=None, prompt=SYNOPSIS_PROMPT, model=None, synthesizer_id=None, **prompt_function_kwargs) -> Synthesizer — the lazy-oa default (mirrors make_llm_formulator/make_llm_selector; oa imported only inside the call, so import ir stays offline).
   • Synthesizer = Callable[[Artifact], str].

   Surface placement. The synopsis is prepended (plan position 0) so it is the first summary surface → the collapsed-tree router (the #48 intent: route from a synopsis match). On with_synopsis(Package()) the synopsis (not the terse description) routes. An empty synopsis (synth returns "") is dropped — the artifact keeps its other surfaces.

   Staleness — reuse the existing ledger mechanism, no new bookkeeping. The wrapper exposes its identity as scalar attrs (synthesizer_id, synopsis_kind) and holds the inner strategy. I make index._strategy_id recurse into nested strategy attributes (a small, generic change: any attr with a decompose method folds in its own _strategy_id). So strategy_id encodes (inner strategy + params) + synthesizer identity → a model/prompt change OR an inner-param change re-synthesizes exactly the affected artifacts through the normal incremental path. Default synthesizer_id = oa:{model}:{sha(prompt)[:12]}; an injected double should pass synthesizer_id= (else falls back to its __qualname__).

   Routing is surface-level, not links-view edges. collapsed_tree (#47) descends synopsis→chunks within an artifact via records_for_artifact (same artifact_id), so synopsis routing needs no edges. I'm therefore not writing synopsis→chunk PARENT/CHILD edges into the #46 links view: that view is artifact-keyed with artifact-id targets (cross-artifact REF/PARENT), whereas synopsis→chunk is surface→surface within one artifact — a different grain. This refines the issue body's wording. (Cross-artifact synopsis edges, if ever wanted, are just another edge_extractor.)

   Known interaction (documenting, not fixing here). Eager edge ingest (build(edge_extractor=...)) calls decompose for every artifact each build, so combining with_synopsis with edge_extractor= re-runs synthesis every rebuild (synthesis lives in decompose). The common case — synopsis routing with no edge_extractor — stays fully incremental (the AC). I'll document the caveat; decoupling edge extraction from full decompose is a separate build() refactor I can file if wanted.

   Proceeding on this.
3. thorwhalen

   added a commit that references this issue on Jun 13, 2026

   with_synopsis — LLM-derived synopsis surfaces as collapsed-tree routi…

   ...

   746d859
4. thorwhalen

   closed this as completedin #55on Jun 13, 2026
5. thorwhalen commented on Jun 13, 2026

   

   thorwhalen

   on Jun 13, 2026

   MemberAuthor

   More actions

   Shipped in ir 0.1.22 (squash-merge 746d859, PR #55).

   ir.with_synopsis(strategy, *, synthesize=None) adds one LLM-derived synopsis surface per artifact (prepended → the collapsed-tree router); make_llm_synthesizer is the lazy-oa default (import ir stays offline). Staleness reuses the ledger — index._strategy_id now recurses into nested strategies, folding inner-strategy params + synthesizer identity into strategy_id, so a model/prompt/inner change re-synthesizes only the affected artifacts.

   Adversarial review (12 agents, every finding verified) caught two should-fixes — both contradicting docstrings shipped in the PR — fixed before merge:

   1. text_key alignment — the default synthesizer summarized text_of(raw) with no text_key, so with_synopsis(Chunked(text_key='body')) summarized the wrong field. Now threads the inner strategy's text_key.
   2. non-identifiable synthesizer — an unnamed lambda / local closure aliased to a shared <lambda>/<locals> qualname → swapping it was silently stale. Now warns and uses a sentinel id (named functions + explicit/stamped ids still track).

   Plus 4 test-hardening nits (Package router precedence, mutation-resistant offline guarantee, default-id content-stability, non-str guard, file-backed round-trip). 365 tests, lint/format clean.

   This completes the cap-1 graph arc (#46 links → #47 traverse → #48 synopsis). Next per ADR #43 sequencing: budget governor + run-log (raglab), then PurposeStore.