Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ config/api_sources.json
# Philosophical documentation (not for version control)
docs/philosophical_docs/

# Tier 1's coverage-gap report (backlog task 015, 2026-08-31). Derived: it is
# rewritten from the observations on every `insightweaver route` run, and it
# quotes the operator's corpus. Task 021 reads it from disk, not from git.
data/routing/

# Database files
*.db
*.db-journal
Expand Down
10 changes: 9 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: help install install-dev test lint format fmt typecheck check pre-commit clean clean-all coverage db-add-watches db-add-observations db-drop-briefing update-deps
.PHONY: help install install-dev test lint format fmt typecheck check pre-commit clean clean-all coverage db-add-watches db-add-observations db-add-routes db-drop-briefing update-deps

# Tool resolution (added 2026-08-24).
# Before this, every recipe called bare `pytest` / `ruff` / `mypy`, which only
Expand Down Expand Up @@ -46,6 +46,7 @@ help:
@echo "Database:"
@echo " make db-add-watches Create the watches table (additive, safe)"
@echo " make db-add-observations Create observations and evidence (additive, safe)"
@echo " make db-add-routes Create the route_candidates table (additive, safe)"
@echo " make db-drop-briefing Drop the deleted briefing product's tables"
@echo " (DESTRUCTIVE; captures to a dump first)"
@echo ""
Expand Down Expand Up @@ -127,6 +128,13 @@ db-add-watches:
db-add-observations:
$(PYTHON) -m src.database.migrations.add_observations_and_evidence

# Additive. Creates Tier 1's output table. The unique constraint on
# (observation_hash, watch_id) it carries is what makes routing idempotent, so
# the table is created from the model rather than from hand-written DDL.
# (2026-08-31, backlog task 015.)
db-add-routes:
$(PYTHON) -m src.database.migrations.add_route_candidates

# Deliberately does NOT pass --confirm. The migration refuses without it, so
# running this target prints what it would destroy and stops; the operator types
# the real command themselves. A Makefile target that drops tables on `make` is
Expand Down
57 changes: 57 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,63 @@ Create the tables with `make db-add-observations` (additive; it does not touch

---

## Tier 1: deterministic routing

Added 2026-08-31 (`backlog/015`). Adjudication is the only tier that calls a
model, and it costs money per item. Tier 1 is what decides which items it ever
sees, so **notification volume scales with the number of live watches rather
than with news volume** -- and that is a property of the code in `src/routing/`,
not an aspiration.

**Triggers compile; they are not interpreted and not read by a model.** A
watch's `triggers` is a list of clauses over `terms`, `entities` and `sources`.
Within a clause every populated field must match (AND) and any value in a field
will do (OR); across clauses, any clause firing fires the watch. That compiles
directly into two regex alternations and a set-membership test. Nothing in
`src/routing/` imports an Anthropic client, and the test that says so removes it
from the interpreter entirely and drives the whole path
(`tests/routing/test_no_model.py`).

**Word boundaries are load-bearing.** Every pattern comes from
`src/matching/entity_matcher.py`'s `compile_terms`, which owns the anchors and
the shouted-acronym case rule -- there is deliberately no second matcher. The
scale being defended, measured on this repository's own 55,249-article corpus:
`nist` appears as a substring in 5,364 titles and at a word boundary in 73;
`mail` is 1,842 against 339.

**Routing is idempotent.** A match becomes a `route_candidates` row, unique on
`(observation_hash, watch_id)`; routing the same observation twice produces one
link. The row also records which clause fired, which is the difference between
"this watch routed 400 observations" and "clause 2 of this watch routed 400".

```bash
# Per watch, how many of the last 500 observations would route -- plus the
# unrouted count and its clusters. Writes no route_candidates rows.
insightweaver route --dry-run

# The same, over every stored observation, recording the links.
insightweaver route --limit 0
```

**Read the unrouted number first.** A high one is the healthy state. A low one
means a trigger is too loose, and a trigger that is too loose does not fail --
it bills. The unrouted observations are clustered by their stored MinHash
signatures and written to `data/routing/unrouted_clusters.json` alongside a
document-frequency histogram of their salient terms. That file is the
coverage-gap signal: the only place a *missing* watch is visible before a
staleness alert fires. `backlog/021` reads it.

**The ceiling test.** `tests/routing/test_ceiling.py` asserts that 1,000
observations against 5 watches route fewer than 20. The number was measured, not
guessed: the corpus is 1,000 real articles sampled from the pre-rewrite archive
and stratified by feed category, the measured baseline is 5 routed / 5 links,
and a clause whose AND became an OR routes 33. The test's docstring records the
measurement and is explicit about what it cannot see.

Create the table with `make db-add-routes` (additive).

---

## Requirements

- Python 3.10+
Expand Down
42 changes: 42 additions & 0 deletions backlog/015-tier1-deterministic-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,45 @@ rather than news volume -- is actually enforced. Tiers 2 through 4 can only pres
in production metrics is one that regresses between releases and is noticed at the invoice. A test
asserting that N observations against M watches routes fewer than K makes the property a gate.
Pick K from a measured baseline, not from intuition, and record how it was measured.

---

## Done 2026-08-31 (branch `task/015-routing`)

Built `src/routing/` (predicate, router, salience, gaps), `src/cli/route.py`,
the `route_candidates` table and its migration. `make check` exits 0 and
`env -u ANTHROPIC_API_KEY pytest tests/ -q` exits 0 (730 passed).

**K = 20, measured, not guessed.** The ceiling fixture is 1,000 *real* articles
lifted from the 55,249-row pre-rewrite archive through a read-only
(`immutable=1`) SQLite handle, stratified 400/600 by feed category (not by
trigger word -- that would have been circular) and stored through the real
`store_observation` path. Measured baseline: 5 of 1,000 route, 5 links, fan-out
0.005. A clause whose AND became an OR routes 33; dropping the term/entity
constraints routes 1,000. K = 20 sits between. Full derivation, including what
the ceiling does *not* gate, is in `tests/routing/test_ceiling.py`'s docstring.
Nothing was written to any database that outlives the measurement; this is not a
backfill and must not be read as one.

**Word boundaries were verified by deletion, not by assertion.** With
`LEFT_BOUNDARY`/`RIGHT_BOUNDARY` emptied, `TestWordBoundariesAreLoadBearing`
goes 15 failed / 3 passed; restored, 18 passed. Every case in that class uses a
non-shouted term (`nist`, `mail`, `ai`, `rev 5`), so the case rule cannot reject
them first -- which is the trap task 010 fell into.

### Two things for you, neither of them blocking

1. **`sources:` is matched by exact equality, and that is a live question.** A
clause saying `sources: [Federal Register]` matches a feed *named* exactly
that. The corpus has three: "Federal Register", "Federal Register - Public
Inspection", "Federal Register - Documents API", and only the last carries
articles -- so `config/watches.example.yaml`'s own source clause matches
nothing here. Equality was chosen because it can only under-route, and
under-routing surfaces in the unrouted clusters rather than on an invoice.
If you meant the family of feeds, say so and it becomes a prefix or a
category match.
2. **Trigger `entities:` currently match only the surface form you wrote.** The
alias registry that used to supply `CoverageEntity.aliases` died with the
beat files in task 012 and nothing replaced it. So `entities: [FedRAMP PMO]`
does not also match "FedRAMP Program Management Office". The one place that
changes is `src.routing.predicate._entity_terms`.
12 changes: 11 additions & 1 deletion src/cli/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
from .colors import accent, header, muted
from .output import set_debug_mode
from .replay import replay_command
from .route import route_command
from .sources import sources_command
from .watch import watch_command

Expand All @@ -33,14 +34,15 @@
"sources": sources_command,
"watch": watch_command,
"replay": replay_command,
"route": route_command,
}


def print_command_refresher():
"""Print a short refresher of available commands."""
click.echo(
f"\n{header('Commands:')} {accent('sources')} | {accent('watch')} | "
f"{accent('replay')} | help | exit\n"
f"{accent('route')} | {accent('replay')} | help | exit\n"
)


Expand All @@ -51,6 +53,9 @@ def print_help():
click.echo(
f" {accent('watch')} - Pre-registered watches and the decisions they serve"
)
click.echo(
f" {accent('route')} - Match Observations to Watches, deterministically"
)
click.echo(
f" {accent('replay')} - Rebuild Evidence from Observations and diff it"
)
Expand All @@ -72,6 +77,11 @@ def print_help():
muted(" Watches are only ever created from those files -- there is no 'watch add'.")
)
click.echo()
click.echo(header("Route command:"))
click.echo(f" {accent('route --dry-run')} - Per watch, how many of the last N route")
click.echo(muted(" Reports the unrouted count and its clusters -- the coverage-gap signal."))
click.echo(muted(" No model is involved anywhere in this tier."))
click.echo()
click.echo(header("Replay command:"))
click.echo(f" {accent('replay --prompt-version X')} - Rebuild Evidence for prompt version X")
click.echo(
Expand Down
197 changes: 197 additions & 0 deletions src/cli/route.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
"""
Route command - which observations are candidate evidence for which watch.

``--dry-run`` is the reporting mode and it writes no ``route_candidates`` rows.
It answers the question this tier exists to make answerable: of the last N
observations, how many would each watch send to the adjudicator, which clause
of which watch is doing the sending, and -- the part that is easy to leave out
and is the whole coverage-gap signal -- how many matched nothing, and what were
they about.

**Read the unrouted number first.** A high one is the expected, healthy state:
most of what a feed publishes is not evidence about anything you have
pre-registered a claim on, and if it were, the architecture would not be doing
its job. A *low* one means a trigger is too loose, and a trigger that is too
loose does not fail -- it bills, because the adjudicator is the only tier that
calls a model and it runs on whatever this one hands it.

The coverage-gap report is written to disk in both modes, because it is a report
derived from the corpus rather than a change to it, and because the mode you
would run to find a gap is the dry run. Its path is printed. Backlog task 021
reads it.

Added 2026-08-31 for backlog task 015.
"""

from datetime import date

import click

from ..database.connection import get_db
from ..database.models import Observation, Watch
from ..routing import (
TriggerCompileError,
compile_watches,
gap_report,
persist,
route,
write_gap_report,
)
from .colors import accent, header, muted, warning

# Why a default at all: near-duplicate clustering of the unrouted set is
# pairwise, so it is quadratic in the number of observations that matched
# nothing -- which is most of them. 500 is a few days of ingestion and runs in
# about a second. --limit 0 means every stored observation and is the honest
# spelling of "I know what I am asking for".
_DEFAULT_LIMIT = 500


def _clause_lines(compiled, per_clause) -> list[str]:
"""One line per trigger clause that fired, so a wide trigger is attributable."""
lines = []
for watch in compiled:
for clause in watch.clauses:
count = per_clause.get((watch.watch_id, clause.index), 0)
if count:
lines.append(f" clause {clause.index}: {count:>6} {clause.describe()}")
return lines


@click.command(name="route")
@click.option(
"--dry-run",
is_flag=True,
default=False,
help="Report what would route and write no route_candidates rows.",
)
@click.option(
"--limit",
type=int,
default=_DEFAULT_LIMIT,
show_default=True,
help="Consider the last N observations. 0 means all of them.",
)
@click.option(
"--gaps-out",
type=click.Path(dir_okay=False),
default=None,
help="Where to write the unrouted cluster report. Defaults to data/routing/.",
)
@click.option(
"--no-gaps",
is_flag=True,
default=False,
help="Skip the unrouted clustering. Faster, and loses the coverage-gap signal.",
)
def route_command(dry_run, limit, gaps_out, no_gaps):
"""Route observations to watches with compiled, deterministic predicates."""
today = date.today()

with get_db() as session:
stored = session.query(Observation).count()
if stored == 0:
click.echo(
warning(
"No observations are stored, so there is nothing to route. Tier 1 reads "
"'observations' and never 'articles'; the 55,249 legacy rows are the "
"pre-rewrite archive. See src/database/models.py."
)
)
return

watch_rows = session.query(Watch).all()
if not watch_rows:
click.echo(
warning(
"No watches are stored. Routing with no watches routes nothing, which is "
"correct and useless. Run 'watch sync' first."
)
)
return

try:
report = route(session, limit=limit or None, today=today)
compiled = compile_watches(
[row for row in watch_rows if row.expires is None or row.expires >= today]
)
except TriggerCompileError as exc:
raise click.ClickException(str(exc))

click.echo(header("ROUTE" + (" (dry run)" if dry_run else "")))
click.echo(
muted(
f"{report.considered} of {stored} stored observation(s), newest first, "
f"against {len(report.watch_ids)} live watch(es)."
)
)
if report.expired_watch_ids:
click.echo(
muted(f" skipped {len(report.expired_watch_ids)} expired: ")
+ muted(", ".join(report.expired_watch_ids))
)
click.echo("=" * 80)

for watch_id in report.watch_ids:
count = report.per_watch[watch_id]
share = count / report.considered if report.considered else 0.0
click.echo(f" {accent(watch_id):<48} {count:>6} ({share:6.2%})")
click.echo()
for line in _clause_lines(compiled, report.per_clause):
click.echo(muted(line))

click.echo()
click.echo(
f" {'routed':<48} {report.routed_count:>6} "
f"({report.routed_count / report.considered:6.2%} of considered)"
)
click.echo(
f" {'unrouted':<48} {report.unrouted_count:>6} "
f"({report.unrouted_count / report.considered:6.2%} of considered)"
)
click.echo(
muted(
f" {len(report.links)} link(s), fan-out {report.fan_out:.2f} "
f"adjudications per observation considered."
)
)

if not no_gaps:
payload = gap_report(session, report)
path = write_gap_report(payload, gaps_out)
clusters = payload["clusters"]
multi = [c for c in clusters if c["size"] > 1]
click.echo()
click.echo(header("UNROUTED CLUSTERS"))
click.echo(
muted(
f"{len(clusters)} cluster(s), {len(multi)} with more than one member. "
f"A large one is a story the environment thought mattered and no watch "
f"can see."
)
)
for cluster in clusters[:10]:
terms = ", ".join(term for term, _ in cluster["terms"][:5])
click.echo(f" {cluster['size']:>4} {cluster['representative_title'][:60]}")
click.echo(muted(f" {terms}"))
if payload["gap_terms"]:
top = ", ".join(f"{t} ({n})" for t, n in payload["gap_terms"][:12])
click.echo()
click.echo(muted(f" gap terms: {top}"))
click.echo()
click.echo(muted(f" written to {path}"))

click.echo()
if dry_run:
click.echo(
muted("Nothing was written to route_candidates. Re-run without ")
+ accent("--dry-run")
+ muted(" to record these links.")
)
return

written = persist(session, report)
click.echo(
f"routed: {written['inserted']} link(s) inserted, "
f"{written['already_linked']} already present"
)
Loading
Loading