Skip to content

tier 1: deterministic routing - #24

Closed
sam-aydlette wants to merge 2 commits into
mainfrom
task/015-routing
Closed

sam-aydlette wants to merge 2 commits into
mainfrom
task/015-routing

Conversation

@sam-aydlette

Copy link
Copy Markdown
Owner

Implements backlog/015. 730 tests (628 → 730), make check exit 0.

Tier 1 is where the rewrite's central property is enforced: notification volume scales with live watches, not news volume. A predicate that is too loose does not fail — it bills, because Tier 2 is the only LLM tier and runs on whatever this hands it.

K = 20, measured

1,000 observations built from real article text lifted from the frozen articles archive through a read-only immutable=1 handle, stratified by feed category rather than by trigger word — stratifying by trigger word would define the corpus by the answer. Committed as a fixture so the test asserts against exactly what K was measured on.

Five deliberate regressions place K:

fan-out
baseline 0.0050
word boundaries deleted 0.0050
shouted case rule deleted 0.0050
both deleted 0.0060
clause AND → OR 0.0370
term/entity constraints dropped 5.0000

K = 20 is 4× baseline and below the cheapest regression it can see. The reviewer independently reproduced all five numbers by monkeypatching and re-running the shipped fixture, rather than reading the table.

The author disclosed that this test cannot see its most expensive landmine

Deleting the anchors moves the ceiling by zero. That was verified against the shipped test, not just the harness, and written into the test's docstring rather than left in a report:

predicate tests, anchors stripped:  15 failed
ceiling test,    anchors stripped:  10 passed

Every surface form in a realistic compliance watch set is either a shouted acronym (saved by the case rule) or a phrase with no common host word. The division: boundaries are gated by the predicate tests, AND/OR and match-all by the ceiling. A ceiling test presented as covering boundary regressions would have been a false guarantee, and nobody would have checked.

Boundary tests avoid the trap task 010 fell into

Every case uses a non-shouted term (nist, mail, ai, rev 5) compiled case-insensitively, so the case rule cannot reject a collision before the anchors run. That is exactly how task 010's boundary class came to pass with anchoring deleted from the codebase.

Verified by stripping the anchors: 15 fail, restored 18 pass. git diff on entity_matcher.py shows the anchors untouched in the commit.

One matcher, not two

entity_matcher was refactored to expose compile_terms() / matches_any(), and routing calls them. The reviewer confirmed reuse by diff rather than docstring claim. Same for payload_text() — one definition of "the text of an observation".

No LLM is reachable, proved with a control

A subprocess installs a meta-path hook that raises on any anthropic or src.llm import before src loads, drops the API key, and drives ingestion → compile → route → persist → cluster → write. A control test asserts the hook actually blocks — without it, the primary test proves only that code ran. A second class parses the routing package's ASTs, because a grep is satisfied by a comment saying "imports no model".

Idempotent, run not described

RUN 1: 5 inserted, 0 already present
RUN 2: 0 inserted, 5 already present
RUN 3: 0 inserted, 5 already present

Plus a test inserting a duplicate around persist and asserting the DB raises. The pre-read makes the report honest; the unique constraint makes the table honest.

Two semantics surfaced rather than assumed

  • sources: matches feed name by exact equality. Chosen because it can only under-route — which surfaces as an unrouted cluster — rather than over-route, which bills. If the operator means the family, it becomes a prefix match.
  • entities: match only the surface form written. The alias registry died with the beat files in task 012, so FedRAMP PMO does not match FedRAMP Program Management Office. Single change point: _entity_terms.

Both are in the task file with their reasoning, not silently decided.

A finding worth more than the code

All three shipped example watches route zero against the entire 55,249-article corpus. fedramp appears as a substring in 5 articles; cmmc in 6.

That is not a routing defect — it is the coverage problem in a new place, and it is exactly what the unrouted report exists to surface. Real run over 1,000 observations: 987 clusters, top gap terms security (180), trump (114), virginia (96), federal (90).

The old briefing showed you something regardless of coverage. This architecture, working perfectly, shows you nothing. That is an argument for treating the Position repo and its watches as the real gate on v0.

Also in this PR

src/sources/rss_adapter.py's docstring claimed the live pipeline "still fetches feeds through fetch_all_active_feeds exactly as before". That stopped being true in task 025. Flagged by 025's IC, which was correctly forbidden from touching the file; corrected here because 015 is what makes the claim consequential — this adapter is now the only way an RSS article enters the corpus.

sam-aydlette and others added 2 commits August 31, 2026 15:17
Adjudication is the only tier that calls a model and it costs money per item.
Tier 1 decides what it ever sees, so "notification volume scales with live
watches rather than news volume" is a property of src/routing/, not a slogan.
A predicate that is slightly too loose does not raise, does not log and does
not fail a test that only checks it matches what it should. It bills.

src/routing/predicate.py
  Compiles a watch's triggers -- a disjunction of conjunctions over terms,
  entities and a source allowlist -- into two regex alternations and a set
  membership test. Every pattern comes from entity_matcher.compile_terms,
  extracted from compile_entities for this task so there is one matcher and
  one set of anchors rather than two. Sources match by equality, not
  substring: "Federal Register" must not be satisfied by "Federal Register
  Watch Blog".

  It refuses rather than skips. A clause that constrains nothing matches the
  whole corpus; a clause whose declared field is all blanks silently loses a
  conjunct and becomes broader than it reads. The YAML loader already rejects
  both, but it is not the only way JSON reaches watches.triggers -- a
  hand-edited row, a restored dump, a migration written in a hurry.

src/routing/router.py, route_candidates
  route() computes and writes nothing; persist() writes. The dry run is not a
  mode of the writer. A match becomes one row unique on
  (observation_hash, watch_id), so routing the same observation twice produces
  one link -- pre-read for an honest report, unique constraint because two
  concurrent runs would both pass the pre-read. The row records which clause
  fired: "watch X routed 400" is a bill, "clause 2 of watch X routed 400" is a
  diff. Expired watches are skipped and named, because "nothing routed" and
  "every watch expired last week" must not look the same.

src/routing/gaps.py, salience.py
  The unrouted count is the coverage-gap signal -- the only place a missing
  sensor is visible before a staleness alert fires, and a bare integer makes it
  useless. Unrouted observations are grouped by their stored MinHash
  signatures (the same grouping the rest of the system uses, not a new
  algorithm) and labelled with the words they share; a document-frequency
  histogram catches the subject covered by forty different stories no two of
  which are near-duplicates. Written to data/routing/unrouted_clusters.json
  under a schema string. Task 021 reads it. This module proposes nothing --
  invariant 6 is lost one convenient path at a time.

insightweaver route [--dry-run]
  Per watch and per clause, how many of the last N observations route; then the
  routed and unrouted counts, the fan-out in adjudications per observation, the
  clusters, and where the report went.

WORD BOUNDARIES, VERIFIED BY DELETION
  Task 010 found its own boundary tests passing for the wrong reason: every
  case used a shouted acronym, so the case rule rejected the collision before
  the anchors were consulted, and the class passed with LEFT_BOUNDARY and
  RIGHT_BOUNDARY deleted. Every case in TestWordBoundariesAreLoadBearing uses a
  NON-shouted term -- nist, mail, ai, rev 5 -- compiled case-insensitively, so
  the anchors are the only thing that can reject them. Emptying both anchors:
  15 failed, 3 passed. Restored: 18 passed. The scale being defended, measured
  on this repository's corpus: nist is a substring of 5,364 titles and a
  boundary match in 73; the whole-text substring count for opm is 1,706,
  almost all of it the word "development", against 14.

THE CEILING, AND HOW K WAS MEASURED
  tests/routing/test_ceiling.py: 1,000 observations against 5 watches route
  fewer than 20.

  The corpus is REAL article text. 1,000 rows were lifted from the 55,249-row
  pre-rewrite archive through a read-only (immutable=1) SQLite handle,
  stratified 400/600 by feed category -- deliberately not by trigger word,
  which would have defined the corpus by the answer -- and selected by BLAKE2b
  of (id, guid) so the sample does not depend on row order or a PRNG. The test
  stores them through the real store_observation path, so the observations it
  routes have real hashes, real payloads and real signatures. Nothing was
  written back to any database; this was a measurement, not a backfill.

  Measured baseline: 5 of 1,000 route, 5 links, fan-out 0.005. Every widening
  regression was then measured against the same fixture: boundaries deleted, 5;
  case rule deleted, 5; both, 6; clause AND became OR, 33; term and entity
  constraints dropped, 1,000. K = 20 sits at four times the baseline and below
  the cheapest regression it can see.

  It cannot see the boundary regressions, and the docstring says so rather than
  implying otherwise -- every surface form in a realistic compliance watch set
  is either a shouted acronym or a phrase with no common host word. Boundaries
  are gated by the predicate tests above.

  A separate assertion pins the measured baseline exactly. The ceiling is the
  gate; the baseline is the tripwire.

NO MODEL, PROVED TWO WAYS
  A subprocess installs a meta-path hook that raises on any import of anthropic
  or src.llm before a line of src is loaded, drops ANTHROPIC_API_KEY, and drives
  the whole path: ingestion, compilation, routing, persistence, clustering and
  writing the report. A control asserts the hook actually blocks, without which
  the first test proves only that the code ran. A second class parses the
  routing package's ASTs -- a grep would be satisfied by a comment claiming no
  model is imported.

Two live questions are written into the task file rather than assumed away: a
sources clause matches a feed name exactly (so the shipped example's "Federal
Register" clause matches none of this corpus's three Federal Register feeds),
and trigger entities match only the surface form written, because the alias
registry died with the beat files in task 012.

make check exits 0; env -u ANTHROPIC_API_KEY pytest tests/ -q exits 0.
730 tests, 89 of them new.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M9gY2miAUXijbFm25YnFcq
It said the live pipeline still fetches through fetch_all_active_feeds
'exactly as before'. That stopped being true in task 025, which closed both
that function and fetch_and_store_feed because they wrote articles with no
observation and Tier 1 routing cannot see those.

Flagged by 025's IC, which was correctly forbidden from touching this file.
Corrected here because 015 is the task that makes the claim consequential:
this adapter is now the only way an RSS article enters the corpus.
@sam-aydlette

Copy link
Copy Markdown
Owner Author

Superseded. This implemented backlog/015 under the tier-based architecture (grid-axis triggers, estate route naming), which was rescoped away on 2026-09-22 in favor of the decision monitor (docs/PLAN.md). Deterministic routing was rebuilt from scratch as backlog/028 (src/routing/route.py) and lands in #25. Closing without merging; branch task/015-routing left in place in case anything in its diff is worth mining, but it should not be merged as-is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant