Skip to content

The decision monitor: replace the deleted briefing product, tasks 026-033 - #25

Merged
sam-aydlette merged 11 commits into
mainfrom
estate/026-phase0-hygiene
Sep 24, 2026
Merged

sam-aydlette merged 11 commits into
mainfrom
estate/026-phase0-hygiene

Conversation

@sam-aydlette

Copy link
Copy Markdown
Owner

The decision monitor: backlog tasks 026 to 033, plus a first real run

Replaces the old briefing product (deleted 2026-08-31) with a decision monitor: pre-registered
watches, deterministic routing, one model call per candidate pair, and a brief that is a
deterministic view over the result. docs/PLAN.md is the plan; each backlog/0NN-*.md carries
the task's acceptance, out-of-scope, landmines, the decisions taken while building it, and the
review findings applied against it. make check is green at every commit (770 tests on the
last one).

commit what
45c1c56 (026) keychain credentials, audited model client, secret scanning -- the API key moves to the OS keychain with no fallback; every model request is written to a local audit log before it is sent; detect-secrets in pre-commit and CI
8e0eb1a (027) sweep -- delete what has no caller; fold estate into insightweaver; drop the REPL
a091a80 (028) ingest and route -- every configured source through one store path with bounded concurrency; deterministic word-boundary routing to watches; the unrouted report
dc7909a (029) adjudicate -- one structured Claude call per routed pair, every outcome recorded, nothing retried; replay over routed pairs
58c1e9e (030) belief and resolution -- an append-only belief ledger, grading by hand, retirement instead of deletion
b372b52 (029 follow-up) failure classification, so an API outage leaves pairs pending rather than marking them permanently failed
0cac270 (031) brief -- header, MOVED, DUE, WATCHING, QUIET as a derived view, deterministic to the byte; run; a golden end-to-end test with a frozen clock
1dac7cc (032) onboarding -- the /onboard skill: an interview that transcribes, never authors
eebd83d (033) docs -- README, GETTING_STARTED, CONCEPTS, CONTRIBUTING and the skills rewritten from the tree; tests/test_docs.py fails when a document names a command, option, make target or path that does not exist; make db-init for a fresh database
0df9f44 feeds -- a duplicate source entry found during the first real run
cfe3d1e brief -- compact source header by default (--verbose for the full list), added after the first real run's header buried the sections worth reading daily under ~70 mostly-successful source lines

Validated by a real run, not just fixtures

After 033, the branch was onboarded with real Position/watch data (via /onboard) and run
end to end: ingest against 72 configured sources, route, a real adjudicate call against
the live API, and brief. Routing and adjudication behaved as designed -- two items that
matched a broad trigger were correctly judged not-evidence by the model, including one case
where a name match was accurate but the underlying claim wasn't. One adjudication's stated
rationale was found to misdescribe the text it read even though its verdict was probably still
right; recorded as a live finding, not fixed here, since it's model behavior rather than a bug
in this codebase.

Decisions a reviewer may want to overturn

  • DUE looks 30 days ahead (HORIZON_DAYS); the plan gives no number.
  • A Position that cannot be read does not stop the brief; the header, DUE and the banner say so.
  • run chains ingest, route, adjudicate, brief; syncing watches is a separate step.
  • The header prints the last fetch attempt, labelled as a failure when it was one, and folds
    answered sources into a count by default; the source row keeps no separate time of the last
    success.
  • Principle 3 (frame visibility) is stated in the README as not implemented.

Not yet done

  • The three-week run the plan ends on has not started; task 034 is written from its numbers.

🤖 Generated with Claude Code

…anning

Backlog task 026, the first task of docs/ESTATE_PLAN.md. Establishes three
properties every later phase relies on, and nothing else:

- Secrets have one home. The Anthropic key is read from the OS keychain
  through keyring when a model client is built, never at import and never
  from .env, with no environment fallback. `estate auth` is the only write
  path and never prints a value.
- Model calls have one door and it is logged. src/llm/audit.py writes each
  request before it is sent and the usage or error after. The client returns
  usage, chooses its model by role (Sonnet 5 triage, Opus 5 synthesis), and
  passes effort through output_config on the 1.x SDK.
- A leaked key cannot be committed. detect-secrets runs in pre-commit and in
  the CI security job, which could not fail before this.

Also: the plan itself with its decisions recorded; backlog 019/020/023
superseded by the laptop plan, 018 rewritten for it, 015/016 rescoped into
Phase 1d, 022 updated; CLAUDE.md's through-line paragraph moved from
Questions to nodes; types-PyYAML added so make typecheck passes in a fresh
venv; requirements recompiled (drops dev pins nothing declared or imported).
…n for the decision monitor

Backlog task 027. 8,200 lines removed, none of them reachable from a
console script, a migration, or a live module. Each candidate was
adversarially reviewed by an agent told to find a live caller, a build
step, a CI step, a packaging entry or a standing commitment; none was
found, and three independent hunters then found the second tier this
list had missed (historical drop migrations, unread settings fields, an
unused palette, a dev helper in production code, four dependencies
nothing imports). backlog/027-sweep.md records every deletion and the
two overridden "keep" notes (deduplicator, coverage_probe) with reasons.

Also: the estate console script and src/estate/ fold into insightweaver
(auth is a subcommand; credentials live in src/config/); the REPL is
gone; FeedMatcher only loads config/feeds/; requirements recompiled.

docs/PLAN.md replaces docs/ESTATE_PLAN.md. The estate graph and its
sensors are deferred with reasons; what remains is the decision monitor.
Tasks 028 to 031 are queued from it. The reachability test the sweep
built found fourteen modules unreachable from any command -- the whole
ingest, routing and model stack -- which is the finding this plan exists
to close; it lands with task 029.
…erministic routing to watches

Backlog task 028 (tasks 015 rescoped). The repository has had no way to
grow its corpus since the legacy write path closed on 2026-08-31; now
`insightweaver ingest` builds an adapter for every feed in config/feeds/
(RSS included) and runs them with bounded concurrency through the one
store path, so a feed added to config reaches the database by
construction.

Tier 1: a watch's trigger clauses compile through the entity matcher's
word-boundary and shouted-term rules (AND across a clause's fields, OR
within a field, any clause fires, first index recorded) into a `routes`
table, idempotent per pair, no model call -- proved by a test that
blocks the import of src.llm while routing runs. The regression gate
plants twenty whole-word matches and a hundred look-alikes in a
thousand observations and checks the routed rows by identity; with the
boundary anchors removed, 61 route instead of 20.

`route --dry-run` reports per watch, and reports the unrouted count by
source and by near-duplicate cluster, bounded so a rebuild cannot hand
the O(n^2) grouper the whole corpus. The task file records what the
adversarial review found and what was deferred.
…recorded, nothing retried

Backlog task 029 (task 016 rescoped). The single stochastic component: a
ClaudeAdjudicator registered as prompt version claude-v1, its prompt,
effort, caps and response schema fingerprinted and pinned by a test so a
changed prompt cannot keep the old name. The schema is derived from the
pydantic verdict model with the range keywords the grammar rejects
removed; pydantic enforces the ranges on receipt.

`insightweaver adjudicate` asks about every routed pair without an
answer under this version, commits each answer as it is recorded, and
records a refusal, a truncated reply or an invalid verdict as a failed
row with its audit id and cost, never retried. A missing key or a
request the API rejects aborts the run instead of becoming verdicts.
`--dry-run` prints the exact text and never builds a client.

The client is synchronous now, raises a typed failure that carries the
audit id and usage, and takes the structured-output schema as a request
parameter. Replay reads exactly the routed pairs, bounds by routed
observations, and refuses an unrouted corpus. The reachability test the
sweep built lands here: every module under src/ now has a caller.

The task file records what the adversarial review found: three blockers
fixed before commit, and what was recorded rather than changed.
…ement instead of deletion

Backlog task 030 (tasks 017 and 021 cut to what has data behind it). A
watch_beliefs table holds every belief value with its source and date --
'file' when the watch file set it at sync, 'principal' when the operator
wrote it with `watch believe` -- and refuses UPDATE and DELETE from the
ORM and from a sqlite3 shell alike. watches.belief is the registration
value and is never overwritten; a file that is behind the operator's own
updates shows as a change with a source. There is no update rule: belief
moves only when the operator moves it.

`watch resolve` grades a watch once, by hand, with no unresolve and no
path from evidence. sync retires a watch that leaves the file and
restores one that returns; it never deletes a row with history on it.
Liveness has one definition: routing links open, unexpired watches;
adjudication and replay skip retired and resolved ones and keep expired
ones, because expiry is routing's gate and a date filter in replay would
make it depend on the day it ran.

models.py became a package, one module per table family, mechanically.
The task file records the review's findings and the fixes.
… instead of failed

A connection error, a timeout, a 429 or a 5xx was recorded per pair as a
verdict; one bad hour would have marked every pending pair failed with no
answer given. ModelCallFailed.outcome now says answered, rejected or
unavailable, and only an answered failure is recorded. Non-API exceptions are
audited and re-raised. A max_tokens cut during thinking is named as such.
replay --commit refuses --limit, since commit would delete every row outside
the bound. The prompt fingerprint includes the model's own schema so a bound
enforced on receipt moves the version.
…rministic to the byte, and run

select_brief reads evidence, watch state, routing recency and source state
into plain data, pure in (db, as_of, since, position); render prints it in
terminal or markdown with every citation carrying its observation hash and
prompt version. Every heading prints with a zero when empty. A briefs row
per render sets the next default window to the last brief before today, so
a second run the same morning reproduces the first. run chains ingest,
route, adjudicate and brief and stops at the first failure.

After review: staleness is tested on routes.routed_at, not the item's date;
evidence for a watch that expired or was resolved in the window is shown
with its state; a failed fetch is printed as a failed attempt, never as a
fetch, through one record_attempt stamp; routed pairs no adjudication has
answered are counted in QUIET; a 200 that is not a feed is an error in the
RSS adapter; beliefs are read as of the brief's moment; delivery precedes
the briefs row; a Position error reaches the header whole.

Golden end-to-end test over the recorded Federal Register week, a reprinting
feed, an empty feed and an unreachable one, with a scripted model client and
a frozen clock. make golden regenerates it.
…to the private files and syncs them

The operator answers, the skill writes, the loaders validate and watch sync
is the only write path; it proposes no decision, belief, claim or trigger
(invariant 6). Task 024's interview half is done this way; its review loop
stays parked until there are resolved watches to review.
…rewritten from the tree, with a test that they name only what exists

Every sentence written from the code as of 2026-09-23 and reviewed against
it: what each command writes, what the audit log holds, what the observation
hash contains, what sync does to a belief the operator moved, what a rebuild
does to QUIET's clock. The five principles keep their headings; the third
says it is not implemented. The README shows the golden brief whole.

tests/test_docs.py fails when a document names a command, option, make
target or repository path the tree lacks; tests/test_invariants.py pins that
nothing in src sends, serves or listens and that only replay deletes
evidence. Writing the guide found that nothing created the schema on a fresh
database: make db-init (src/database/migrations/create_schema.py) does. The
example watches file named a source that matches nothing; .env.example gained
DATABASE_URL; pyproject's URLs point at the real repository.
Configured twice, once in geographic/states/virginia.json and once in
domains/civic_engagement.json with a slightly different URL (no www.),
which is why ensure_source's exact-URL dedup never caught it. Kept the
civic_engagement.json copy: higher relevance_score and tags (legislation,
general_assembly, watchdog) that match state-politics watches better.
A first real run against 72 configured sources buried the four sections
worth reading daily under ~70 lines of mostly-successful source status.
A source that answered, whatever it returned, is now folded into a
summary count (configured/fetched/failed/never fetched). A source whose
last attempt failed is still printed in full, every time, with no flag
needed -- invariant 5 (silence distinguishable from breakage) is about
failures being visible, not about every source having its own line, and
the failed ones are exactly what the operator needs to act on.

--verbose restores the original always-list-everything header.

Golden files regenerated (make golden) and the README's embedded brief
excerpt resynced to match. backlog/031-brief.md gets a dated addendum
rather than an edit to its original ACCEPTANCE/LANDMINES, since those
record what was actually reviewed at the time.

Also fixed a pre-existing mypy gap in tests/brief/test_render.py (two
dict-splat test helpers untyped enough to fail the pre-commit mypy hook,
which unlike 'make typecheck' does check tests/) hit while touching the
same file for the header change.
@sam-aydlette
sam-aydlette merged commit f7db333 into main Sep 24, 2026
5 checks passed
@sam-aydlette
sam-aydlette deleted the estate/026-phase0-hygiene branch September 24, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant