Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions backlog/012-archive-the-briefing-product.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Delete the briefing product -- beats, questions, predictions, frames, synthesis, rendering -- porting only five modules whose mechanics the new tiers need.
REPO: InsightWeaver
STATUS: QUEUED # QUEUED | IN_PROGRESS | PARKED | DONE | FAILED
SIZE: large
ACCEPTANCE: `make check` passes, plus: roughly 8,900 lines of source are **deleted, not archived** -- `src/render/`, `src/prompts/`, `src/config/beats.py`, the beat/question/prediction/standing-agenda/institutional-activity/frame/synthesis modules in `src/context/`, and every CLI command except what the new pipeline needs; their tests are deleted with them; **`src/render/email.py`'s `smtplib` send path goes with the rest**, closing the standing violation of invariant 1; the five modules named below are **moved and kept**, with their tests, because the new tiers need their mechanics; a migration drops the tables belonging to deleted concepts; `insightweaver --help` lists only surviving commands; and the deletion lands as **one commit** with a message naming what went and why, so `git log` is the record rather than an `archive/` directory.
OUT OF SCOPE: An `archive/` directory. Superseded on 2026-08-31: the operator's call is that git history is the rollback path and a directory of dead code in the tree is a liability, not documentation. Deleting `src/sources/`, `SOURCES.md`, `src/database/connection.py`, `src/config/feed_matcher.py`, `src/config/settings.py`, `src/rss/`, or the five ported modules. Building any part of the new pipeline. Dropping the `articles` table -- that is task 014's decision.
LANDMINES: **Five modules encode findings that cost real debugging and must be ported, not rewritten.** `src/context/entity_matcher.py` (168) carries word-boundary matching: across the live corpus, 5,364 titles contain the substring `nist` and 73 match at a word boundary, and its boundary test was found to pass with the anchors deleted from the codebase because case-sensitivity rejected the collisions first -- a rewrite reintroduces both. `src/context/claude_client.py` (212) carries the retired-model outage of 2026-06-15 and the `ThinkingBlock` parsing failure, each of which cost a live run. `src/context/coverage_probe.py` (488) is a "did anything match this in N days" engine, which is what task 018's staleness check is. `src/utils/cadence.py` (117) is interval parsing for task 013. `src/processors/deduplicator.py` (433) is an explicit keep. **Do not delete a module because its concept died if its mechanics are load-bearing downstream** -- name each ported module in the PR with the task that needs it. **The database still holds 55,249 articles, 46 questions, 33 predictions and 42 frames**; dropping tables is destructive and irreversible against that data, so back up before the migration and say in the PR that you did. `main` is protected -- open a PR.
---
Rewritten 2026-08-31 after the operator's decision: beats and questions are removed, Position
replaces them, and the old code is deleted rather than archived because git already holds it.

## What goes

| area | lines |
|---|---|
| `src/context/` -- synthesis, topics, curation, frames, questions, predictions, standing agenda, beat scope, institutional activity, decision routing | ~3,900 |
| `src/cli/` -- brief, frames, diet, questions, predictions, forecast, decisions, beat, stake, scope | ~3,000 |
| `src/render/` -- terminal, markdown, html, email, document | 1,834 |
| `src/config/beats.py` | 665 |
| `src/prompts/` | 508 |

Plus most of 844 tests. This is a deletion of roughly the whole product built to date.

## What is ported, and why it is not sentiment

Five modules whose concepts die and whose mechanics the new tiers need:

- **`entity_matcher.py` -> Tier 1 routing.** Word boundaries are the difference between 73 and
5,364 matches on one term in this corpus. Its test class was also found to pass with anchoring
removed, because shouted terms were rejected by case-sensitivity before the anchors ran. Both
findings are encoded here and a blank-file rewrite loses them.
- **`claude_client.py` -> Tier 2.** Encodes a retired model that failed silently for ten weeks and
a response-shape change that discarded a completed synthesis.
- **`coverage_probe.py` -> task 018.** A staleness check is "did anything match this in N days",
which is what this already is.
- **`cadence.py` -> task 013.** Interval parsing and next-review arithmetic.
- **`deduplicator.py` -> Tier 0.** Explicit keep in the brief.

## Why one commit and no `archive/`

A directory of dead code is read as current by every future reader and every agent, and it drags
the old product's shape forward -- which is the outcome the rewrite exists to avoid. `git log` is
the rollback path, and the commit message is where the reasoning lives.
37 changes: 37 additions & 0 deletions backlog/013-position-and-watch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Introduce Position and Watch as the system's atomic units, with so_what enforced in the schema.
REPO: InsightWeaver
STATUS: QUEUED # QUEUED | IN_PROGRESS | PARKED | DONE | FAILED
SIZE: medium
ACCEPTANCE: `make check` passes, plus: `position.yaml` is loaded from git, hand-authored, and validated -- decisions carry a deadline, and a Position over two pages **warns loudly rather than failing**, naming the drift from stakes into interests; a `watches` table and loader exist carrying `id`, `claim`, `belief`, `so_what`, `triggers`, `expires`, `staleness_alert_days`; **a Watch with an empty, whitespace-only, or absent `so_what` is rejected at load with a clear error and is not stored**, with a test that bites; a Watch whose `so_what` names no decision present in the Position is rejected the same way; `belief` outside 0.0-1.0, an `expires` in the past, and a `staleness_alert_days` under 1 are each rejected; `insightweaver watch list` shows each watch, its belief, its decision, and days to expiry; and no command can create a Watch except from the checked-in files.
OUT OF SCOPE: Routing, adjudication, state transitions, alerts, or notification -- this task defines the units and nothing that acts on them. Migrating existing Questions, Predictions, Decisions or Beats onto the new model; that is task 014's decision to make once Observations exist. Any LLM call. Proposing candidate watches. A web or TUI editor for `position.yaml` -- it is human-edited in git, deliberately.
LANDMINES: **The system never authors its own watches (invariant 6), so every write path here must be closed, not merely unused.** A `watch add` CLI command would satisfy no acceptance criterion and would be the seam through which auto-generation later arrives; do not build one. **`so_what` enforcement must reject rather than default.** Task 011 established this pattern for a different field and its landmine applies verbatim: the ledger accumulated 25 unfalsifiable predictions precisely because a missing field was tolerated. **The existing schema already contains most of this and it is empty.** `decision_factors` has columns `name`, `what_would_update_me`, `current_state_note` -- which is a trigger, a belief state, and a decision link under different names -- and `decisions` and `decision_evidence` both hold **0 rows** as of 2026-08-31. Read `docs/CONCEPTS.md` § "Decisions, Factors, and Evidence" before designing a new table; if Watch is better expressed as an evolution of Decision+Factor than as a parallel structure, say so in the PR rather than building both. `main` is protected -- open a PR.
---
Written 2026-08-31.

## The finding the implementer needs before touching the schema

The brief describes Watch as a new atomic unit. The audit found the repository already models
something very close, and never populated it:

| brief's concept | existing column | rows |
|---|---|---|
| `so_what` naming a decision | `decision_evidence.decision_id` -> `decisions.name` | 0 |
| trigger | `decision_factors.what_would_update_me` | 0 |
| belief state | `decision_factors.current_state_note` | 0 |
| evidence direction | `decision_evidence.direction` | 0 |
| trigger predicate | `predictions.trigger_condition` | 33 rows, 25 unfalsifiable |

That is not an argument for reusing the tables as they stand -- `current_state_note` is prose where
`belief` is a float, and the 33 predictions demonstrate what happens when a trigger field accepts
prose. It **is** an argument that the implementer should decide deliberately between evolving
Decision+Factor and creating Watch alongside it, and record the reasoning. Two overlapping models
for the same idea is the outcome to avoid; the repository already has that problem between
`README.md` and `CLAUDE.md` and this task should not add a third instance of it.

## Why the Position size check warns rather than fails

The brief says a Position over two pages "has drifted into describing interests rather than stakes;
flag that, do not silently accept it." A hard length limit would be enforcing a proxy: a
three-page Position full of genuine deadlines is better than a one-page Position of vague
interests. The check names the drift and lets the human judge, which is the same reason
`coverage_probes` reports INCONCLUSIVE rather than dropping a probe.
27 changes: 27 additions & 0 deletions backlog/014-observations-immutable-and-replayable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Make Observations immutable and content-addressed, and Evidence derived, so the corpus can be replayed against a changed prompt and the diff inspected.
REPO: InsightWeaver
STATUS: QUEUED # QUEUED | IN_PROGRESS | PARKED | DONE | FAILED
SIZE: medium
ACCEPTANCE: `make check` passes, plus: an `observations` table stores normalized adapter output keyed by content hash, with the raw payload retained verbatim and never mutated after write; a write attempting to change a stored observation **fails loudly** rather than updating; adapters in `src/sources/` emit Observations through one path; near-duplicate detection by MinHash groups items that differ only in boilerplate, with the threshold recorded in config and a test showing two real near-duplicates grouping and two distinct items not; an `evidence` table links (observation, watch) with direction and magnitude, and every row records the prompt version that produced it; `insightweaver replay --prompt-version X` rebuilds Evidence from Observations alone and **prints a diff against the stored Evidence without writing**, with `--commit` required to persist; and a test proves replay is deterministic given the same prompt version.
OUT OF SCOPE: The adjudication prompt itself and its content -- this task builds the replay harness, task 016 writes what runs inside it. Routing. State transitions. Deleting or migrating the 55,249 existing `articles` rows; decide whether Observations supersede, wrap, or coexist with that table and record the reasoning, but do not perform a destructive migration in this task. Any change to `SOURCES.md`'s recorded-basis rule.
LANDMINES: **This invariant is the one that makes a stochastic component testable, and it is worthless if replay is not byte-reproducible.** A prompt version that is not recorded per row, or an adapter that normalizes with a timestamp in the payload, silently breaks it -- and it breaks in the direction of looking fine. **`src/sources/base.py` already has `content_hash()` and `ARTICLE_FIELDS`** from task 005; extend that seam rather than introducing a second hashing scheme. **The `articles` table is 55,249 rows and is what every existing query reads**; a parallel `observations` table that only new adapters write leaves the system with two corpora and no rule for which is authoritative. State the rule in the PR. Content-addressing means the hash **is** the identity: an adapter that includes a fetch timestamp or a session ID in the hashed payload will store the same article repeatedly and the dedup will look broken when the hashing is what is wrong. Test that first. `main` is protected -- open a PR.
---
Written 2026-08-31.

Invariant 3 is the one that makes the LLM tier reviewable. Everything downstream of adjudication is
deterministic and unit-testable; adjudication itself is not, and the only way to test it is to hold
the inputs fixed and inspect what a prompt change does to the outputs. That requires Observations
to be immutable and Evidence to be a pure function of (Observations, prompt version).

**Why `--commit` is required rather than default.** A replay that writes by default makes the
before-state unrecoverable at exactly the moment you wanted to compare against it. Printing the
diff and requiring an explicit commit costs one flag and preserves the thing the invariant exists
to provide.

**The open question this task must answer, not assume.** The repository has 55,249 `articles` rows
written by the RSS path and the Federal Register adapter. Observations may supersede that table,
wrap it, or run alongside it during a transition. Each has a real cost -- superseding means a
migration of 55k rows, wrapping means two shapes for one concept, coexisting means a rule nobody
will remember in six months. The implementer picks and writes down why; what is not acceptable is
two corpora with no stated rule, because the audit found exactly that pattern already exists in
this repository between `migrations/` and `src/database/migrations/`.
17 changes: 17 additions & 0 deletions backlog/015-tier1-deterministic-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Route Observations to Watches with compiled deterministic predicates, so the model never sees the bulk of the corpus.
REPO: InsightWeaver
STATUS: QUEUED # QUEUED | IN_PROGRESS | PARKED | DONE | FAILED
SIZE: medium
ACCEPTANCE: `make check` passes, plus: each Watch's triggers compile to a deterministic predicate over entities, terms and source allowlists, with **no LLM call anywhere in this tier** -- proved by a test that removes the Anthropic client and drives the full routing path; word-boundary matching is used throughout and reuses `src/context/entity_matcher.py`'s primitives rather than a second implementation; `insightweaver route --dry-run` reports, per watch, how many of the last N observations would route, and reports the **unrouted count and its clusters**; routing is idempotent -- the same observation routed twice produces one link; a test asserts that a corpus of 1,000 observations against 5 watches routes fewer than a stated ceiling, so a regression that widens routing fails rather than merely costing money; and the unrouted cluster output is written somewhere task 021 can consume it.
OUT OF SCOPE: Adjudication -- routing decides *candidacy*, not whether something is evidence. Any change to what the adjudication prompt says. Proposing watches from unrouted clusters; that is task 021, and this task only has to make the signal available. Tuning the shipped watches' triggers. Belief updates or state transitions.
LANDMINES: **This tier is the cost control, and a routing predicate that is too loose does not fail -- it bills.** The ceiling test exists because "most observations match nothing and die here" is an architectural claim that needs a number attached or it decays silently. **Word boundaries are load-bearing and the repository has already been bitten**: of 55,249 articles, 5,364 contain the substring `nist` and 73 match it at a word boundary; `mail` is 1,842 versus 339. Task 010 also found that a boundary test can pass for the wrong reason -- its shouted-term cases were rejected by case-sensitivity before the anchors were consulted, so the class passed with anchoring deleted from the codebase. Any boundary test written here must be verified by stripping `LEFT_BOUNDARY`/`RIGHT_BOUNDARY` and confirming it fails. **The unrouted count is the coverage-gap signal and is the only place a missing sensor becomes visible before a staleness alert fires**; logging it as a bare integer with no clustering makes it useless. `main` is protected -- open a PR.
---
Written 2026-08-31.

Tier 1 is where the rewrite's central property -- notification volume scaling with live watches
rather than news volume -- is actually enforced. Tiers 2 through 4 can only preserve it.

**Why a ceiling test rather than a dashboard.** An architectural property that is only observable
in production metrics is one that regresses between releases and is noticed at the invoice. A test
asserting that N observations against M watches routes fewer than K makes the property a gate.
Pick K from a measured baseline, not from intuition, and record how it was measured.
21 changes: 21 additions & 0 deletions backlog/016-tier2-adjudication.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Adjudicate routed (observation, watch) pairs in the single LLM tier, with structured output and a versioned prompt.
REPO: InsightWeaver
STATUS: QUEUED # QUEUED | IN_PROGRESS | PARKED | DONE | FAILED
SIZE: medium
ACCEPTANCE: `make check` passes, plus: one prompt, versioned and stored with every Evidence row it produces, runs only on pairs Tier 1 routed; output is structured and validated -- is-evidence, direction, magnitude, satisfies-which-trigger, confidence -- and a response failing validation is **recorded as a failed adjudication rather than silently dropped or retried into a different answer**; the tier is the only place in the codebase that constructs a model request, proved by a test asserting no other module imports the client; `insightweaver adjudicate --dry-run` prints what would be sent and its token estimate without calling; replay from task 014 reproduces Evidence for a given prompt version; and per-run token and cost are recorded, because the repository currently has **no token accounting at all** and a cost-controlled architecture that cannot measure its own cost is asserting the property, not holding it.
OUT OF SCOPE: Belief updates, trigger firing, hysteresis, state transitions -- adjudication reports what an observation *is*, and task 017 decides what it *does*. Batching or Bedrock migration; get the semantics right at one-call-per-pair first and optimise once volume is known. Prompt tuning beyond a first version that passes replay. Multiple prompts or a prompt per watch type.
LANDMINES: **The model has been retired underneath this repository once already.** `claude-sonnet-4-20250514` reached end of life on 2026-06-15 and every synthesis failed for ten weeks while the CLI printed a duration and exited 0, because the exception was caught and logged. Any adjudication path must make an API failure loud at the tier boundary. **The response shape changed too**: `content[0].text` was correct when the model did not think and returns a `ThinkingBlock` now; `src/context/claude_client.py` selects text blocks by type and raises on a refusal -- reuse it, do not reimplement the extraction. **Do not add fallbacks**; `CLAUDE.md` forbids concealing problems behind them, and an adjudication that silently degrades to a default verdict is the worst possible instance -- it writes wrong Evidence that the state machine then acts on. Structured output is a request parameter, not a parsing convention. `main` is protected -- open a PR.
---
Written 2026-08-31.

Invariant 4 puts exactly one stochastic component in the system. That is what makes the other four
tiers unit-testable, and it means this tier's contract has to be narrow enough to validate: a
verdict object, not prose.

**Why a failed adjudication is recorded rather than retried.** A retry that produces a different
answer to the same input breaks task 014's replay determinism, which is the only mechanism for
reviewing a prompt change. Record the failure, leave the pair unadjudicated, and let it be visible.

**Why token accounting lands here.** The audit found the repository has none. Every cost claim in
this architecture -- "Tier 1 is the cost control", "the model must never see the bulk of the
corpus" -- is currently unfalsifiable. This is the tier where a number can be attached.
Loading
Loading