Skip to content

delete the briefing product - #20

Merged
sam-aydlette merged 2 commits into
mainfrom
task/012-delete-briefing
Aug 31, 2026
Merged

sam-aydlette merged 2 commits into
mainfrom
task/012-delete-briefing

Conversation

@sam-aydlette

Copy link
Copy Markdown
Owner

Implements backlog/012. The largest change in this repository's history.

89 files removed · 115 changed · +1,156 / −21,979 · tests 1006 → 451, zero skips · make check exit 0

That is most of what this repository has built. It is intended: beats and questions are removed, Position replaces them, and the old code is deleted rather than archived because git already holds it and dead code in the tree reads as current to every future reader.

What went

area lines
src/context/ — synthesis, topics, curation, frames, questions, predictions, standing agenda, beat scope, institutional activity, decision routing 3,948
src/cli/ — brief, frames, diet, questions, predictions, forecast, decisions, beat, stake, scope 2,659
src/render/ — including email.py, closing the standing violation that only an Alert may generate mail 1,834
src/config/beats.py 665
concept migrations 610
src/prompts/ 593
src/pipeline/orchestrator.py 387
tests (36 files) 9,597

models.py went 783 → 116 lines. Nineteen models dropped; RSSFeed and Article kept.

The five modules that had to survive

All registered as git renames at 66–98% similarity — genuine ports, not blank-file rewrites. The reviewer independently confirmed the matching logic is byte-identical where it matters.

module new home needed by
entity_matcher.py src/matching/ Tier 1 routing
coverage_probe.py src/matching/ task 018 staleness
claude_client.py src/llm/ Tier 2 adjudication
cadence.py src/utils/ task 013
deduplicator.py src/processors/ Tier 0

This mattered because two of them encode findings that cost real debugging: word boundaries are the difference between 73 and 5,364 matches on one term in this corpus, and that module's own boundary test was once found to pass with the anchors deleted, because case-sensitivity rejected the collisions first. Verified preserved through the move — NIST matches "NIST guidance", not "administration", not "MINISTRY".

An entanglement the plan did not anticipate: both matchers imported value objects from the deleted beats.py. Those are lifted verbatim into src/matching/terms.py, and run_coverage_probes now takes probes and feed URLs rather than a BeatConfig — which is what a staleness check wants anyway.

The migration

Drops 19 tables. Verified independently:

without --confirm  -> exit 1, 46 questions untouched
with --confirm     -> 19 tables dropped, 619 rows dumped to timestamped .sql
articles           -> 55,249   SURVIVED
rss_feeds          -> 212      SURVIVED

Reversibility is scoped honestly rather than overclaimed: downgrade() recreates the schema empty; only --restore brings rows back from the dump. PROTECTED_TABLES is double-enforced with a runtime assert that fires if the protect and drop lists ever overlap.

Two disclosed deviations

Two commits, not one. The acceptance said one. The second is mine: four hand-authored coverage probes lived in the beat config and went with it — the FedRAMP reinstatement, CISA BOD 22-01, the OPM RIF corrections, the mail-in voting EO. Real verified events, and exactly the shape task 018 needs. They survive in git at 673a8dd, but the loader is gone, so a future implementer would have to know they existed to look. Recovered to config/probes/known-events.json with provenance.

src/cli/sources.py reduced, not deleted. Its entire content was frame-derived. Rather than reimplement against a proxy or invent a metric, those signals were removed and feed inventory kept. Deleting would have widened scope; inventing would have fabricated.

A verification hazard worth recording

The venv's editable install can resolve src to the live checkout, so a deleted module silently imports from the wrong tree instead of failing — a naive import check is a false green. The author caught this and re-verified with the meta-path finder stripped; the reviewer independently confirmed the gate ran against this worktree. Worth knowing for every future task in this repo.

Surviving surface

Commands:
  sources  Configured feeds and the corpus they have contributed.

That is the whole CLI until Tier 0 lands.

Docs are not rewritten — README.md and GETTING_STARTED.md carry a superseded banner rather than a guess, since the new pipeline does not exist yet.

InsightWeaver is being redefined from a daily briefing generator into a
monitoring system for one user's decisions. Beats and questions are removed and
Position replaces them, so the old product is deleted rather than archived: git
history is the rollback path, and a directory of dead code in the tree reads as
current to every future reader and drags the old product's shape forward.

Deleted (89 files, ~20,500 lines; ~10,900 of it source):

  src/render/       1,834  terminal, markdown, html, document, and email.py --
                           whose smtplib send path was a standing violation of
                           the invariant that only an Alert may generate mail
  src/context/      3,948  synthesizer, topic_matcher, curator, frame_manager,
                           cross_cluster_reconciler, question_matcher,
                           prediction_tracker, standing_agenda, beat_scope,
                           institutional_activity, decision_router
  src/cli/          2,659  brief, frames, diet, questions, predictions,
                           forecast, decisions, beat, stake, scope
  src/config/beats.py 665  beat file loading and validation
  src/prompts/        593
  src/pipeline/       387  orchestrator; it ran the briefing pipeline and could
                           not compile without NarrativeSynthesizer/BeatConfig
  src/database/       610  the migrations that created the dropped concepts
  src/utils/          192  base_formatter, used only by the deleted renderer
  config/beats/        88  the one beat definition, now unreadable

plus 9,597 lines of their tests. src/database/models.py drops nineteen models
and keeps two. Test count 1006 -> 451, with no skips added.

Ported, not deleted -- their concepts die, their mechanics are load-bearing:

  src/context/entity_matcher.py  -> src/matching/entity_matcher.py   (Tier 1)
  src/context/coverage_probe.py  -> src/matching/coverage_probe.py   (task 018)
  src/context/claude_client.py   -> src/llm/claude_client.py         (Tier 2)
  src/context/_json.py           -> src/llm/_json.py                 (Tier 2)
  src/utils/cadence.py              stays                            (task 013)
  src/processors/deduplicator.py    stays                            (Tier 0)

Each keeps its tests, and each encodes a finding a blank-file rewrite loses.
entity_matcher carries word-boundary matching: 5,364 titles in this corpus
contain the substring "nist" and 73 match at a word boundary, and its own
boundary test was found to pass with the anchors deleted because case
sensitivity rejected the collisions first. claude_client carries a retired model
that failed silently for ten weeks and a ThinkingBlock response-shape change that
discarded a completed synthesis. coverage_probe is already "did anything match
this in N days", which is what a staleness check is.

The two matchers depended on CoverageEntity/CoverageProbe/BeatConfig from the
deleted beats.py. The two value objects are lifted verbatim into
src/matching/terms.py; run_coverage_probes no longer takes a BeatConfig and
resolves feeds itself -- the caller names the probes and the feed scope, which
is what a staleness check wants anyway.

Also here:

  src/database/migrations/drop_briefing_tables.py drops the nineteen tables
  belonging to the deleted concepts. It refuses without --confirm (no prompt, no
  env var), captures schema and rows to a timestamped dump first, and treats
  articles and rss_feeds as protected with a runtime assert -- articles holds
  55,249 rows and its fate is task 014's decision. downgrade() recreates the
  schema; only --restore brings rows back, and the distinction is deliberate.

  src/cli/sources.py is reduced, not redesigned. Its frame-uniqueness and
  gap-filling signals were computed from article_frames and frame_gaps, both
  deleted. They are removed rather than reimplemented against a proxy: the module
  already refused to report claim survival for want of real inputs, and inventing
  a replacement metric would break that rule in the same file. What remains is
  the feed inventory and stored-article counts, which are measured.

  README.md and GETTING_STARTED.md carry a banner marking them superseded rather
  than a speculative rewrite; the new pipeline does not exist yet.

insightweaver --help now lists one command, sources.
The IC flagged these as collateral: they lived in the beat config and went
with it. They are hand-authored, verified past events in the operator's
domain -- the FedRAMP reinstatement, CISA BOD 22-01, the OPM RIF corrections,
the mail-in voting EO -- and task 018's staleness check needs exactly this
shape.

They are in git at 673a8dd, but the loader for that format is now gone, so a
future implementer would have to know they existed to look. Recovered to
config/probes/known-events.json with their provenance recorded.
@sam-aydlette
sam-aydlette merged commit ed12869 into main Aug 31, 2026
5 checks passed
@sam-aydlette
sam-aydlette deleted the task/012-delete-briefing branch August 31, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant