Skip to content

close the legacy article write path - #23

Merged
sam-aydlette merged 2 commits into
mainfrom
task/025-legacy-write-path
Aug 31, 2026
Merged

sam-aydlette merged 2 commits into
mainfrom
task/025-legacy-write-path

Conversation

@sam-aydlette

Copy link
Copy Markdown
Owner

Implements backlog/025, which unblocks 015. 628 tests, make check exit 0.

Task 014 made observations authoritative and established that exactly one code path constructs an Article — src/sources/store.py:store_items, which writes the article and its observation in one transaction. RSSFetcher.fetch_and_store_feed bypassed it.

Shape chosen: raise, not route

fetch_and_store_feed now raises LegacyWritePathClosed and stores nothing.

Routing it through store_items would have meant rebuilding RawItems from its dicts — reimplementing RSSAdapter inline — leaving a duplicate ingestion path to be held in step with the real one forever, for a caller that does not exist. Raising removes the capability rather than the symptom.

The function keeps its name and signature deliberately, so any caller outside this machine fails loudly at the call site with a message naming the replacement, rather than getting an ImportError.

The catch that made this better than specified

The IC also closed fetch_all_active_feeds, for a reason worth reading:

_rate_limited_fetch catches every per-feed exception into a summary dict, so a raise underneath would have returned normally with "0 of 44 feeds succeeded" and a zero exit code — the exact silent failure the task exists to remove.

A fix that only closed the inner function would have produced a corpus refresh that reported success while ingesting nothing.

Proof the enforcement bites

Verified two ways by the IC and once more by me with my own probe module:

new module doing db.add(Article(...)):
  AssertionError: these modules construct an Article directly: ['rss/_probe.py']
probe removed:
  1 passed

And the legacy entry point, run rather than described:

before: articles=55249 observations=0
fetch_and_store_feed   -> LegacyWritePathClosed
fetch_all_active_feeds -> LegacyWritePathClosed
after:  articles=55249 observations=0

A hole I closed rather than documented

The IC found src/processors/normalizer.py::ArticleStorage is a latent second write path: it db.add()s Article objects it is handed, so it never constructs one and the grep cannot see it. It argued the path is inert — nothing under src/ can produce an Article to pass it — and wrote that argument into the test's docstring. Scoping it out was defensible.

The argument is correct and the shape is still wrong. A documented hole in an invariant is a hole, and this one rests on a second fact — that nothing can produce an Article — which is precisely what this task just made true and which a later task could quietly undo.

The whole module turned out to be dead: neither ArticleStorage nor ArticleNormalizer has a production caller, only tests keeping them alive. Both deleted, with their tests. 657 → 628.

grep 'Article(' src/ --exclude store.py --exclude models.py
  (nothing)

The invariant is now structural rather than argued.

Context worth recording

While checking for external callers I found insightweaver-daily.timer, a live systemd user timer firing nightly. It runs scripts/scheduled_report.py, deleted long ago in ab085dd — it has failed on every fire for at least 14 days with No such file or directory, and nothing surfaced that.

It never reaches Python, so it is not a caller of the legacy path and is out of scope here. It is the same failure class the rewrite exists to fix, running on the operator's own machine, and it belongs to task 020's deployment work.

Residual, disclosed not fixed

src/sources/rss_adapter.py's module docstring still says the live pipeline fetches through fetch_all_active_feeds "exactly as before", which is now false. That file is explicitly out of scope for this task. One-line correction, worth doing with 015.

RSSFetcher.fetch_and_store_feed inserted Article rows directly, with no
Observation beside them. Task 014 made observations the authoritative record
and Tier 1 routing reads them, so an article stored that way can never route
and nothing surfaces the gap: the corpus grows, routing finds nothing, and the
hole looks like a source problem rather than an ingestion one.

It now raises LegacyWritePathClosed rather than routing through store_items.
The work it did is already done correctly by src/sources/rss_adapter.py, which
reads the same feed through the same normalizer and stores through
store_items -- article and observation in one transaction. Rewiring this second
entry point would have kept a duplicate ingestion path that must be held in
step with the adapter path forever, for no caller. Reading is untouched:
fetch_feed and normalize_article are what the adapter uses.

fetch_all_active_feeds is closed the same way, at the top rather than
underneath: _rate_limited_fetch catches every per-feed exception into a summary
dict, so a raise further down would return normally with zero articles and a
zero exit code -- the same silent nothing-happened.

Task 014's one-write-path test gains the matching half for Article, sharing a
helper with the Observation check. Construction is the chokepoint asserted, not
session.add: nothing under src/ can insert an unobserved article without first
constructing one, and only store_items may. Verified to bite by reintroducing a
second path twice (a new module, and the old write back inside fetcher.py) and
watching it fail both times.

Tests drive the real entry points against a real session and assert the
refusal, that articles and observations both stay at zero, and that the feed
row is untouched -- no last_fetched stamp and no error_count bump toward
auto-deactivation at ten.
The IC found that ArticleStorage.store_articles is a latent second write
path: it db.add()s Article objects it is handed, so it never constructs one
and the grep test cannot see it. It argued the path is inert because nothing
under src/ can produce an Article to pass it, and wrote that argument into
the test's docstring.

The argument is correct and the shape is still wrong. A documented hole in an
invariant is a hole, and it relies on a second fact -- that nothing can
produce an Article -- which is exactly what this task just finished making
true and which a future task could quietly undo.

The whole module is dead: neither ArticleStorage nor ArticleNormalizer has a
production caller, only tests keeping them alive. Deleted both, and their
tests. 657 -> 628 tests.

grep for Article( outside store.py and models.py now returns nothing, so the
one-write-path invariant is structural rather than argued.
@sam-aydlette
sam-aydlette merged commit 23da2fe into main Aug 31, 2026
5 checks passed
@sam-aydlette
sam-aydlette deleted the task/025-legacy-write-path branch August 31, 2026 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant