Repository navigation
close the legacy article write path - #23
Merged
Merged
Conversation
RSSFetcher.fetch_and_store_feed inserted Article rows directly, with no Observation beside them. Task 014 made observations the authoritative record and Tier 1 routing reads them, so an article stored that way can never route and nothing surfaces the gap: the corpus grows, routing finds nothing, and the hole looks like a source problem rather than an ingestion one. It now raises LegacyWritePathClosed rather than routing through store_items. The work it did is already done correctly by src/sources/rss_adapter.py, which reads the same feed through the same normalizer and stores through store_items -- article and observation in one transaction. Rewiring this second entry point would have kept a duplicate ingestion path that must be held in step with the adapter path forever, for no caller. Reading is untouched: fetch_feed and normalize_article are what the adapter uses. fetch_all_active_feeds is closed the same way, at the top rather than underneath: _rate_limited_fetch catches every per-feed exception into a summary dict, so a raise further down would return normally with zero articles and a zero exit code -- the same silent nothing-happened. Task 014's one-write-path test gains the matching half for Article, sharing a helper with the Observation check. Construction is the chokepoint asserted, not session.add: nothing under src/ can insert an unobserved article without first constructing one, and only store_items may. Verified to bite by reintroducing a second path twice (a new module, and the old write back inside fetcher.py) and watching it fail both times. Tests drive the real entry points against a real session and assert the refusal, that articles and observations both stay at zero, and that the feed row is untouched -- no last_fetched stamp and no error_count bump toward auto-deactivation at ten.
The IC found that ArticleStorage.store_articles is a latent second write path: it db.add()s Article objects it is handed, so it never constructs one and the grep test cannot see it. It argued the path is inert because nothing under src/ can produce an Article to pass it, and wrote that argument into the test's docstring. The argument is correct and the shape is still wrong. A documented hole in an invariant is a hole, and it relies on a second fact -- that nothing can produce an Article -- which is exactly what this task just finished making true and which a future task could quietly undo. The whole module is dead: neither ArticleStorage nor ArticleNormalizer has a production caller, only tests keeping them alive. Deleted both, and their tests. 657 -> 628 tests. grep for Article( outside store.py and models.py now returns nothing, so the one-write-path invariant is structural rather than argued.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
backlog/025, which unblocks 015. 628 tests,make checkexit 0.Task 014 made observations authoritative and established that exactly one code path constructs an
Article—src/sources/store.py:store_items, which writes the article and its observation in one transaction.RSSFetcher.fetch_and_store_feedbypassed it.Shape chosen: raise, not route
fetch_and_store_feednow raisesLegacyWritePathClosedand stores nothing.Routing it through
store_itemswould have meant rebuildingRawItems from its dicts — reimplementingRSSAdapterinline — leaving a duplicate ingestion path to be held in step with the real one forever, for a caller that does not exist. Raising removes the capability rather than the symptom.The function keeps its name and signature deliberately, so any caller outside this machine fails loudly at the call site with a message naming the replacement, rather than getting an
ImportError.The catch that made this better than specified
The IC also closed
fetch_all_active_feeds, for a reason worth reading:A fix that only closed the inner function would have produced a corpus refresh that reported success while ingesting nothing.
Proof the enforcement bites
Verified two ways by the IC and once more by me with my own probe module:
And the legacy entry point, run rather than described:
A hole I closed rather than documented
The IC found
src/processors/normalizer.py::ArticleStorageis a latent second write path: itdb.add()sArticleobjects it is handed, so it never constructs one and the grep cannot see it. It argued the path is inert — nothing undersrc/can produce anArticleto pass it — and wrote that argument into the test's docstring. Scoping it out was defensible.The argument is correct and the shape is still wrong. A documented hole in an invariant is a hole, and this one rests on a second fact — that nothing can produce an
Article— which is precisely what this task just made true and which a later task could quietly undo.The whole module turned out to be dead: neither
ArticleStoragenorArticleNormalizerhas a production caller, only tests keeping them alive. Both deleted, with their tests. 657 → 628.The invariant is now structural rather than argued.
Context worth recording
While checking for external callers I found
insightweaver-daily.timer, a live systemd user timer firing nightly. It runsscripts/scheduled_report.py, deleted long ago inab085dd— it has failed on every fire for at least 14 days withNo such file or directory, and nothing surfaced that.It never reaches Python, so it is not a caller of the legacy path and is out of scope here. It is the same failure class the rewrite exists to fix, running on the operator's own machine, and it belongs to task 020's deployment work.
Residual, disclosed not fixed
src/sources/rss_adapter.py's module docstring still says the live pipeline fetches throughfetch_all_active_feeds"exactly as before", which is now false. That file is explicitly out of scope for this task. One-line correction, worth doing with 015.