fix(cycle): cache empty propose_takes scans#1218
Open
AdityaRajeshGadgil wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
propose_takesusedtake_proposalsas both the operator-facing queue and the per-page idempotency cache. That works only when the extractor produces at least one proposal. The valid result[]left no row behind, so unchanged pages could re-spend extractor tokens every autopilot cycle.This PR splits those two jobs:
take_proposal_page_scansas the successful extractor-call cache, keyed by(source_id, page_slug, content_hash, prompt_version)and written even for[]take_proposalsas proposal rows only, with dedup moved to(source_id, page_slug, content_hash, prompt_version, claim_text)so same-page multi-claim output is not collapseddryRunintopropose_takesso dry runs do not call the LLM or write proposal/cache rowsTest Coverage
bun test ./test/propose-takes.test.ts ./test/propose-takes-cache-schema.test.tsbun test ./test/schema-bootstrap-coverage.test.ts ./test/propose-takes.test.ts ./test/propose-takes-cache-schema.test.tsbun run typecheckbun run check:allbun run buildgit diff --cached --checkbun run ci:local:diffcould not run in this environment because Docker is not installed (docker: command not found). The diff-aware gate reached Docker startup before failing.Pre-Landing Review
Two independent review passes were run before opening this PR. The first review found two blockers — wrong-shape JSON could still be cached as
[], and the new cache table needed explicit RLS coverage — plus an insert-count nit. This version fixes all three:[]remains a valid cacheable empty resulttake_proposal_page_scanson Postgres, with a PGLite override for the local engineRETURNING id, soproposals_insertedreflects rows actually inserted afterON CONFLICT DO NOTHINGThe second review passed with no blocking findings.