Skip to content

fix(review): bound impact-map query cache retention#4992

Open
JSONbored wants to merge 1 commit into
mainfrom
codex/fix-unbounded-impact-map-query-cache
Open

fix(review): bound impact-map query cache retention#4992
JSONbored wants to merge 1 commit into
mainfrom
codex/fix-unbounded-impact-map-query-cache

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • Prevent unbounded storage growth from attacker-controlled, per-file impact-map query fingerprints by ensuring expired cache rows do not accumulate.
  • Remove retaining large retrieved context blobs in the cache since computeImpactMap only needs metrics.paths for rendering.
  • Make repo-scoped eviction efficient via an index so periodic cleanup operations can target stale rows without full-table scans.

Description

  • Evict stale rows on cache reads and proactively delete rows older than the TTL before inserting a fresh result by adding a cutoff helper and DELETE ... WHERE fetched_at < ? in putCachedImpactMapQuery and deleting the expired single-row on stale getCachedImpactMapQuery misses in src/review/impact-map.ts.
  • Stop storing the retrieved context body for impact-map entries and write an empty context payload while preserving metrics_json in putCachedImpactMapQuery so per-entry storage cost is minimized.
  • Add a (project, repo, fetched_at) index in the raw migration and Drizzle schema so repo-scoped stale-row deletes are efficient (migrations/0132_impact_map_query_cache.sql, src/db/schema.ts).
  • Extend the cachingStorageStub and add two regression unit tests that assert expired rows are evicted and cached rows keep only metrics (no context) in test/unit/impact-map.test.ts.
  • Regenerate worker runtime types to satisfy the cf-typegen check (worker-configuration.d.ts).

Testing

  • Ran the focused unit tests with npx vitest run test/unit/impact-map.test.ts and all impact-map tests passed (24 tests passed).
  • Verified schema and type drift with npm run db:schema-drift:check, npm run cf-typegen:check, npm run typecheck, and npm run db:migrations:check, all of which succeeded locally.
  • Attempted the full gate npm run test:ci; early checks completed but the full suite did not finish cleanly in this environment (long-running unrelated queue coverage runs surfaced and the run was not completed here).
  • npm audit --audit-level=moderate could not complete in this environment due to the npm audit endpoint returning 403 Forbidden.

Codex Task

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored closed this Jul 11, 2026
@JSONbored JSONbored reopened this Jul 11, 2026
@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.31%. Comparing base (7bbf529) to head (891d7f1).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4992   +/-   ##
=======================================
  Coverage   94.31%   94.31%           
=======================================
  Files         454      454           
  Lines       39034    39038    +4     
  Branches    14234    14234           
=======================================
+ Hits        36813    36817    +4     
  Misses       1572     1572           
  Partials      649      649           
Flag Coverage Δ
shard-1 46.91% <33.33%> (+<0.01%) ⬆️
shard-2 33.23% <0.00%> (-0.01%) ⬇️
shard-3 31.36% <0.00%> (-0.01%) ⬇️
shard-4 33.05% <0.00%> (-0.01%) ⬇️
shard-5 33.54% <100.00%> (+<0.01%) ⬆️
shard-6 45.23% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/db/schema.ts 73.07% <ø> (ø)
src/review/impact-map.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gittensory-orb gittensory-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 11, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 11, 2026

Copy link
Copy Markdown

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - fixes required

Review updated: 2026-07-12 01:10:35 UTC

5 files · 1 AI reviewer · 1 blocker · readiness 100/100 · CI failing · dirty

🛑 Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.

Review summary
This PR bounds unbounded growth of impact_map_query_cache by evicting expired rows on read and proactively deleting stale rows scoped to (project, repo) before each insert, adds a matching index in both the migration and Drizzle schema, and stops persisting the retrieved context blob (keeping only metrics_json). The core logic is sound and the two new regression tests genuinely exercise computeImpactMap end-to-end rather than fabricating state. The one real problem is that the index is added by editing the already-existing migrations/0132_impact_map_query_cache.sql file in place rather than via a new migration.

Blockers

  • migrations/0132_impact_map_query_cache.sql:17-18 — this file already exists (the diff shows it as modified, not added) and its CREATE TABLE predates this PR per the 'fix(review): impact-map re-embeds + re-queries the vector index up to 20x per pass with no result cache #4500' comment already in the file, meaning it is very likely already applied to production D1; D1's migration ledger tracks applied state by filename, so appending `CREATE INDEX ...` to an already-applied file silently no-ops on existing deployments and only takes effect on fresh DBs, causing schema drift — add the index in a new file (e.g. migrations/0133_impact_map_query_cache_index.sql) instead of editing 0132.
Nits — 5 non-blocking
  • src/review/impact-map.ts:118 — the new `DELETE ... WHERE fetched_at < ?` runs on every cache write (not just periodically), which is fine given the new index but is an extra round-trip per put; consider whether this should be probabilistic/periodic instead of unconditional if write volume is high.
  • The PR doesn't cite an issue number it closes; per repo convention, contributor PRs should link the eligible open issue this addresses (the `fix(review): impact-map re-embeds + re-queries the vector index up to 20x per pass with no result cache #4500` reference is only a pre-existing code comment for the original cache table, not a link for this change).
  • worker-configuration.d.ts carries a large unrelated regenerated diff (workerd bump, email/workflow API type changes) — expected from `cf-typegen` but worth calling out explicitly in the PR description as pure generated churn so reviewers don't need to diff it by hand.
  • test/unit/impact-map.test.ts's cachingStorageStub DELETE-matching regex logic is a bit intricate (matchesExactRow vs matchesExpiredRepoRow) — a short comment explaining the two DELETE call shapes it's simulating would help future maintainers.
  • Split the index addition into its own migration file per the schema-migrations convention so it can't be mistaken for editing an applied migration.

Why this is blocked

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.

CI checks failing

  • validate
  • validate-code
  • validate-tests-merge
  • validate
  • validate-tests (5)
  • validate-tests (1)
  • validate-tests (3)
  • validate-tests (4)
  • validate-tests (6)
  • validate-tests (2)
  • validate-code
  • build + boot smoke test
Signal Result Evidence
Code review ❌ 1 blocker 1 reviewer
Linked issue ✅ No-issue rationale PR body explains why no issue is linked.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 45 registered-repo PR(s), 37 merged, 415 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 45 PR(s), 415 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, Ruby, Go, JavaScript, MDX, Shell, Solidity
  • Official Gittensor activity: 45 PR(s), 415 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@gittensory-orb gittensory-orb Bot added the manual-review Gittensor contributor context label Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aardvark codex gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant