Make dead_code and outline cost follow the scoped repo, not the whole store #1002

Why

analyze kind=dead_code and explore operation=outline (get_repo_outline) take time in proportion to the whole store, not to the repo the session is bound to. The tools benchmark runs every tool against a store holding only gortex, then against a store holding six repos (n=15 per tool):

• dead_code p50: 6.0 s on the gortex-only store, 17.4 s on the six-repo store. Tiny repos in the six-repo store take 17-19 s too.
• outline p50: 4.3 s on the gortex-only store, 12.6 s on the six-repo store (12.6-13.7 s for every repo).

Both are slow-lane tools. One dead_code call or two outline calls keep the slow lane full for 13-19 s, and other slow calls queue behind them. Adding a repo to the store should not make tools slower on a repo that has nothing to do with it.

Current state (main)

dead_code

• internal/mcp/tools_enhancements.go handleFindDeadCode calls analysis.FindDeadCode(..., s.getProcesses(), nil, opts) over the whole store. Only afterwards does it filter rows to the session and repo allow-set. That filter calls reader.GetNode(e.ID) once per entry through analyzeNodeVisible, so it makes one store read per row.
• internal/analysis/deadcode.go FindDeadCode runs three whole-store scans:
  • buildIfaceRequiredMethods, which calls IfaceImplementsRows() and reads every implements edge.
  • collectDeadCodeCandidates, which calls DeadCodeCandidates and reads every candidate node.
  • g.NodesByKind(graph.KindFile), which reads every file node to find entry-point files.
• FindDeadCodeOptions has no repo field, only SkipCrossRepoNodes. DeadCodeEntry carries only ID, name, kind, file path and line.
• The store interfaces DeadCodeCandidator and IfaceImplementsScanner in internal/graph/store.go, and their SQLite versions DeadCodeCandidatesContext and IfaceImplementsRows in internal/graph/store_sqlite/store_analysis.go, take no scope argument. Since commit 4408bb2 these reads are paged and bound to the request context, but they still cover the whole store.

outline

• internal/mcp/tools_outline.go handleGetRepoOutline does the following for a bound session:
  • It loads every scoped node (scopedNodesLight) into an inScope ID map, and computes total nodes and the language breakdown from that list.
  • It walks reader.AllEdges() only to count the edges that have both endpoints in scope.
  • It filters the global getCommunities() and getHotspots() results by membership in inScope.
• mostImportedFiles passes every scoped node ID to FileImportCounts(scope), and falls back to an AllEdges() loop on backends that lack that call. entryPoints runs a global FindNodesByName("main") and then filters it with inScope.
• Per-repo node and edge aggregates already exist and can be reused: Store.RepoStats() in internal/graph/store_sqlite/store.go and Graph.RepoStats() in internal/graph/graph.go.

v0.64.11 is planned to replace the outline's AllEdges() edge count with a SQL count by repo prefix. That change aims for outline ≤ 1 s and leaves the rest of the outline as it is. This issue builds on it.

What to do

dead_code

• Add RepoPrefixes []string to FindDeadCodeOptions. In handleFindDeadCode, fill it from the resolved repo allow-set and the session scope.
• Add scoped store variants:
  • DeadCodeCandidatesContext(ctx, kinds, inEdgeKinds, repoPrefixes), which filters with repo_prefix IN (...) on the candidate side only. Incoming edges from any repo still count as usage, so a symbol whose only callers are in another repo stays alive.
  • IfaceImplementsRows(repoPrefixes), which returns only implementing types in scope but keeps interfaces from any repo.
  • NodesByKindInRepos, used for entry-point files.
  • An in-memory reference implementation of each, plus coverage in the storetest conformance suite.
• Keep only the processes whose entry point is in scope.
• Remove the one-GetNode-per-row post-filter. Carry the repo prefix and workspace on DeadCodeEntry internally, as fields that are not serialized, so the JSON output does not change.

outline

• On the bound-session path:
  • Take total nodes, the per-language breakdown and the edge count from per-repo aggregates instead of scopedNodesLight. Use RepoStats, or add a StatsForRepos(prefixes) that also counts edges with both endpoints in the scoped repos.
  • Add FileImportCountsForRepos(prefixes): a GROUP BY over import edges joined to the target node's repo prefix. It replaces the call that passes every node ID.
  • Find entry points with a repo-scoped query (FindNodesByNameInRepos or equivalent).
  • Use per-repo community and hotspot results when they are cached. Otherwise compute hotspots for the scoped repos only.
• Cache the outline response per (repo-prefix set, graph revision), so repeat calls cost O(1) until the next mutation.
• If you add per-repo counters, they must stay exact across untrack, re-track and retirement. Until a counter is marked ready, fall back to COUNT.

Out of scope

• Making the remaining tools independent of store size (call_chain, explore.context, callers, search and the other AllEdges() call sites in internal/mcp). That work is tracked separately.
• Slow-lane and dispatch changes.
• The empty hotspot results that hotspot consumers return before the first analysis pass.
• Changes to the dead-code heuristics.

Done when

• dead_code p50 for each repo in the six-repo store is ≤ 1.2x the p50 of the same repo on a store holding only that repo. That means gortex ≤ 7.2 s and small repos ≤ 1 s, measured with the tools benchmark (n=15) on the reference host. Current candidate results: all six ratio-of-p50 checks pass; both estimators and the corpus-2 paired-ratio qualification are retained.
• TestDeadCodeScopedRowsIndependentOfOtherRepos exists and passes. It counts the rows returned by the store reads behind a scoped dead_code call (candidates, implements rows and file nodes) and checks that the count for one repo changes by ≤ 5% when a larger unrelated repo is added to the fixture store.
• TestDeadCodeScopedMatchesPostFiltered exists and passes: on a multi-repo fixture, the scoped result equals the old result (whole-store scan, then filter).
• TestDeadCodeCrossRepoCallerKeepsSymbolAlive exists and passes: a symbol whose only caller is in another repo is not reported as dead.
• Uncached outline p50 is ≤ 300 ms for every repo on the six-repo store. A repeat call with no mutation in between is ≤ 50 ms. Current candidate n=15 results: 90 first calls and 90 repeats pass; fresh response cache per daemon start and ordinary readiness before measurement.
• TestOutlineBoundSessionNeverScansAllEdges exists and passes: a counting reader records zero AllEdges and zero AllNodes calls on the bound-session path.
• TestOutlineGoldenSixRepo exists and passes: for the same scope, the output is identical to the current implementation.
• If per-repo counters are added, TestRepoEdgeCounterExactAfterRetrack exists and passes: counts equal a fresh COUNT after untrack, re-track and retirement. Not applicable: this implementation adds no incrementally maintained per-repo counters. It uses exact SQL aggregates and revision-bound prepared summaries; a mutation invalidates the summary and restores the exact query path.
• No regression: unbound-session outline output is byte-identical to the current output, and on a single-repo store the p50 of both tools is ≤ 1.10x the v0.64.10 p50. Final candidate n=15 results and actual-edit cache check: all 45 groups / 135 calls succeeded; all 15 unbound candidate/main bodies match. Bound candidate/tag p50 ratios are 0.000247 (outline) and 0.323155 (dead code).
• go test -race ./internal/analysis/... ./internal/mcp/... ./internal/graph/... passes. CI on current candidate 18c027826 is green: 25 successful checks and one expected skip. The test-selection artifact includes all seven required packages in the Linux race jobs (96 affected packages per OS), with the prescribed SQLite/libc race flags. Local validation also passed 180 affected race tests and the full SQLite suite (3,265 passing records, zero failures, seven documented skips).

How to verify

• Unit and golden tests: go test -race -run 'DeadCode|Outline|RepoEdgeCounter' ./internal/analysis/... ./internal/mcp/... ./internal/graph/...
• Add BenchmarkDeadCodeScopedSixRepo, then run go test -bench DeadCodeScoped -run ^$ ./internal/analysis/....
• Tools benchmark: on the same host, run the per-tool sweep (n=15) against a gortex-only store and then against the six-repo store. Compare dead_code and outline p50 for each repo with the targets above, and with v0.64.10 on the single-repo store.
• Outline cache: call outline twice on the same repo with no edit in between, then once after an edit. The second call should hit the cache and the third should not.

Dependencies

Blocked by (must land first):

• Outline: count in-scope edges and imports in SQL instead of walking every edge in the store #994 — The per-repo outline aggregates extend the hotfix prefix count.

Related issues

• Related: Hotspot consumers besides analyze kind=hotspots return silent empty results before the first analysis pass #849

Planning

• Milestone: v0.64.12
• Priority: latency (first priority)
• Estimate: ~5 engineer-days