Roadmap: codedb capability gaps & improvements (from the #528 audit)

[justrach]

Tracking issue for forward-looking improvements surfaced by the multi-agent capability audit run alongside #528 (CLI hardening) and #530 (audit bug fixes).

This is a roadmap / epic, not a defect — so it intentionally departs from the "every issue ships a failing test" rule (that's for bugs). Per-item, test-backed issues will be split out when each is picked up.

🥇 Top picks — highest leverage, mostly reusing what already exists

1. Call-path queries + PageRank ranking — impact: high, effort: M
   codegraph.buildEdges already computes the full call graph but discards the edges, keeping only in-degree. Retain them to unlock codedb_callpath A→B (shortest call chain — the Bug: edit.zig advisory lock not released on error paths #1 follow-up after finding a bug) and PageRank-over-imports as a much stronger ranking signal. Near-free given the substrate exists; pure-Zig, low-RSS. Suggested first pickup.

2. format=json output + per-result provenance — impact: high, effort: L
   All tool output is prose; errors are bare string prefixes with no machine-readable code; results carry no confidence, no "which index tier produced this," no "this file was past the 15k-trigram cap (recall incomplete)." Add a JSON output mode + response metadata so agents can threshold on results and tell "not found" from "not indexed."

3. Prefix / fuzzy / kind-filtered symbol search — impact: high, effort: S
   codedb_symbol is exact-name only. The symbol index is already a StringHashMap; add a sorted name list + reuse the existing Smith-Waterman scorer → symbol parse_*, kind=interface, "all *Manager". (Also fixes a Go symbol-kind parse bug.)

4. Git-churn / recency fused into ranking — impact: high, effort: M
   git.zig is ~31 LOC and ranking is 100% static-structural — zero temporal signal. Shell out to git log for churn + map diffs→symbols → codedb_git, "what changed in this PR", churn-boosted ranking.

5. Token-budget-aware context packing — impact: high, effort: M
   "Token-efficient" is a stated core value, yet every truncation is by lines/bytes, never model tokens (e.g. codedb_context silently cuts symbol bodies at 40 lines). Add max_tokens + a value-density knapsack packer: "give me the best 8k tokens for this task."

🥈 Bigger strategic bets (audit blind-spots & competitive gaps)

• Whole-repo orientation / cold-start map — no "README for a stranger"; codedb_context needs keywords you may not have yet. A PageRank-ranked codedb_overview (Aider/Cursor repo-map style).
• Type-resolved find-references — codedb_callers is whole-word text search → false positives for get/init. A lightweight receiver_type::method qualifier index.
• Build/compiler-diagnostic ingestion — nothing consumes zig build/tsc/cargo check output to map errors onto symbols. "Show the symbols implicated by current build errors" is a top agent task and is absent.
• Also unmodeled: structural/AST-pattern search (ast-grep/Comby-class), semantic/embedding retrieval, package-manifest graph (package.json/Cargo.toml/go.mod), config↔code linkage ("where is this key read"), and session working-memory (cross-call de-dup).

🥉 Smaller cleanups (grouped)

• Parsing: doc-comment extraction into Symbol.detail; Python async def/decorated symbols dropped; wrong Go symbol kinds; no Makefile/Dockerfile; no shebang/content detection.
• Edit/refactor: symbol-rename, move/delete, atomic multi-file edit groups, synchronous post-edit validation, dry_run diff for str_replace.
• Indexing: OS-native fs events (kqueue/inotify/FSEvents) vs polling; nested per-dir .gitignore; snapshot FORMAT_VERSION validated/migrated on load; content-based generated/vendored/minified exclusion.
• Security/observability: HTTP serve has zero auth; per-tool error-rate/p99 metrics; update signature verification beyond SHA-256.

---

Generated from a 9-agent capability audit (7 dimension maps → blind-spot/competitive critic → prioritized synthesis).

[justrach]

Status sweep (2026-06-11) — checking the top picks against what has actually shipped:

1. Call-path queries + PageRank — ✅ done. callpath is a CLI command and MCP tool, the resolved call graph is retained on the Explorer (edges + adjacency), and PageRank (damping 0.85) is the default centrality signal in ranked search (CODEDB_IN_DEGREE_CENTRALITY falls back to in-degree). The roadmap's premise that buildEdges discards edges is no longer true. PR feat(#546,#550): defines-first tier0 candidates + call-graph distance ranking #608 additionally folds a query-specific graph-distance signal into ranking.
2. format=json + provenance — ✅ largely done. codedb_search and codedb_symbol take format=json with provenance/meta; machine-readable error objects exist on that path.
3. Prefix / fuzzy / kind-filtered symbol search — ✅ done. codedb_symbol supports exact, prefix=, glob pattern=, kind=, and fuzzy=true.
4. Git-churn / recency in ranking — 🟡 partial. codedb_changes exists (MCP + CLI bridge, cli: codedb_changes has no CLI command — 'codedb changes' parses as a root directory #578), but no temporal signal is fused into ranking. The ranking half is tracked concretely as Enhancement: query-specific retrieval signals (call-graph distance to matched symbols; git co-change) #550's git co-change signal.
5. Token-budget-aware context packing — ❌ not started (no max_tokens anywhere; truncation is still lines/bytes).

The 🥈 strategic bets and 🥉 cleanups are otherwise unstarted. Keeping this open as the backlog tracker — picks 4 (via #550) and 5 are the live candidates.

[justrach]

Closing — the 🥇 top picks are now 5/5 shipped:

1. Call-path queries + PageRank ranking — callpath CLI command + MCP tool over the retained call graph; PageRank (damping 0.85, 20 iters) is the default centrality signal in ranked search, in-degree behind CODEDB_IN_DEGREE_CENTRALITY. Extended in PR feat(#546,#550): defines-first tier0 candidates + call-graph distance ranking #608 with a query-specific call-graph distance multiplier.
2. format=json + provenance — codedb_search / codedb_symbol take format=json with provenance meta and machine-readable error objects.
3. Prefix / fuzzy / kind-filtered symbol search — codedb_symbol supports exact, prefix=, glob pattern=, kind=, fuzzy=true.
4. Git-churn / recency fused into ranking — PR feat(#550): git co-change ranking signal #609: git log --name-only parsed into a bounded per-file co-change map, folded into both ranking paths as a ≥1 multiplier seeded by the queried symbol's defining files (CODEDB_NO_COCHANGE opts out). codedb's ranking is no longer git-history-blind; codedb_changes (cli: codedb_changes has no CLI command — 'codedb changes' parses as a root directory #578) covers the diff-surfacing half.
5. Token-budget-aware context packing — PR feat(#531): max_tokens two-step packing for codedb_context #610: codedb_context takes max_tokens; sections render first, then are admitted by value order under the budget and emitted in document order, with one-line markers for whatever was omitted.

The 🥈 strategic bets (whole-repo orientation map, type-resolved find-references, diagnostic ingestion, AST-pattern search, …) and 🥉 cleanups (incl. the multi-class file-role item that moved here from #550) remain a wishlist. Per this issue's own protocol — "per-item, test-backed issues will be split out when each is picked up" — they should become focused issues at pickup time rather than holding the tracker open; this closed issue stays linkable as the source list.

Top picks ship with 0.2.5825 (suite 814/814).