Add learnings-suggest hook: federated keyword matching & section-level indexing

[ahoym]

Summary

Implements a comprehensive learnings suggestion system that matches user prompts against a federated corpus of learnings, guidelines, and skill references. The system uses dual-layer indexing (section-level preferred, file-level fallback) to inject contextual <learnings-suggestions> hints into prompts, along with telemetry and staleness detection.

Key Changes

• UserPromptSubmit hook (learnings-suggest): Aho-Corasick-based keyword matching against both section-level (from sections.json) and file-level (from .keyword-index.json) indexes. Scores hits by keyword weight, filters by staleness, and respects quoted-term bypass. Injects up to 3 suggestions as a hint block. Never blocks a prompt.

• Section-level indexing (learnings-index-build): Walks the federated learnings corpus, parses Markdown with pulldown-cmark, extracts H2/H3 sections with line ranges, and builds an inverted keyword index. Supports explicit file-level keywords (via **Keywords:** convention) and deduplicates section anchors. Outputs ~/.claude/claude-artifacts/ast/sections.json.

• Read telemetry (learnings-read-log): PostToolUse hook that logs all Read operations on learnings/guidelines/skill-references files to reads.jsonl for offline hit-rate analysis.

• Staleness detection (learnings-staleness.py): SessionStart hook that warns when the keyword index is stale (≥5 files changed since last rebuild), with git-based anchor tracking.

• Analysis tooling (analyze.py): Cross-references suggest.jsonl and reads.jsonl to compute tier hit rates, identify top performers, and surface noise/coverage gaps.

• Build system: Rust Cargo project with platform-specific binaries (aarch64/x86_64, Darwin/Linux), idempotent bootstrap script, and optional cross-compilation via cargo-zigbuild.

• Hook registration: Added to settings.json under hooks.UserPromptSubmit, hooks.PostToolUse, and hooks.SessionStart.

• Setup integration: Updated setup-claude.sh to symlink the hooks/ directory and run the bootstrap build.

Notable Implementation Details

• Dual indexing strategy: Section-level hits are preferred (more precise), but file-level hits supplement them and serve dense files where section extraction may be incomplete. Covered paths are deduplicated.

• Staleness downgrade: If a source file is newer than sections.json, section references are downgraded to file-level (anchor/lines dropped) to avoid stale line numbers.

• Quoted-term bypass: Keywords in double quotes force inclusion even below the MIN_SCORE threshold, enabling explicit learnings requests.

• Telemetry design: Prompt hashes (not full text) are logged for privacy; session IDs enable cross-referencing with Read logs; tier classification (strong ≥3, weak <3) supports hit-rate stratification.

• Silent failure: All hooks are designed to never block a prompt/session/tool call — any I/O or parsing error results in a no-op.

• Provider resolution: Supports both learnings-providers.json entries (localPath, projectLocal) and runtime discovery via git anchors for commit-based staleness tracking.

https://claude.ai/code/session_01LnbK3YBtPx3GQUbCrkc43S

[ahoym]

Team Review: Add learnings-suggest hook: federated keyword matching & section-level indexing

A well-designed learnings suggestion system with solid fail-safe discipline throughout — no hook ever blocks a prompt, the atomic index write is correct, and the two-tier matching architecture (section-level preferred, file-level supplement) is clean and extensible. The main areas to address are a behavioral gap from duplicated provider loading logic, a SCHEMA_VERSION type mismatch that's latent now but a trap on the next bump, and the global scope of hook registration which adds per-prompt overhead in every project session.

Reviewed by: claude-config-reviewer, architecture-reviewer, python-engineer

Findings

14 finding(s) — see inline comments.

Positive Signals

• Fail-safe discipline is excellent: every wrapper shell script exits 0 on a missing or non-executable binary; the Python hooks catch all exceptions at the outer boundary with a clear # noqa: BLE001 rationale. The system never blocks a session under any failure mode.
• Atomic index write (tmp file + rename) in index-build.rs correctly prevents a partial sections.json from poisoning the hook on concurrent reads.
• The staleness downgrade in run() — demoting a section hit to file-level when the source file is newer than sections.json — is a thoughtful degradation path that keeps suggestions usable without requiring a full rebuild after every content edit.
• setup-claude.sh is updated to include hooks/ in the symlink list and runs bootstrap.sh idempotently, preserving the single-step provisioning invariant.
• FILTER_PREFIXES is a single named constant; adding a new exclusion is a one-line change in an obvious location.
• The load_jsonl function in analyze.py handles missing files, empty lines, and per-line JSONDecodeError gracefully — correct for a log-file reader where individual record corruption should never abort the whole analysis.

---

• Co-Authored with Claude Code (Claude Sonnet 4.6)
• Persona: claude-config-reviewer, architecture-reviewer, python-engineer
• Role: Team-Reviewer

[ahoym]

Team Re-review: Add learnings-suggest hook: federated keyword matching & section-level indexing

All 14 findings addressed across 6 focused commits (da3836d–17d8b79). Each commit maps cleanly to one or more prior findings; the fixes are scoped, no new issues introduced.

Reviewers (this cycle): architecture-reviewer, claude-config-reviewer, python-engineer
Carried forward: none

Previous Findings

• ✅ 14 resolved (code change verified)

New Findings

0 new findings.

Positive Signals

• src/lib.rs extraction is exactly right: Provider, home(), expand_tilde(), and providers(include_project_local: bool) are now shared across all three binaries with SCHEMA_VERSION: u64 unified at the crate root. The boolean parameter makes the projectLocal behavioral split explicit and auditable.
• run() decomposition into match_sections, match_files, merge_and_dedup, compute_forced, and format_block achieves the intended structure — each phase is independently testable and run() reads as a clean coordinator.
• The early-exit guard ([ -f "$HOME/.claude/learnings-providers.json" ] || exit 0) is a minimal, correct solution to the global-hooks-scope concern: hooks stay in settings.json while eliminating noise on non-dotfiles sessions without restructuring the gitignore contract.

---

• Co-Authored with Claude Code (claude-sonnet-4-6)
• Persona: architecture-reviewer, claude-config-reviewer, python-engineer
• Role: Team-Reviewer

[ahoym]

Team Re-review: Add learnings-suggest hook: federated keyword matching & section-level indexing

c455c46 extracts five inline magic literals across Rust and Python to named constants — artifacts_dir() in lib.rs, three keyword-scanning constants in index-build.rs, LEARNINGS_SUBPATHS in read-log.rs, and LEARNINGS_DIRS in learnings-staleness.py. Pure refactor with no logic changes; cross-reference comments between the mirrored Rust/Python constants are accurate and symmetric.

Reviewers (this cycle): orchestrator inline review (subagent-skip: <100 lines, pure constant extraction)
Carried forward: architecture-reviewer, claude-config-reviewer, python-engineer

Previous Findings

• ✅ 14 resolved (verified in prior cycle — no change in status)

New Findings

0 new findings.

Positive Signals

• artifacts_dir() correctly deduplicates the three-part join chain (home().join(".claude").join("claude-artifacts").join("ast")) that previously appeared verbatim in main.rs, index-build.rs, and read-log.rs. The docstring cross-reference to analyze.py’s ART constant is accurate.
• The LEARNINGS_SUBPATHS ↔ LEARNINGS_DIRS cross-reference comments are symmetric and explain the form difference (leading / vs claude/ prefix) — exactly what a future maintainer needs to keep them in sync.
• KEYWORDS_HEADER_SEARCH_LIMIT comment clarifies it’s a search budget, not a header-size assertion — the distinction matters if a file’s header is malformed.

---

• Co-Authored with Claude Code (claude-sonnet-4-6)
• Persona: architecture-reviewer, claude-config-reviewer, python-engineer
• Role: Team-Reviewer