Hybrid retrieval: BM25 + RRF + optional rerank hook

[thorwhalen]

Closes #16. Refs #1 (the seam deferred from the now-closed #7).

What

Adds a hybrid retrieval mode to ir.search:

• mode='dense' | 'lexical' | 'hybrid' — default dense, so existing callers are unaffected (byte-identical path, verified by the unchanged pipeline tests).
• dense — exact brute-force cosine (unchanged).
• lexical — Okapi BM25 over the candidate texts.
• hybrid — dense + BM25 fused by Reciprocal Rank Fusion (rank-based, sidestepping the cosine/BM25 score-scale mismatch).
• Optional rerank hook (any ef.Reranker); default None keeps retrieval offline (no model download / API call).
• New kwargs: rrf_k, fetch_k, rerank, bm25 (Okapi params).
• CLI: ir search <corpus> <query> --mode dense|lexical|hybrid.

Why

Per the retrieval research (misc/docs/ir_02), dense-only retrieval fails silently on exact identifiers, code, and rare terms — exactly the regime of short, identifier-heavy capability text (skill / package / tool names). BM25 + RRF + rerank is the production-consensus fix; ToolRet found lexical BM25 beating several dense models on tool retrieval.

Reuse, not reinvention

• vd.bm25_lexical_search — BM25, run over the already-filtered candidates via a zero-copy mapping view (record_id -> obj with .text/.metadata); no vectors duplicated.
• vd.reciprocal_rank_fusion — RRF.
• ef.rerank / ef.Reranker — the rerank contract.
• Graceful degrade: if vd is unavailable, hybrid falls back to dense and lexical returns no results, each with a warning — a missing optional dep never hard-fails a search.

Validation

• 37 tests pass (6 new in tests/test_hybrid.py), ruff clean.
• New tests prove the value: a rare identifier (zxqv) ranks last under dense, first under lexical, and is lifted under hybrid; the rerank hook reorders across all three modes (incl. per_artifact=False).
• Validated end-to-end on the real packages corpus with MiniLM: all three modes return sensible rankings (hybrid RRF scores ~1/(k+1) magnitude as expected).
• Passed an adversarial multi-lens review (correctness / reuse / API-backcompat / tests); the four minor/nit findings it surfaced are addressed in this branch.

Follow-ups (not in this PR)

• Unify ir's storage behind a vd.Collection to reuse vd.hybrid_search wholesale.
• Document / query expansion + instruction-tuned embedders (ir_02 Stage 2/3).
• A concrete CLI reranker (e.g. a local cross-encoder) on top of the programmatic hook.