Skip to content

Add source-grounded retrieval inspector #15

Description

@WalksWithASwagger

Context

RAG is already useful, but the user needs to see why a chunk was retrieved and whether it helped. The latest source-grounded tools make citations and source visibility table stakes.

Source: docs/WHISPERFORGE-MASTER-PLAN-2026-05-18.md (wf-retrieval-inspector, Phase 2).

Linear: BC-223.

Acceptance Criteria

  • Expose retrieved KB chunks per stage with source path, heading/offset when available, score, and reason/query context.
  • Show always-included voice anchors separately from semantic retrieval hits.
  • Persist retrieval metadata into run artifacts so it can be inspected after the session.
  • Add a compact UI or exported report that compares legacy KB dump vs RAG for the current capture/profile.
  • Keep Auto/Always/Never RAG behavior compatible with current settings.

Tests/Evals

  • Unit tests for retrieval metadata shape and serialization.
  • Fixture test that proves voice anchors and semantic hits are distinguishable.
  • No network-dependent eval required.

Verification

  • make test
  • make eval-fixture
  • git diff --check

Agent Instructions

Build on whisperforge_core/rag instead of adding a parallel retrieval path. Keep UI density modest; this is an inspector, not a second app.

Out of Scope

Do not replace the embedding model or rewrite RAG scoring unless required by tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:featureUser-facing feature delivery.priority:p0Critical or first-wave delivery work.repo:spektorAIWork in the WalksWithASwagger/spektorAI repository.review-readyAgent-created PR is ready for human review.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions