SFF-061: Private repository and knowledge retrieval adapters

[jmrpineda]

Reported by: Codex
Requested by: jmr.pineda
Priority: P1
Affected surfaces: Execution tools, repository context retrieval, RAG/CAG integration, prompt assembly, phase execution context, gateway adapters
Constraints: First cut should be read-only; connectors must be normalized behind SpecForge contracts; do not make arbitrary network or filesystem access the default retrieval path; preserve OSS-local current-repo retrieval while routing shared or secrets-backed retrieval through a gateway.

Summary

SpecForge should provide governed retrieval adapters for private repositories, documentation corpora, and code or architecture knowledge sources so model-backed phases can use richer context without collapsing into unrestricted tool access, while keeping the packaging boundary between OSS-local retrieval and Central or compatible gateway retrieval explicit.

Problem / opportunity

Once SpecForge defines a governed execution-tool policy, the next value layer is practical retrieval. Teams want the model to consult related private repositories, internal docs, semantic search, and code graphs. Today that need would either force oversized prompts, ad hoc provider-specific tool wiring, or unsafe direct access. A normalized retrieval layer lets SpecForge support RAG and CAG as harness capabilities rather than as opaque environment hacks. The additional product constraint is that the public OSS runtime must remain valuable, so current-repo retrieval should stay local while shared knowledge services, private cross-repo access, and secrets-backed retrieval become gateway-routed capabilities.

Requested behavior

Add a first wave of read-only retrieval adapters exposed as governed execution tools. These tools should cover the main knowledge-access use cases needed by SpecForge phases, such as repository search, file reads from approved private repositories, document retrieval, semantic knowledge retrieval, and code or architecture graph queries where available, with explicit routing defaults for OSS-local versus gateway-routed execution.

Scope

• In scope: Read-only retrieval-tool contracts; adapter abstraction for private repos, docs, vector or semantic retrieval, and code-graph retrieval; normalized result shapes; source attribution in returned context; allowlist-based repository or corpus scoping; routing defaults for local-only, local-preferred, central-preferred, and central-required retrieval tools.
• Out of scope: Broad write access to external systems; unrestricted web browsing by default; implementing every enterprise connector in the first cut; provider-owned retrieval as the system of record.

Acceptance criteria

1. SpecForge exposes a first useful set of governed read-only retrieval tools for private repository and knowledge access.
2. Retrieval results have normalized response shapes with source identity that can be persisted or cited in execution evidence.
3. Repository and corpus reach can be constrained by policy so a phase cannot query arbitrary sources outside its envelope.
4. The first retrieval wave makes a clear packaging distinction between OSS-local current-repo retrieval and gateway-routed shared or external knowledge access.

Notes

• Candidate first tools:
  • repo_search
  • repo_read_file
  • knowledge_retrieve
  • graph_query
• Each tool should return enough metadata for attribution, deduplication, and later evidence inspection, including source ID, source path or URI, retrieval reason, and truncation metadata where relevant.
• Product packaging guidance is captured in doc/execution-tool-packaging.md.