diff --git a/CHANGELOG.md b/CHANGELOG.md index 36bf98f..eaae298 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,9 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.5.1] - 2026-08-15 + ### Added -- Added TinyFish Search and Fetch provider support through `TINYFISH_API_KEY`. +- TinyFish Search and Fetch provider support through `TINYFISH_API_KEY`. + +### Changed + +- Search filter support is now explicit and strictly enforced; `web_search` honors abort signals during filter resolution. +- Provider catalog is centralized so registration order and metadata stay consistent across the registry. +- Provider operation policy is unified behind a single execution hook shared by all providers. +- Config refresh is aligned with the tool lifecycle; reload payloads are validated to prevent config corruption. +- Content extraction has a fallback seam with simplified error handling and hardened cancellation propagation. +- Updated Biome schema to `2.5.8`. ## [0.5.0] - 2026-07-21 diff --git a/README.md b/README.md index 12175f8..835cebf 100644 --- a/README.md +++ b/README.md @@ -7,15 +7,11 @@ Web search, content extraction, documentation lookup, and deep research tools for [Pi](https://github.com/earendil-works/pi). -## What's new in 0.5.0 +## What's new in 0.5.1 -> **Dashboard replaces typed `/tools` subcommands.** Run `/tools` to open the interactive dashboard with three tabs: **Providers** (toggle, key, default, run tests), **Status** (budgets, session outcomes, latency), and **Activity** (recent provider events). Typed subcommands such as `/tools status`, `/tools enable`, `/tools key`, `/tools test`, `/tools default`, and `/tools monitor on|off` are no longer supported. +> **TinyFish provider added.** `TINYFISH_API_KEY` enables keyless search and fetch through the new TinyFish provider — see the [provider table](#available-providers) for setup. > -> **Provider budgets are enforceable.** Hard, managed, unlimited, and shared-pool policies replace the old `monthlyQuota` settings. Budget status is reported per provider in the Status tab. -> -> **`openai-codex` requires Pi OAuth.** The `OPENAI_API_KEY` fallback is gone. Run `/login` and select `openai-codex` before using it. To search with `OPENAI_API_KEY` instead, configure `openai-web-search`. -> -> **Usage persistence moved.** Provider usage now lives at `$PI_CODING_AGENT_DIR/cache/pi-tools/usage.json` using a version 2 UTC-period ledger. +> **Internal architecture deepened.** Search filter enforcement, provider catalog, operation policy, runtime lifecycle, and content extraction now share clearer seams. No public API changes. ## Install diff --git a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-1-filters.md b/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-1-filters.md deleted file mode 100644 index f271b6d..0000000 --- a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-1-filters.md +++ /dev/null @@ -1,459 +0,0 @@ -# Pi Tools Phase 1: Explicit Search Filter Semantics Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Stop search providers from silently ignoring requested domain/date filters. - -**Architecture:** Add a small filter-support contract to `SearchProvider`. Providers declare whether domain filters are native, locally post-filtered, or unsupported, and whether date filters are native or unsupported. `web_search` removes unsupported candidates before execution and applies the existing safe domain post-filter where declared. - -**Tech Stack:** TypeScript, native `URL`, existing filter helpers, Vitest. - ---- - -## Atomic result - -After this phase, an unfiltered search behaves exactly as before. A filtered search either uses a provider that enforces every requested filter, locally filters domains when the provider declares that mode, or returns an explicit error naming unsupported filter groups. Provider ordering, budgets, fallback, and fusion remain unchanged for eligible providers. - -## File map - -Modify: - -- `src/providers/types.ts` — define `SearchFilterMode`, `SearchFilterSupport`, and require `SearchProvider.filterSupport`. -- `src/utils/filters.ts` — add safe host-based domain post-filtering and unsupported-filter detection. -- `src/providers/http-adapter.ts` — carry filter-support metadata through the shared HTTP adapter. -- `src/providers/http-providers.ts` — declare support for each HTTP provider configuration. -- `src/providers/duckduckgo.ts` — declare query-domain post-filter support and unsupported date semantics. -- `src/providers/exa.ts` — declare native domain/date support. -- `src/providers/tavily.ts` — declare native domain support and unsupported date semantics. -- `src/providers/serper.ts` — declare native domain/date support. -- `src/providers/tinyfish.ts` — declare native domain/date support. -- `src/providers/firecrawl.ts`, `src/providers/jina.ts`, `src/providers/ollama.ts`, `src/providers/openai-codex.ts`, `src/providers/openai-web-search.ts`, `src/providers/parallel.ts`, `src/providers/searxng.ts`, `src/providers/sofya.ts` — declare unsupported filter groups instead of accepting silent no-ops. -- `src/providers/registry.ts` — preserve `filterSupport` when wrapping registered providers. -- `src/tools/web-search.ts` — reject or filter candidates before fallback/fusion and apply local domain filtering. -- Search-provider test fixtures in `tests/config-manager.test.ts`, `tests/commands/tools.test.ts`, `tests/commands/tools-actions.test.ts`, `tests/commands/tools-dashboard.test.ts`, `tests/providers/registry.test.ts`, `tests/providers/types.test.ts`, and `tests/tools/web-search.test.ts` — satisfy the required provider contract. - -Create: - -- `tests/providers/filter-support.test.ts` — support matrix and domain filtering contract tests. - -## Tasks - -### Task 1: Add the filter-support contract and failing tests - -**Files:** - -- Modify: `src/providers/types.ts` -- Modify: `src/utils/filters.ts` -- Create: `tests/providers/filter-support.test.ts` -- Modify: `tests/providers/types.test.ts` - -- [ ] **Step 1: Define the support types.** - -Add the following types beside `SearchFilters` in `src/providers/types.ts`: - -```ts -export type SearchFilterMode = "native" | "post-filter" | "unsupported"; - -export interface SearchFilterSupport { - domains: SearchFilterMode; - dates: Exclude; -} - -export const UNSUPPORTED_SEARCH_FILTERS: SearchFilterSupport = { - domains: "unsupported", - dates: "unsupported", -}; -``` - -Add `readonly filterSupport: SearchFilterSupport;` to `SearchProvider`. Requiring the field makes a production adapter’s filter contract visible at its interface; test doubles will use `UNSUPPORTED_SEARCH_FILTERS` when they do not model filters. - -- [ ] **Step 2: Add safe domain filtering and unsupported-group detection.** - -Append these functions to `src/utils/filters.ts`: - -```ts -import type { - SearchFilterSupport, - SearchFilters, - SearchResult, -} from "../providers/types.ts"; - -function hostMatchesDomain(urlValue: string, domain: string): boolean { - try { - const host = new URL(urlValue).hostname.toLowerCase(); - const normalized = domain - .trim() - .toLowerCase() - .replace(/^\.+|\.+$/g, ""); - return ( - normalized.length > 0 && - (host === normalized || host.endsWith(`.${normalized}`)) - ); - } catch { - return false; - } -} - -export function filterResultsByDomains( - results: SearchResult[], - filters?: SearchFilters, -): SearchResult[] { - if (!filters?.includeDomains?.length && !filters?.excludeDomains?.length) - return results; - - return results.filter((result) => { - const included = - !filters.includeDomains?.length || - filters.includeDomains.some((domain) => - hostMatchesDomain(result.url, domain), - ); - const excluded = - filters.excludeDomains?.some((domain) => - hostMatchesDomain(result.url, domain), - ) ?? false; - return included && !excluded; - }); -} - -export function unsupportedSearchFilters( - filters: SearchFilters | undefined, - support: SearchFilterSupport, -): string[] { - const unsupported: string[] = []; - if ( - (filters?.includeDomains?.length || filters?.excludeDomains?.length) && - support.domains === "unsupported" - ) { - unsupported.push("domains"); - } - if ( - (filters?.startDate || filters?.endDate) && - support.dates === "unsupported" - ) { - unsupported.push("dates"); - } - return unsupported; -} -``` - -Keep `applyDomainFilters()` unchanged; it remains useful for providers that translate domains into upstream query syntax. The new post-filter must compare exact hosts and subdomains so `notexample.com` does not satisfy `example.com`. - -- [ ] **Step 3: Write the failing contract tests.** - -Add tests to `tests/providers/filter-support.test.ts`: - -```ts -import { describe, expect, it } from "vitest"; -import { - filterResultsByDomains, - unsupportedSearchFilters, -} from "../../src/utils/filters.ts"; -import { - UNSUPPORTED_SEARCH_FILTERS, - type SearchFilterSupport, - type SearchResult, -} from "../../src/providers/types.ts"; - -const results: SearchResult[] = [ - { title: "Docs", url: "https://docs.example.com/a", snippet: "docs" }, - { title: "Root", url: "https://example.com/b", snippet: "root" }, - { title: "Lookalike", url: "https://notexample.com/c", snippet: "other" }, -]; - -describe("search filter support", () => { - it("filters included domains by host and subdomain", () => { - expect( - filterResultsByDomains(results, { includeDomains: ["example.com"] }), - ).toEqual(results.slice(0, 2)); - }); - - it("filters excluded domains without matching lookalike hosts", () => { - expect( - filterResultsByDomains(results, { excludeDomains: ["example.com"] }), - ).toEqual([results[2]]); - }); - - it("reports unsupported domain and date groups", () => { - expect( - unsupportedSearchFilters( - { - includeDomains: ["example.com"], - startDate: "2026-01-01", - }, - UNSUPPORTED_SEARCH_FILTERS, - ), - ).toEqual(["domains", "dates"]); - }); - - it("allows native domains and dates", () => { - const support: SearchFilterSupport = { domains: "native", dates: "native" }; - expect( - unsupportedSearchFilters( - { includeDomains: ["example.com"], endDate: "2026-08-12" }, - support, - ), - ).toEqual([]); - }); -}); -``` - -Update every `SearchProvider` test double to include `filterSupport: UNSUPPORTED_SEARCH_FILTERS`. Do not give a fixture native support unless that test is specifically testing filter behavior. - -- [ ] **Step 4: Run the focused tests and confirm they fail for the missing contract.** - -Run: - -```bash -pnpm exec vitest run tests/providers/filter-support.test.ts tests/providers/types.test.ts -``` - -Expected: collection/type failures until the new types and functions exist. - -### Task 2: Declare support on production adapters - -**Files:** - -- Modify: `src/providers/http-adapter.ts` -- Modify: `src/providers/http-providers.ts` -- Modify: `src/providers/duckduckgo.ts` -- Modify: `src/providers/exa.ts` -- Modify: `src/providers/tavily.ts` -- Modify: `src/providers/serper.ts` -- Modify: `src/providers/tinyfish.ts` -- Modify: `src/providers/firecrawl.ts` -- Modify: `src/providers/jina.ts` -- Modify: `src/providers/ollama.ts` -- Modify: `src/providers/openai-codex.ts` -- Modify: `src/providers/openai-web-search.ts` -- Modify: `src/providers/parallel.ts` -- Modify: `src/providers/searxng.ts` -- Modify: `src/providers/sofya.ts` - -- [ ] **Step 1: Make the shared HTTP adapter carry support metadata.** - -Add an optional field to `HttpSearchConfig` and copy it onto the returned provider: - -```ts -export interface HttpSearchConfig { - // existing fields... - filterSupport?: SearchFilterSupport; -} - -export function createHttpSearchProvider( - apiKey: string, - config: HttpSearchConfig, -): SearchProvider { - return { - name: config.name, - label: config.label, - filterSupport: config.filterSupport ?? UNSUPPORTED_SEARCH_FILTERS, - async search(/* existing parameters */) { - // existing request and parser implementation - }, - }; -} -``` - -Use a new object spread or a frozen constant if needed to avoid a provider mutating the shared unsupported object. - -- [ ] **Step 2: Mark native and post-filter adapters.** - -Use these exact declarations: - -```ts -const NATIVE_DOMAIN_DATE_FILTERS: SearchFilterSupport = { - domains: "native", - dates: "native", -}; - -const NATIVE_DOMAIN_FILTERS: SearchFilterSupport = { - domains: "native", - dates: "unsupported", -}; - -const POST_FILTER_DOMAIN_FILTERS: SearchFilterSupport = { - domains: "post-filter", - dates: "unsupported", -}; -``` - -Set `filterSupport` as follows: - -- Brave Search, Exa, Serper, and TinyFish: `NATIVE_DOMAIN_DATE_FILTERS`. -- Tavily: `NATIVE_DOMAIN_FILTERS`. -- DuckDuckGo: `POST_FILTER_DOMAIN_FILTERS`; its date timelimit is approximate and its end date is not enforced, so dates remain unsupported. -- Brave LLM, fastCRW, LangSearch, Linkup, Marginalia, Perplexity, WebSearchAPI, and You.com: `UNSUPPORTED_SEARCH_FILTERS`. -- Firecrawl, Jina, Ollama, OpenAI Codex, OpenAI web search, Parallel, SearXNG, and Sofya: `UNSUPPORTED_SEARCH_FILTERS`. - -For class-based providers, add `readonly filterSupport = ...` beside `name` and `label`. For HTTP providers, pass the value in each `createHttpSearchProvider` config. Import the constants as type-safe values from `src/providers/types.ts`. - -- [ ] **Step 3: Preserve metadata through registry wrappers.** - -In `ProviderRegistry.registerProvider()`, copy the source metadata into the wrapped search provider: - -```ts -provider: { - name: provider.name, - label: provider.label, - filterSupport: provider.filterSupport, - search: async (query, maxResults, signal, filters) => { - this.consume(options.name, { capability: "search", maxResults }); - return provider.search(query, maxResults, signal, filters); - }, -}, -``` - -Do not apply filtering in the registry; the registry should preserve provider policy metadata and budget behavior, while the web-search module decides how requested filters affect a call. - -- [ ] **Step 4: Run provider contract tests.** - -Run: - -```bash -pnpm exec vitest run tests/providers/filter-support.test.ts tests/providers/http-adapter.test.ts tests/providers/registry.test.ts tests/providers/types.test.ts -``` - -Expected: all focused tests pass. - -### Task 3: Enforce the contract in `web_search` - -**Files:** - -- Modify: `src/tools/web-search.ts` -- Modify: `tests/tools/web-search.test.ts` - -- [ ] **Step 1: Add the explicit unsupported-filter result shape.** - -Extend `WebSearchDetails` without changing existing fields: - -```ts -interface WebSearchDetails { - provider: string; - resultCount: number; - unsupportedFilters?: string[]; - fusionMeta?: { - providersUsed: string[]; - degraded: boolean; - results: Array<{ url: string; providers: string[] }>; - }; -} -``` - -After `buildFilters(params)`, partition candidates with `unsupportedSearchFilters(filters, provider.filterSupport)`. If a candidate has unsupported groups, remove it. If no candidates remain and the original candidate list was non-empty, return: - -```ts -{ - content: [{ type: "text", text: `Search error: no eligible provider supports requested filters: ${groups.join(", ")}` }], - details: { provider: "none", resultCount: 0, unsupportedFilters: groups }, -} -``` - -If the original candidate list is empty, preserve the existing `Search error: No search providers available` result. - -- [ ] **Step 2: Apply only declared local domain post-filtering.** - -Add a local function in `web-search.ts`: - -```ts -function executeSearch( - provider: SearchProvider, - query: string, - maxResults: number, - signal: AbortSignal | undefined, - filters: SearchFilters | undefined, -): Promise { - return provider - .search(query, maxResults, signal, filters) - .then((results) => - provider.filterSupport.domains === "post-filter" - ? filterResultsByDomains(results, filters) - : results, - ); -} -``` - -Use `executeSearch()` in both the fusion candidate mapping and the ordinary `executeWithFallback()` mapping. Do not post-filter native providers; their upstream contract already enforces domains and local filtering could alter provider ranking unexpectedly. - -- [ ] **Step 3: Add regression tests for explicit and post-filter behavior.** - -Add tests to `tests/tools/web-search.test.ts` that prove: - -1. An unsupported date provider is not called and returns a message containing `dates`. -2. A provider with `domains: "post-filter"` returns only exact-domain/subdomain results. -3. A fusion call excludes unsupported candidates before execution and still fuses the eligible providers. -4. An unfiltered call still sends the same candidate list and returns the same result formatting. - -Use providers shaped like this in the tests: - -```ts -const unsupported: SearchProvider = { - name: "unsupported", - label: "Unsupported", - filterSupport: UNSUPPORTED_SEARCH_FILTERS, - search: vi.fn().mockResolvedValue([]), -}; - -const postFiltered: SearchProvider = { - name: "post-filtered", - label: "Post-filtered", - filterSupport: { domains: "post-filter", dates: "unsupported" }, - search: vi.fn().mockResolvedValue([ - { title: "good", url: "https://docs.example.com/a", snippet: "good" }, - { title: "bad", url: "https://other.example/a", snippet: "bad" }, - ]), -}; -``` - -- [ ] **Step 4: Run focused tool tests.** - -Run: - -```bash -pnpm exec vitest run tests/tools/web-search.test.ts tests/providers/filter-support.test.ts -``` - -Expected: all tests pass. - -### Task 4: Complete fixture updates and commit the phase - -**Files:** - -- Modify: `tests/config-manager.test.ts` -- Modify: `tests/commands/tools.test.ts` -- Modify: `tests/commands/tools-actions.test.ts` -- Modify: `tests/commands/tools-dashboard.test.ts` -- Modify: `tests/providers/registry.test.ts` -- Modify: `tests/providers/types.test.ts` - -- [ ] **Step 1: Update all remaining SearchProvider fixtures.** - -Add `filterSupport: UNSUPPORTED_SEARCH_FILTERS` to each fixture returned or inline-constructed in the listed tests. Keep the field explicit rather than using a cast, so future test providers cannot silently omit the contract. - -- [ ] **Step 2: Run the complete phase gate.** - -Run: - -```bash -pnpm exec vitest run tests/providers/filter-support.test.ts tests/providers/http-adapter.test.ts tests/providers/registry.test.ts tests/providers/types.test.ts tests/tools/web-search.test.ts tests/commands/tools.test.ts tests/commands/tools-actions.test.ts tests/commands/tools-dashboard.test.ts tests/config-manager.test.ts -pnpm check -git diff --check -``` - -Expected: focused tests and the full suite pass; only the documented pre-existing Biome and Node-engine warnings remain. - -- [ ] **Step 3: Commit the atomic phase.** - -```bash -git add src/providers src/utils/filters.ts src/tools/web-search.ts tests -git commit -m "feat: make search filter support explicit" -``` - -The commit must not include catalog, runtime lifecycle, operation-policy, or extraction changes. - -## Phase completion gate - -- Filter support is declared on every production `SearchProvider`. -- Unsupported filtered calls fail explicitly before a provider request. -- Post-filter domains use exact host/subdomain matching. -- Unfiltered search behavior is unchanged. -- Focused tests, `pnpm check`, and `git diff --check` pass. diff --git a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-2-catalog.md b/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-2-catalog.md deleted file mode 100644 index 679e890..0000000 --- a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-2-catalog.md +++ /dev/null @@ -1,607 +0,0 @@ -# Pi Tools Phase 2: Single Provider Catalog Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Give provider identity, built-in defaults, and fallback environment names one source of truth. - -**Architecture:** Add a provider catalog beside the provider adapters. Each catalog entry owns a `ProviderMeta`, its default `ProviderConfigEntry`, and its fallback environment variable when one exists. `config.ts`, `ConfigManager`, the provider barrel, and the `/tools` dashboard consume catalog projections. Gemini remains a non-provider service entry in the same catalog module so its existing fallback environment mapping stays centralized without pretending Gemini is a registry provider. - -**Tech Stack:** TypeScript, existing provider metadata factories, existing config merge/validation, Vitest. - ---- - -## Atomic result - -After this phase, adding or removing a built-in provider requires changing one catalog entry and its adapter import. The catalog’s provider-name set, default-config set, and fallback-env set are checked against one another. Existing `allProviders`/dashboard order and config/registration order remain unchanged. User configuration still overrides catalog defaults through the existing merge path, and provider registration still skips disabled, unkeyed, or failing providers exactly as before. - -## File map - -Create: - -- `src/providers/catalog.ts` — provider catalog entries plus derived provider/default/fallback projections. -- `tests/providers/catalog.test.ts` — catalog consistency and projection tests. - -Modify: - -- `src/providers/all.ts` — re-export catalog projections for compatibility. -- `src/config.ts` — consume catalog defaults and fallback environment values; remove duplicate provider maps. -- `src/config-manager.ts` — receive the catalog’s provider metadata projection. -- `src/index.ts` — use catalog metadata for registration and `/tools` display. -- `src/providers/searxng.ts` — use the resolved key injected by `ConfigManager`, removing the config-to-provider value import. -- `tests/config.test.ts` — preserve fallback/default behavior while importing the catalog-backed export. -- `tests/config-manager.test.ts` — use catalog-shaped provider fixtures and verify the injected key path. -- `tests/providers/all.test.ts` — assert the compatibility barrel and catalog contain the same provider set. -- `tests/providers/searxng.test.ts` — pass the resolved key through the provider factory after the config runtime import is removed. - -Do not modify provider request behavior, budgets, selection order, or user-config validation in this phase. - -## Tasks - -### Task 1: Define the catalog and its consistency tests - -**Files:** - -- Create: `src/providers/catalog.ts` -- Create: `tests/providers/catalog.test.ts` -- Modify: `src/providers/all.ts` - -- [ ] **Step 1: Define the catalog entry shape.** - -Add these types and imports in `src/providers/catalog.ts`: - -```ts -import type { ProviderConfigEntry } from "../config.ts"; -import type { ProviderMeta } from "./types.ts"; - -export interface ProviderCatalogEntry { - readonly meta: ProviderMeta; - readonly defaultConfig: ProviderConfigEntry; - readonly fallbackEnv?: string; -} -``` - -Use a type-only import for config types. This keeps the catalog from importing config runtime code while `config.ts` imports catalog values. - -- [ ] **Step 2: Move the provider facts into one ordered catalog.** - -Move the provider imports currently assembled in `src/providers/all.ts` into `src/providers/catalog.ts`. Keep the existing `providerMetas` order for the `allProviders`/dashboard projection, and keep the catalog literal in the existing `DEFAULT_CONFIG.providers` order for config object insertion and `ConfigManager.registerFromConfig` registration order. These are two existing order contracts; do not make one projection silently reorder the other. Resolve imported metadata by name, then define one catalog literal containing each provider’s default config and fallback environment. This keeps identity, defaults, and fallback facts in one entry instead of three parallel maps. - -Define the metadata array first, using the current `all.ts` order: - -```ts -const providerMetas: readonly ProviderMeta[] = [ - ...httpProviders, - context7, - duckduckgo, - exa, - firecrawl, - jina, - ollama, - openaiCodex, - openaiWebSearch, - parallel, - searxng, - serper, - sofya, - tavily, - tinyfish, -]; -``` - -The `providerCatalog` literal below must retain the current `DEFAULT_CONFIG.providers` order shown in the existing `src/config.ts`: `brave`, `brave-llm`, `context7`, `duckduckgo`, `exa`, `fastcrw`, `firecrawl`, `jina`, `langsearch`, `linkup`, `marginalia`, `ollama`, `openai-codex`, `openai-web-search`, `parallel`, `perplexity`, `searxng`, `serper`, `sofya`, `tavily`, `tinyfish`, `websearchapi`, `youcom`. The `allProviders` projection must instead follow the current `all.ts` order: `brave`, `brave-llm`, `fastcrw`, `langsearch`, `linkup`, `marginalia`, `perplexity`, `websearchapi`, `youcom`, `context7`, `duckduckgo`, `exa`, `firecrawl`, `jina`, `ollama`, `openai-codex`, `openai-web-search`, `parallel`, `searxng`, `serper`, `sofya`, `tavily`, `tinyfish`. - -Then use a lookup helper that fails at module load if an adapter import and catalog entry diverge: - -```ts -const providerMetaByName = new Map( - providerMetas.map((meta) => [meta.name, meta] as const), -); - -function catalogEntry( - name: string, - defaultConfig: ProviderConfigEntry, - fallbackEnv?: string, -): ProviderCatalogEntry { - const meta = providerMetaByName.get(name); - if (!meta) throw new Error(`Missing provider metadata for ${name}`); - return { meta, defaultConfig, fallbackEnv }; -} -``` - -The implementation must contain all 23 current provider defaults, including disabled and local/unlimited providers, with these exact values: - -```ts -const providerCatalog: readonly ProviderCatalogEntry[] = [ - catalogEntry( - "brave", - { - enabled: true, - budget: { - mode: "hard", - limit: 5, - period: "month", - unit: "usd", - pool: "brave", - }, - apiKey: "BRAVE_API_KEY", - }, - "BRAVE_API_KEY", - ), - catalogEntry( - "brave-llm", - { - enabled: true, - budget: { - mode: "hard", - limit: 5, - period: "month", - unit: "usd", - pool: "brave", - }, - apiKey: "BRAVE_API_KEY", - }, - "BRAVE_API_KEY", - ), - catalogEntry( - "context7", - { - enabled: true, - budget: { mode: "hard", limit: 1000, period: "month", unit: "request" }, - apiKey: "CONTEXT7_API_KEY", - }, - "CONTEXT7_API_KEY", - ), - catalogEntry("duckduckgo", { enabled: true, budget: { mode: "unlimited" } }), - catalogEntry( - "exa", - { - enabled: true, - budget: { - mode: "hard", - limit: 10, - period: "month", - unit: "usd", - pool: "exa", - }, - apiKey: "EXA_API_KEY", - }, - "EXA_API_KEY", - ), - catalogEntry( - "fastcrw", - { - enabled: false, - budget: { mode: "hard", limit: 500, period: "lifetime", unit: "credit" }, - apiKey: "FASTCRW_API_KEY", - }, - "FASTCRW_API_KEY", - ), - catalogEntry( - "firecrawl", - { - enabled: true, - budget: { mode: "hard", limit: 1000, period: "month", unit: "credit" }, - apiKey: "FIRECRAWL_API_KEY", - }, - "FIRECRAWL_API_KEY", - ), - catalogEntry( - "jina", - { enabled: true, budget: { mode: "managed" } }, - "JINA_API_KEY", - ), - catalogEntry( - "langsearch", - { - enabled: false, - budget: { mode: "hard", limit: 1000, period: "day", unit: "request" }, - apiKey: "LANGSEARCH_API_KEY", - }, - "LANGSEARCH_API_KEY", - ), - catalogEntry( - "linkup", - { - enabled: false, - budget: { mode: "hard", limit: 20, period: "month", unit: "usd" }, - apiKey: "LINKUP_API_KEY", - }, - "LINKUP_API_KEY", - ), - catalogEntry( - "marginalia", - { enabled: false, budget: { mode: "managed" } }, - "MARGINALIA_API_KEY", - ), - catalogEntry( - "ollama", - { - enabled: false, - budget: { mode: "unlimited" }, - apiKey: "OLLAMA_API_KEY", - }, - "OLLAMA_API_KEY", - ), - catalogEntry("openai-codex", { enabled: true, budget: { mode: "managed" } }), - catalogEntry( - "openai-web-search", - { - enabled: true, - budget: { mode: "managed" }, - apiKey: "OPENAI_API_KEY", - }, - "OPENAI_API_KEY", - ), - catalogEntry( - "parallel", - { - enabled: false, - budget: { mode: "managed" }, - apiKey: "PARALLEL_API_KEY", - }, - "PARALLEL_API_KEY", - ), - catalogEntry( - "perplexity", - { - enabled: true, - budget: { mode: "managed" }, - apiKey: "PERPLEXITY_API_KEY", - }, - "PERPLEXITY_API_KEY", - ), - catalogEntry("searxng", { - enabled: false, - budget: { mode: "unlimited" }, - instanceUrl: "http://localhost:8080", - }), - catalogEntry( - "serper", - { - enabled: false, - budget: { - mode: "hard", - limit: 2500, - period: "lifetime", - unit: "request", - }, - apiKey: "SERPER_API_KEY", - }, - "SERPER_API_KEY", - ), - catalogEntry( - "sofya", - { - enabled: false, - budget: { mode: "managed" }, - apiKey: "SOFYA_API_KEY", - }, - "SOFYA_API_KEY", - ), - catalogEntry( - "tavily", - { - enabled: false, - budget: { mode: "hard", limit: 1000, period: "month", unit: "credit" }, - apiKey: "TAVILY_API_KEY", - }, - "TAVILY_API_KEY", - ), - catalogEntry( - "tinyfish", - { - enabled: true, - budget: { mode: "unlimited" }, - apiKey: "TINYFISH_API_KEY", - }, - "TINYFISH_API_KEY", - ), - catalogEntry( - "websearchapi", - { - enabled: false, - budget: { mode: "hard", limit: 2000, period: "month", unit: "credit" }, - apiKey: "WEBSEARCHAPI_API_KEY", - }, - "WEBSEARCHAPI_API_KEY", - ), - catalogEntry( - "youcom", - { - enabled: false, - budget: { mode: "hard", limit: 100, period: "lifetime", unit: "usd" }, - apiKey: "YOUCOM_API_KEY", - }, - "YOUCOM_API_KEY", - ), -]; -``` - -Derive config and fallback projections from `providerCatalog`; keep `allProviders` as the validated adapter-order metadata projection: - -```ts -export { providerCatalog }; - -export const allProviders: ProviderMeta[] = [...providerMetas]; - -export const defaultProviderConfigs: Record = - Object.fromEntries( - providerCatalog.map(({ meta, defaultConfig }) => [ - meta.name, - defaultConfig, - ]), - ); - -export const fallbackEnvMap: Record = Object.fromEntries([ - ...providerCatalog.flatMap(({ meta, fallbackEnv }) => - fallbackEnv ? [[meta.name, fallbackEnv] as const] : [], - ), - ["gemini", "GEMINI_API_KEY"] as const, -]); -``` - -Before exporting projections, validate that every catalog metadata name is unique, every imported metadata name has one catalog entry, and Gemini is not accidentally treated as a provider. A module-load error is preferable to silently dropping a provider from registration: - -```ts -const catalogNames = new Set(providerCatalog.map(({ meta }) => meta.name)); -if (catalogNames.size !== providerCatalog.length) - throw new Error("Duplicate provider catalog name"); -if (catalogNames.size !== providerMetas.length) - throw new Error("Provider metadata and catalog entry counts differ"); -for (const meta of providerMetas) { - if (!catalogNames.has(meta.name)) - throw new Error(`Provider ${meta.name} is missing from catalog`); -} -``` - -Do not export the mutable source tables. Export the catalog and projections only. - -- [ ] **Step 3: Add catalog consistency tests before consumers change.** - -Create `tests/providers/catalog.test.ts`: - -```ts -import { describe, expect, it } from "vitest"; -import { - allProviders, - defaultProviderConfigs, - fallbackEnvMap, - providerCatalog, -} from "../../src/providers/catalog.ts"; - -describe("provider catalog", () => { - it("has one entry per provider and no duplicate names", () => { - const names = providerCatalog.map(({ meta }) => meta.name); - expect(names.length).toBe(23); - expect(new Set(names).size).toBe(names.length); - expect(new Set(allProviders.map(({ name }) => name))).toEqual( - new Set(names), - ); - expect(names).toEqual([ - "brave", - "brave-llm", - "context7", - "duckduckgo", - "exa", - "fastcrw", - "firecrawl", - "jina", - "langsearch", - "linkup", - "marginalia", - "ollama", - "openai-codex", - "openai-web-search", - "parallel", - "perplexity", - "searxng", - "serper", - "sofya", - "tavily", - "tinyfish", - "websearchapi", - "youcom", - ]); - expect(allProviders.map(({ name }) => name)).toEqual([ - "brave", - "brave-llm", - "fastcrw", - "langsearch", - "linkup", - "marginalia", - "perplexity", - "websearchapi", - "youcom", - "context7", - "duckduckgo", - "exa", - "firecrawl", - "jina", - "ollama", - "openai-codex", - "openai-web-search", - "parallel", - "searxng", - "serper", - "sofya", - "tavily", - "tinyfish", - ]); - expect(Object.keys(defaultProviderConfigs)).toEqual(names); - }); - - it("has a valid default config for every provider", () => { - for (const { meta, defaultConfig } of providerCatalog) { - expect(defaultProviderConfigs[meta.name]).toEqual(defaultConfig); - expect(typeof defaultConfig.enabled).toBe("boolean"); - expect(defaultConfig.budget.mode).toMatch(/^(hard|managed|unlimited)$/); - } - }); - - it("keeps fallback env mappings in the catalog projection", () => { - expect(fallbackEnvMap.brave).toBe("BRAVE_API_KEY"); - expect(fallbackEnvMap.exa).toBe("EXA_API_KEY"); - expect(fallbackEnvMap.gemini).toBe("GEMINI_API_KEY"); - expect(fallbackEnvMap["openai-codex"]).toBeUndefined(); - }); -}); -``` - -Run: - -```bash -pnpm exec vitest run tests/providers/catalog.test.ts -``` - -Expected: the new catalog tests pass before the existing consumers are switched. - -- [ ] **Step 4: Keep the compatibility barrel.** - -Replace the provider assembly in `src/providers/all.ts` with: - -```ts -export { allProviders, providerCatalog } from "./catalog.ts"; -``` - -Existing imports of `allProviders` must continue to compile. Do not make callers import individual adapter modules just to access provider metadata. - -### Task 2: Make config consume catalog projections - -**Files:** - -- Modify: `src/config.ts` -- Modify: `src/providers/searxng.ts` -- Modify: `tests/config.test.ts` -- Modify: `tests/providers/searxng.test.ts` - -- [ ] **Step 1: Replace duplicate maps in `config.ts`.** - -Import the value projections from the catalog: - -```ts -import { defaultProviderConfigs, fallbackEnvMap } from "./providers/catalog.ts"; -``` - -Replace the manually maintained `FALLBACK_ENV_MAP` object with: - -```ts -export const FALLBACK_ENV_MAP = fallbackEnvMap; -``` - -Replace the literal `DEFAULT_CONFIG.providers` object with: - -```ts -providers: defaultProviderConfigs, -``` - -Leave `DEFAULT_GITHUB_CONFIG`, `DEFAULT_COMBINE_CONFIG`, `DEFAULT_DEEP_RESEARCH_CONFIG`, Gemini, YouTube, Video, and PDF defaults in `config.ts`; they are not provider catalog entries. Preserve the existing `loadMergedConfig` merge order and validation. - -- [ ] **Step 2: Break the provider-to-config runtime cycle in SearXNG.** - -`src/providers/searxng.ts` currently imports `resolveApiKey` as a runtime value. `ConfigManager` already resolves `providerConfig.apiKey` before calling `meta.create`, so pass the injected `key` directly: - -```ts -create: (key, providerConfig) => ({ - search: new SearXNGProvider({ - instanceUrl: providerConfig?.instanceUrl, - apiKey: key, - }), -}), -``` - -Remove the `resolveApiKey` import. This preserves explicit keys, environment-variable indirection, and shell commands because `ConfigManager` still resolves those values with `resolveApiKey` before calling the factory. Direct extraction paths that use `resolveProviderKey` continue to read the catalog-backed `FALLBACK_ENV_MAP`. Do not add fallback resolution to `ConfigManager` in this phase. - -Update `tests/providers/searxng.test.ts` so the provider-factory test passes the already resolved literal key as the first `create` argument. The factory no longer resolves `providerConfig.apiKey` itself; the `ConfigManager` injection test in Task 3 proves the production path. - -- [ ] **Step 3: Prove defaults and fallback behavior did not drift.** - -Keep the existing `FALLBACK_ENV_MAP` assertions and add a comparison in `tests/config.test.ts` between `Object.keys(loadConfig().providers)` and `Object.keys(defaultProviderConfigs)`. Keep `tests/extract/config-video.test.ts` unchanged; its existing Gemini assertion already covers the public `FALLBACK_ENV_MAP` export. - -Import `defaultProviderConfigs` from `src/providers/catalog.ts` in that test only for the provider-name/order comparison; do not export `DEFAULT_CONFIG` just to make the test observable. - -Run: - -```bash -pnpm exec vitest run tests/providers/catalog.test.ts tests/providers/all.test.ts tests/config.test.ts tests/providers/searxng.test.ts -``` - -Expected: provider count, names, built-in budgets, fallback env resolution, and Gemini config defaults remain unchanged. - -### Task 3: Route registration and dashboard metadata through the catalog - -**Files:** - -- Modify: `src/config-manager.ts` -- Modify: `src/index.ts` -- Modify: `tests/config-manager.test.ts` -- Modify: `tests/providers/all.test.ts` - -- [ ] **Step 1: Pass the catalog projection into `ConfigManager`.** - -Keep `ConfigManager`’s provider metadata argument typed as `readonly ProviderMeta[]` and pass `allProviders` from the catalog. Do not make `ConfigManager` know about individual provider modules. If the constructor is widened from `ProviderMeta[]` to `readonly ProviderMeta[]`, update tests and callers without changing registration behavior. - -- [ ] **Step 2: Build dashboard identity from the catalog-backed provider projection.** - -In `src/index.ts`, build the dashboard tier map from `allProviders`, which is exported by the catalog module in the existing `all.ts` order: - -```ts -const tierMap = new Map( - allProviders.map((meta) => [meta.name, meta.tier]), -); -const allProviderNames = allProviders.map(({ name }) => name); -``` - -The dashboard must continue showing all built-ins, including disabled providers, in the current `all.ts` order. Do not import `providerCatalog` separately just to rebuild the same metadata lookup. - -- [ ] **Step 3: Add the SearXNG injection regression test.** - -In `tests/config-manager.test.ts`, instantiate the real `searxng` metadata with a config key and spy on the resulting provider request headers. Assert that `ConfigManager` resolves the key and injects it into the provider factory. The production module import graph proves the provider no longer imports `config.ts` at runtime; keep the test deterministic by using the existing mocked `resolveApiKey`. - -- [ ] **Step 4: Run the registration and dashboard tests.** - -Run: - -```bash -pnpm exec vitest run tests/config-manager.test.ts tests/providers/all.test.ts tests/commands/tools.test.ts tests/commands/tools-actions.test.ts tests/commands/tools-dashboard.test.ts -``` - -Expected: all provider registration, name/tier display, and configuration command tests pass. - -### Task 4: Complete the phase gate and commit - -- [ ] **Step 1: Search for stale duplicate sources.** - -Run: - -```bash -rg -n "const (FALLBACK_ENV_MAP|DEFAULT_CONFIG)|export const allProviders|providerMeta as" src/config.ts src/providers src/index.ts -``` - -Expected: the provider default map and fallback map exist only as catalog projections; `all.ts` only re-exports catalog values; adapter imports remain in `catalog.ts`. - -- [ ] **Step 2: Run the complete phase gate.** - -```bash -pnpm exec vitest run tests/providers/catalog.test.ts tests/providers/all.test.ts tests/config.test.ts tests/config-manager.test.ts tests/providers/searxng.test.ts tests/extract/config-video.test.ts tests/commands/tools.test.ts tests/commands/tools-actions.test.ts tests/commands/tools-dashboard.test.ts -pnpm check -git diff --check -``` - -Expected: focused tests and the full suite pass with only the documented pre-existing Biome and Node-engine warnings. - -- [ ] **Step 3: Commit the atomic phase.** - -```bash -git add src/providers/catalog.ts src/providers/all.ts src/providers/searxng.ts src/config.ts src/config-manager.ts src/index.ts tests/providers/catalog.test.ts tests/config.test.ts tests/providers/searxng.test.ts tests/config-manager.test.ts tests/providers/all.test.ts -git commit -m "refactor: centralize provider catalog" -``` - -The commit must not include operation execution, tool re-registration, or extraction fallback changes. - -## Phase completion gate - -- Every built-in provider has exactly one catalog entry and one default config. -- Fallback env names, including the Gemini service fallback, are catalog-derived. -- `allProviders`, `ConfigManager`, config loading, and the dashboard consume catalog projections. -- User overrides and provider registration behavior are unchanged. -- Focused tests, `pnpm check`, and `git diff --check` pass. diff --git a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-3-operation-policy.md b/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-3-operation-policy.md deleted file mode 100644 index 3dac22d..0000000 --- a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-3-operation-policy.md +++ /dev/null @@ -1,485 +0,0 @@ -# Pi Tools Phase 3: Capability-Aware Provider Operation Policy Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Make provider selection, budgets, activity logging, fallback, fusion, and outcome metrics follow one capability-aware policy for search, fetch, code search, docs, and research. - -**Architecture:** Keep provider registration and budget ownership in `ProviderRegistry`. Store tier metadata for every registered capability, including research. Add one shared attempt runner for activity and outcome hooks. Route every provider-backed operation through registry wrappers and the shared runner. Research becomes a first-class registered capability backed by the existing `ProviderOperation` cost model; it is not a special direct Exa client path. - -**Tech stack:** TypeScript, native `AbortSignal`, existing activity monitor and budget registry, Vitest. - -## Readiness decision - -The original plan was not ready to implement. This revision resolves the following issues found against the clean Phase 3 branch at commit `4e35d6e`: - -- `fetch`, `code-search`, and `docs` registrations discard tier metadata, so automatic selection still depends on registration order. -- Explicit search and docs selections can bypass budget eligibility. -- `best-performing` exists only for search; fetch, code search, docs, and research cannot use the configured strategy. -- Fallback and fusion duplicate activity/timing/outcome instrumentation. -- `web_fetch`, code search, and docs tools bypass the shared attempt path. -- Research already has a `ProviderOperation` cost model, but the tool bypasses the registry and reserves through `beforeResearch`. -- Fusion does not define cancellation propagation when concurrent attempts are active. -- The previous file map omitted `tests/tools/web-search.test.ts` and the Exa research/provider tests needed for the new registration contract. -- The documented baseline was stale. The current clean branch passes 89 test files and 1,427 tests; the environment is Node 23.11.0 while the package requires Node >=24.15.0. - -## Atomic result - -After this phase: - -- `auto` selects eligible providers in tier order for search, fetch, code search, docs, and research. -- `best-performing` ranks eligible providers for every capability with active provider metrics; providers without samples retain the existing neutral score and stable registration order. -- An explicit provider name wins only when that registered capability exists and is budget-eligible; an exhausted explicit provider returns no candidate instead of bypassing the budget or silently selecting another provider. -- Hard budgets are consumed by the registered wrapper before delegation for search, fetch, code search, docs, and research. -- One shared attempt runner owns API activity start/completion/error logging and success/failure hooks for sequential fallback, fusion, direct capability tools, and research. -- Budget rejections are not failure metrics. Caller cancellation is rethrown and is not a provider failure. -- Search fusion retains its targeted/all batching, concurrency, provider-used/failed semantics, and RRF output. -- Research keeps its query deduplication, mode defaults, report rendering, output files, trust/key gating, and response shape while using registered research candidates. - -No tool is re-registered dynamically in this phase; that remains Phase 4. - -## Locked boundaries - -This phase may change: - -- provider capability types and registry wrappers; -- provider selection helpers and their budget eligibility; -- shared execution instrumentation; -- provider-backed tool factory seams and strategy routing; -- Exa research registration. - -This phase must not change: - -- ConfigManager refresh/listener behavior or tool re-registration; -- extraction fallback routing or the Phase 5 extraction seam; -- search filter support and post-filtering introduced in Phase 1; -- search fusion batching, target calculation, RRF scoring, or output formatting; -- result/error shapes, error sanitization, trust gating, credential resolution, or hard-budget rollback behavior; -- package dependencies, Node engine requirements, or unrelated lint warnings. - -Research budgeting is intentionally moved from the tool-local `beforeResearch` callback into the first-class registry wrapper. Adding a new research provider later will use the same `ResearchProvider` interface and selection path. - -## File map - -Modify: - -- `src/providers/types.ts` — add `ResearchProvider`, extend `ProviderMeta.create`, and define the shared capability alias. -- `src/providers/exa-deep-research.ts` — implement the registered research-provider shape without changing the HTTP request/response behavior. -- `src/providers/exa.ts` — return the Exa research client from the Exa metadata factory. -- `src/providers/registry.ts` — retain tier metadata for every capability map, add research registration/wrapping, centralize selection, expose execution hooks, and preserve budget semantics. -- `src/providers/execute.ts` — export fallback candidates/hooks, add `executeAttempt`, and use it in `executeWithFallback`. -- `src/providers/fusion.ts` — use `executeAttempt`, add signal propagation, and preserve concurrent fusion behavior. -- `src/tools/web-search.ts` — pass shared hooks to fallback/fusion while preserving Phase 1 filter eligibility. -- `src/tools/web-fetch.ts` — pass shared hooks through the existing registered-provider fallback. -- `src/tools/code-search.ts` — execute through `executeAttempt`. -- `src/tools/web-docs-search.ts` — execute through `executeAttempt`. -- `src/tools/web-docs-fetch.ts` — execute through `executeAttempt`. -- `src/tools/web-research.ts` — resolve registered research candidates and execute each unique query through shared fallback/hooks. -- `src/index.ts` — construct one registry hook set, pass it to all provider-backed tools, and route fetch/code/docs/research selection through the current strategy. -- `tests/providers/registry.test.ts` — cover all capability maps, selection order, explicit eligibility, research operation metadata, and budgets. -- `tests/providers/execute.test.ts` — cover the shared attempt contract and fallback delegation. -- `tests/providers/fusion.test.ts` — cover shared hooks, activity behavior, cancellation, and existing fusion semantics. -- `tests/providers/exa.test.ts` — verify the Exa factory exposes the research capability without changing search/fetch/code behavior. -- `tests/providers/exa-deep-research.test.ts` — preserve the research client HTTP contract and assert it satisfies the provider shape. -- `tests/tools/web-search.test.ts` — verify shared-hook injection remains compatible with filter and fusion behavior. -- `tests/tools/web-fetch.test.ts` — cover shared hooks through provider fallback. -- `tests/tools/code-search.test.ts` — cover attempt success/failure and unchanged rendered error output. -- `tests/tools/web-docs-search.test.ts` — cover attempt execution and unchanged propagated provider errors. -- `tests/tools/web-docs-fetch.test.ts` — cover attempt execution and unchanged propagated provider errors. -- `tests/tools/web-research.test.ts` — cover registered candidates, fallback, operation budget metadata, cancellation, and unchanged reports. -- `tests/index.test.ts` — verify tool construction remains unchanged and conditional research registration still follows current availability. -- `tests/index-strategy.test.ts` — verify the configured strategy reaches every capability resolver. - -Create: None. - -Delete: None. - -## Contracts to implement - -### Capability and registered provider contracts - -In `src/providers/types.ts`: - -```ts -import type { - DeepResearchParams, - DeepResearchResponse, - ExaDeepType, -} from "../research/types.ts"; - -export interface ResearchProvider { - readonly name: string; - readonly label: string; - deepResearch( - params: DeepResearchParams, - signal?: AbortSignal, - ): Promise; -} - -export type ProviderCapability = ProviderOperation["capability"]; -``` - -Extend the return type of `ProviderMeta.create()` with `research?: ResearchProvider`. Keep `ProviderOperation` discriminated and unchanged in meaning. `DeepResearchParams.numResults` is resolved by `web_research` before delegation, so the registry can derive the exact existing cost operation from the request: - -```ts -{ - capability: "research", - type: params.type, - maxResults: params.numResults ?? 10, - contentTypes: 2 + (params.summaryQuery ? 1 : 0), -} -``` - -In `src/providers/exa-deep-research.ts`, add `name = "exa"` and `label = "Exa"` and keep `deepResearch()` behavior unchanged. In `src/providers/exa.ts`, return one `ExaDeepResearchClient` as `research` from the metadata factory alongside the existing `ExaProvider` capabilities. The factory still receives the resolved key from ConfigManager; it must not resolve credentials itself. - -### Registry selection contract - -Replace the search-only tier registration shape with one internal reusable shape: - -```ts -interface RegisteredProvider { - provider: T; - tier: ProviderTier; -} -``` - -Use it for search, fetch, code-search, docs, and research maps. Every registered capability must retain the `options.tier` supplied by `registerProvider()`. Preserve Phase 1 search `filterSupport` metadata on the wrapped search provider. - -The registry must have these capability maps: - -```ts -private searchProviders = new Map>(); -private fetchProviders = new Map>(); -private codeSearchProviders = new Map>(); -private docsProviders = new Map>(); -private researchProviders = new Map>(); -``` - -Generalize eligibility and performance selection over a capability map keyed by the registered policy name. Use the existing score exactly: - -```text -successRate * 0.5 + speed * 0.3 + quality * 0.2 -``` - -Use the registry key, not `provider.name`, to look up metrics and budgets. A capability is scored only when its registered map contains that key. Providers without active samples receive the existing neutral score of `0.5`; stable map order breaks ties. All automatic and performance selections filter through `isEligible()`. - -Keep these search methods as compatibility wrappers: - -```ts -selectSearchCandidates(name?: string): SearchProvider[]; -selectSearchByPerformance(name?: string): SearchProvider | undefined; -selectSearchByPerformanceAll(): SearchProvider[]; -selectSearchForFusion(strategy: SelectionStrategy, name?: string): SearchProvider[]; -``` - -Their explicit-name behavior must now return a provider only if it exists in the search map and is budget-eligible. Automatic search retains tier order; best-performing search retains the current score formula. - -Add strategy-aware methods: - -```ts -selectFetchCandidates(strategy: SelectionStrategy = "auto"): FetchProvider[]; -selectCodeSearch(strategy: SelectionStrategy = "auto"): CodeSearchProvider | undefined; -selectDocs(name?: string): DocsProvider | undefined; -selectDocsByStrategy( - strategy: SelectionStrategy, - name?: string, -): DocsProvider | undefined; -selectResearchCandidates( - strategy: SelectionStrategy = "auto", - name?: string, -): ResearchProvider[]; -``` - -`auto` returns all eligible candidates in tier order for fetch and research, and the first eligible candidate for code search and docs. `best-performing` returns all scored eligible fetch/research candidates, and the first scored eligible code/docs candidate. Explicit names return only the named eligible provider. `selectDocs(name?)` remains the existing auto/name compatibility wrapper. - -### Budget wrapper contract - -Keep each wrapper’s existing `consume()` call before provider delegation. Add the research wrapper with the exact operation derivation above. `unregisterAll()` must delete research registrations as well as the other capability maps, while leaving usage counters available for re-registration. - -The wrapper must preserve: - -- hard-budget persistence before delegation; -- rollback when persistence fails; -- `BudgetExceededError` identity and message; -- managed/unlimited behavior; and -- shared pool counters. - -### Shared attempt contract - -In `src/providers/execute.ts`, export: - -```ts -export interface FallbackCandidate { - name: string; - execute: () => Promise; -} - -export interface ExecutionHooks { - onSuccess?: (providerName: string, latencyMs: number) => void; - onFailure?: (providerName: string) => void; -} - -export interface ExecuteAttemptOptions extends ExecutionHooks { - candidate: FallbackCandidate; - operation: string; - signal?: AbortSignal; - activityQuery?: string; -} - -export async function executeAttempt( - options: ExecuteAttemptOptions, -): Promise; -``` - -`executeAttempt()` must: - -1. call `signal?.throwIfAborted()` before logging or delegating; -2. log one API activity start with `activityQuery ?? operation`; -3. measure latency around the candidate; -4. check cancellation after the candidate resolves and before calling `onSuccess`; -5. call `onSuccess` and log status 200 only for a completed, non-cancelled result; -6. log a sanitized error with `sanitizeError()` for a rejected attempt; -7. call `onFailure` only for ordinary provider errors, never for `BudgetExceededError` or caller cancellation; and -8. rethrow the original error so the caller controls fallback, aggregation, or tool rendering. - -Do not convert cancellation into an `AggregateProviderError`. A pre-attempt cancellation produces no activity entry, matching the current fallback behavior. An in-flight cancellation may complete its activity entry as an error but must not call the failure hook. - -Reduce `executeWithFallback()` to candidate iteration, `executeAttempt()` delegation, and its existing empty-candidate/final `AggregateProviderError` behavior. Keep the existing operation names and error aggregation shape. - -### Fusion contract - -Add `signal?: AbortSignal` to `FusionOptions`. Check it before each batch and pass it to every `executeAttempt()` call. Use `activityQuery: \`fusion:${candidate.name}\``. Keep batches concurrent with `Promise.all`; do not route fusion through sequential fallback. - -Preserve these existing rules: - -- successful empty results call `onSuccess`, complete activity with status 200, and are not added to `providersUsed`; -- ordinary errors are added to `providersFailed` and the aggregate error list; -- budget rejections are added to the aggregate error list but not `providersFailed` and do not call `onFailure`; -- cancellation rejects the fusion operation instead of becoming a provider failure; -- targeted/all batch sizing, provider order, `degraded`, and RRF output do not change; and -- a run with no usable results throws `AggregateProviderError("search", errors)` with the existing messages. - -### Tool factory contracts - -Use one production `ExecutionHooks` object from `registry.getExecutionHooks()`. The registry method returns fresh closures: - -```ts -getExecutionHooks(): ExecutionHooks { - return { - onSuccess: (providerName, latencyMs) => - this.recordOutcome(providerName, { success: true, latencyMs }), - onFailure: (providerName) => - this.recordOutcome(providerName, { success: false }), - }; -} -``` - -Add an optional `ExecutionHooks` argument to provider-backed factories without removing their existing callback arguments unless the current factory has no compatibility value. Production code must pass the shared object, while existing direct factory tests may keep their legacy callback arguments. The factory behavior must be: - -- `web_search`: pass the shared hooks to both `executeWithFallback()` and `executeWithFusion()`; keep the existing result-quality callback separate; retain filter eligibility before candidate execution. -- `web_fetch`: pass the shared hooks into the existing registered-provider `executeWithFallback()` path; do not move fallback into extraction before Phase 5. -- `code_search`: wrap the one selected provider with `executeAttempt(operation: "code-search")`; retain its sanitized error-as-text result and missing-provider message. -- `web_docs_search`: wrap `searchLibrary()` with `executeAttempt(operation: "docs-search")`; retain provider errors as thrown tool failures. -- `web_docs_fetch`: wrap `getContext()` with `executeAttempt(operation: "docs-fetch")`; retain provider errors as thrown tool failures. -- `web_research`: accept research candidates, not an API-key resolver or `beforeResearch`; wrap each unique query with `executeWithFallback(operation: "research")` and the shared hooks. - -`web_research` must keep the existing request construction, query deduplication, mode defaults, response merge, source deduplication, report formats, file writes, append-entry record, and output text. A missing candidate list continues to produce the existing disabled/unavailable error. Budget reservation occurs inside the selected registry wrapper immediately before the Exa client call. - -In `src/index.ts`: - -1. construct `const executionHooks = registry.getExecutionHooks()` once after the registry; -2. pass that object to web search, web fetch, code search, docs search, docs fetch, and web research; -3. pass `configManager.current.selectionStrategy` to fetch, code-search, docs, and research resolvers; -4. keep search’s current default-provider, explicit-provider, combine, filter, and fusion behavior; -5. keep current session-start conditional registration for docs and research; and -6. remove the direct research `resolveApiKey`/`beforeResearch` wiring from the tool factory while preserving current key/trust gating through the registered Exa capability. - -## Tasks - -### Task 1: Register and select every provider capability - -**Files:** - -- Modify: `src/providers/types.ts` -- Modify: `src/providers/exa-deep-research.ts` -- Modify: `src/providers/exa.ts` -- Modify: `src/providers/registry.ts` -- Modify: `tests/providers/registry.test.ts` -- Modify: `tests/providers/exa.test.ts` -- Modify: `tests/providers/exa-deep-research.test.ts` - -- [ ] **Step 1: Add the research provider interface and Exa registration.** - -Add `ResearchProvider`, extend `ProviderMeta.create()`, and make `ExaDeepResearchClient` satisfy the interface. Return the client from Exa’s metadata factory. Do not change the Exa request body, headers, response normalization, or error behavior. - -- [ ] **Step 2: Generalize registered capability storage.** - -Replace the search-only tier shape with `RegisteredProvider` for all five maps. Add the research map, wrapper, unregister behavior, and the research operation derivation. Keep all existing wrapper method signatures and budget semantics. - -- [ ] **Step 3: Generalize tier and performance selection.** - -Factor the existing eligibility and score calculation over a capability map. Preserve the score weights, neutral score, stable tie order, and search compatibility methods. Make every explicit selection budget-aware. - -- [ ] **Step 4: Add strategy-aware capability selectors.** - -Implement the fetch, code-search, docs, and research methods with the exact contracts above. Automatic fetch/research candidates must be tier ordered even when registration order differs. Performance selection must exclude exhausted providers. - -- [ ] **Step 5: Add registry and Exa tests.** - -Use at least three providers at tiers 1, 2, and 3 with intentionally different registration order. Assert: - -1. auto search/fetch/research order is tier 1 → 2 → 3; -2. best-performing fetch/research order follows existing metrics; -3. best-performing code/docs return the highest-scored eligible provider; -4. explicit exhausted providers return no candidate for every capability; -5. `selectDocs("named")` remains compatible when the named provider is eligible; -6. the research wrapper calls `usageCost` with the exact type/result/content-count operation before delegation; -7. Exa metadata returns search, fetch, code-search, and research instances; and -8. existing search filter support, search ordering, and budget tests remain valid. - -Run: - -```bash -pnpm exec vitest run tests/providers/registry.test.ts tests/providers/exa.test.ts tests/providers/exa-deep-research.test.ts -``` - -### Task 2: Centralize one provider-attempt execution path - -**Files:** - -- Modify: `src/providers/execute.ts` -- Modify: `src/providers/fusion.ts` -- Modify: `src/providers/registry.ts` -- Modify: `tests/providers/execute.test.ts` -- Modify: `tests/providers/fusion.test.ts` - -- [ ] **Step 1: Implement `executeAttempt()`.** - -Move the current fallback activity/timing/hook logic into the shared function. Add sanitized activity errors, budget exclusion, and cancellation propagation without changing the aggregate error text. - -- [ ] **Step 2: Delegate sequential fallback to the shared attempt.** - -Keep `executeWithFallback()` responsible only for empty-candidate validation, iteration, error collection, and returning the first successful result/provider name. - -- [ ] **Step 3: Delegate fusion attempts to the shared attempt.** - -Replace the duplicated fusion instrumentation with `executeAttempt()`, pass the signal, and preserve concurrent batches and budget/provider-failure semantics. - -- [ ] **Step 4: Expose registry outcome hooks.** - -Add `getExecutionHooks()` with fresh closures. Keep `recordResultQuality()` separate because it remains search-specific. - -- [ ] **Step 5: Test the attempt lifecycle.** - -Prove: - -1. success creates one activity start/completion pair and one success hook; -2. failure then success creates one error and one completion, with one failure and one success hook; -3. budget rejection is aggregated without a failure hook; -4. pre-attempt and in-flight cancellation are rethrown without a failure hook; -5. activity errors are sanitized; and -6. fusion uses the same hooks while retaining concurrent batching, empty-result success, budget rejection, and `providersFailed` behavior. - -Run: - -```bash -pnpm exec vitest run tests/providers/execute.test.ts tests/providers/fusion.test.ts -``` - -### Task 3: Route direct capability tools through the policy - -**Files:** - -- Modify: `src/tools/web-search.ts` -- Modify: `src/tools/web-fetch.ts` -- Modify: `src/tools/code-search.ts` -- Modify: `src/tools/web-docs-search.ts` -- Modify: `src/tools/web-docs-fetch.ts` -- Modify: `src/tools/web-research.ts` -- Modify: `src/index.ts` -- Modify: `tests/tools/web-search.test.ts` -- Modify: `tests/tools/web-fetch.test.ts` -- Modify: `tests/tools/code-search.test.ts` -- Modify: `tests/tools/web-docs-search.test.ts` -- Modify: `tests/tools/web-docs-fetch.test.ts` -- Modify: `tests/tools/web-research.test.ts` -- Modify: `tests/index.test.ts` -- Modify: `tests/index-strategy.test.ts` - -- [ ] **Step 1: Add shared hook seams to tool factories.** - -Pass one registry hook set from `src/index.ts` to all provider-backed factories. Preserve direct factory-test compatibility where the existing callback arguments are part of the current test/API seam. - -- [ ] **Step 2: Update search and fetch.** - -Pass hooks to both search execution paths and the registered fetch fallback. Preserve Phase 1 filter filtering, result quality recording, current fetch pipeline fallback boundary, sanitized combined errors, cache behavior, multi-URL concurrency, and result shapes. - -- [ ] **Step 3: Update code search and docs tools.** - -Use `executeAttempt()` with operation names `code-search`, `docs-search`, and `docs-fetch`. Preserve missing-provider messages, code-search error-as-text behavior, docs thrown-error behavior, progress updates, truncation, storage, and signal forwarding. - -- [ ] **Step 4: Make research first-class at the tool boundary.** - -Resolve registered research candidates from the registry, construct the same `DeepResearchParams` as today, and run each unique query through shared sequential fallback. Remove `resolveExaApiKey` and `beforeResearch` from the research factory. Do not instantiate `ExaDeepResearchClient` in the tool. - -- [ ] **Step 5: Route all non-search strategies and test integration.** - -Use the current `selectionStrategy` for fetch, code-search, docs, and research. Add index tests that spy on each selector under `auto` and `best-performing`; use configured Exa/Context7 availability where necessary so the conditional tools are exercised. Add tool tests for hook invocation, research fallback, cancellation, and unchanged output/error contracts. - -Run: - -```bash -pnpm exec vitest run tests/tools/web-search.test.ts tests/tools/web-fetch.test.ts tests/tools/code-search.test.ts tests/tools/web-docs-search.test.ts tests/tools/web-docs-fetch.test.ts tests/tools/web-research.test.ts tests/index.test.ts tests/index-strategy.test.ts -``` - -### Task 4: Complete the phase gate and commit - -- [ ] **Step 1: Search for stale direct or duplicate paths.** - -Run: - -```bash -rg -n "activityMonitor\\.log(Start|Complete|Error)|recordOutcome\\(|beforeResearch|resolveExaApiKey|new ExaDeepResearchClient|selectFetchCandidates\\(|selectCodeSearch\\(|selectDocs\\(" src/providers src/tools src/index.ts -``` - -Expected: - -- provider attempt activity lifecycle is implemented in `executeAttempt()`; -- production tools receive registry hooks rather than recording outcomes themselves; -- `beforeResearch` and direct `ExaDeepResearchClient` construction are absent from `web-research.ts` and `index.ts`; -- registry selectors are called with the configured strategy where applicable; and -- fusion contains no second timing/callback implementation. - -- [ ] **Step 2: Run the focused phase gate.** - -```bash -pnpm exec vitest run tests/providers/registry.test.ts tests/providers/execute.test.ts tests/providers/fusion.test.ts tests/providers/exa.test.ts tests/providers/exa-deep-research.test.ts tests/tools/web-search.test.ts tests/tools/web-fetch.test.ts tests/tools/code-search.test.ts tests/tools/web-docs-search.test.ts tests/tools/web-docs-fetch.test.ts tests/tools/web-research.test.ts tests/index.test.ts tests/index-strategy.test.ts -``` - -- [ ] **Step 3: Run repository verification.** - -```bash -pnpm check -git diff --check -git status --short -``` - -Expected in the current environment: 89 test files and 1,427 tests pass; Node 23.11.0 emits the existing package-engine warning; Biome reports only documented pre-existing warnings. Do not fix unrelated warnings or change the engine declaration. - -- [ ] **Step 4: Commit the atomic phase.** - -```bash -git add src/providers src/tools src/index.ts tests -git commit -m "refactor: unify provider operation policy" -``` - -The commit must not include ConfigManager listeners, tool re-registration, extraction fallback changes, new dependencies, or unrelated formatting/lint cleanup. - -## Phase completion gate - -- Tier and performance selection is capability-aware and budget-aware for search, fetch, code search, docs, and research. -- Research is a registered, resolved-key, budget-wrapped provider capability; no tool-local reservation callback remains. -- Fallback, fusion, direct capability calls, and research share one attempt lifecycle. -- Budget rejections and cancellations preserve their non-failure semantics. -- Search fusion concurrency and output behavior are unchanged. -- Existing tool output/error, trust, credential, cache, and storage contracts remain intact. -- Focused tests, `pnpm check`, `git diff --check`, and intended-file status pass. diff --git a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-4-runtime-lifecycle.md b/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-4-runtime-lifecycle.md deleted file mode 100644 index 3e0e43f..0000000 --- a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-4-runtime-lifecycle.md +++ /dev/null @@ -1,348 +0,0 @@ -# Pi Tools Phase 4: Aligned Configuration and Tool Lifecycle Implementation Plan - -> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox syntax for tracking. - -Goal: Make provider refresh, current configuration, tool definitions, and the native OpenAI web-search rewrite share one session lifecycle. - -Architecture: Keep ConfigManager and ProviderRegistry as the existing runtime seams. ConfigManager emits one callback only after a valid semantic config change has been applied. src/index.ts owns one session-local registerTools() closure that consumes the manager’s current config and re-registers the same seven tool names. Pi replaces a tool definition by name, so registration updates guidance and definitions without accumulating duplicates. Provider resolvers and the native rewrite refresh before reading current state; web_search and web_research also receive current-config getters for settings captured at execution time. - -Tech stack: TypeScript, Node isDeepStrictEqual, Pi ExtensionAPI.registerTool, existing config TTL refresh, Vitest. - ---- - -## Readiness decision - -The previous plan was not ready to implement against the clean post-Phase-3 merge at commit 0ba1098. This revision resolves the concrete mismatches found in the repository: - -- Pi’s real registerTool() is name-keyed, but tests/helpers.ts was append-only and could hide duplicate-registration regressions. -- Optional-tool registration behavior is covered by tests/index-research.test.ts; tests/index-guidance.test.ts only tests a guidance utility and is not the right integration target. -- before_provider_request reads configManager.current without refreshing, so the native OpenAI web-search rewrite would remain stale after a config reload. -- web_search captures combineConfig, and web_research captures deepResearchConfig.modeDefaults; registration replacement alone cannot update an already-running definition unless those factories have execution-time getters. -- diffConfig() compares provider structures with order-sensitive JSON.stringify, so the callback needs a semantic equality rule that also prevents false changes from reordered object keys. -- Existing trust and registration tests expect optional tools to be absent. They must be changed to assert stable definitions with unavailable execution when credentials or enablement are missing. -- The current baseline is 89 test files and 1,459 tests. pnpm check exits successfully with the repository’s existing Biome warnings and the environment’s Node 23.11.0 versus the package requirement of Node >=24.15.0. - -## Atomic result - -After this phase: - -- A valid changed config updates the manager’s current config and provider registry before notifying the session. -- Identical semantic config, TTL hits, and malformed reloads do not notify the session. -- Provider enable/disable/key/config changes, selection strategy, guidance, combine settings, and deep-research enablement/mode defaults are visible during the same session. -- web_search, web_fetch, web_read, code_search, web_docs_search, web_docs_fetch, and web_research are registered once at session start and replaced by name after a changed refresh. -- Docs and research tools remain callable definitions when unavailable and return their existing unavailable/disabled errors instead of disappearing from the session. -- The native OpenAI web-search rewrite observes the current openai-web-search configuration after refresh. -- A malformed reload leaves the previous providers, config, and active tool definitions untouched. -- No duplicate active tool names are created by initial registration or refresh. -- Trust gating, credential handling, provider selection algorithms, extraction fallbacks, and existing tool result/error contracts remain unchanged. - -## Runtime flow - -session_start --> ConfigManager loads config and registers providers --> src/index.ts defines registerTools() and registers the seven stable tool names - -tool resolver / /tools reload / before_provider_request --> ConfigManager.refresh() --> strict load + semantic comparison + provider diff --> apply provider changes --> assign current config and TTL --> notify registerTools() when changed - -tool execution / native rewrite --> read current config and current registry state - -## Locked boundaries - -This phase may change: - -- ConfigManager change notification and semantic provider/config comparison. -- Session-local tool registration and current-config getters. -- Stable optional-tool definitions and their integration tests. -- The native OpenAI web-search rewrite’s refresh path. -- Test-host behavior needed to match Pi’s name-keyed registration. - -This phase must not change: - -- Provider selection algorithms, scoring, budgets, or execution hooks introduced in Phase 3. -- Extraction fallback routing or provider seams owned by Phase 5. -- Provider catalog contents, trust rules, credential resolution, SSRF allow ranges, or error sanitization. -- Tool parameter schemas, output formats, or unavailable messages. -- The currently unused deepResearch.outputSchema configuration behavior; this phase only makes already-consumed enabled and modeDefaults values current. -- Package dependencies or Node engine requirements. - -No new runtime coordinator, dependency, or production test accessor is needed. - -## File map - -Modify: - -- src/config-manager.ts — add the optional change listener and semantic change detection. -- src/index.ts — own the refreshable registration closure, stable optional tools, current execution getters, and refresh-aware native rewrite config getter. -- src/tools/web-search.ts — read combine settings through an optional current-config getter while preserving positional callers. -- src/tools/web-research.ts — read enablement and mode defaults through an optional current-config getter while preserving positional callers. -- tests/helpers.ts — model Pi’s same-name replacement behavior. -- tests/config-manager.test.ts — cover listener timing, semantic no-op reloads, non-provider config changes, environment-backed key changes, and malformed reloads. -- tests/index.test.ts — cover stable tool names, guidance replacement, native rewrite refresh, uniqueness, and trust-safe unavailable behavior. -- tests/index-research.test.ts — replace conditional-registration assertions with stable-definition and live availability transitions. -- tests/index-strategy.test.ts — prove selection strategy changes take effect after a forced refresh. -- tests/tools/web-search.test.ts — cover current combine settings through the new getter. -- tests/tools/web-research.test.ts — cover current deep-research enablement and mode defaults through the new getter. - -No changes are required in src/session.ts, tests/session.test.ts, or tests/index-guidance.test.ts: the session handler already accepts a config getter, and the utility-level guidance tests remain valid. - -Create: None. - -Delete: None. - -## Contracts to implement - -### ConfigManager listener and equality - -In src/config-manager.ts: - -type ConfigChangeListener = (config: PiToolsConfig) => void; - -Store an optional listener and append it after the existing optional modelRegistry constructor argument: - -constructor( -cwd: string, -registry: ProviderRegistry, -providerMetas: readonly ProviderMeta[], -modelRegistry?: ModelRegistry, -onChange?: ConfigChangeListener, -) - -The listener is not called during construction. Initial provider registration and initial tool registration remain explicit. - -Use isDeepStrictEqual from node:util for both complete-config comparison and provider-entry structure comparison in diffConfig(). Keep resolved-key comparison separate so an environment- or command-backed key change is detected even when the serialized config is unchanged. - -On a forced or expired refresh: - -1. clear the existing credential cache; -2. load the strict merged config; -3. compute the provider change set; -4. compute providerChanged from the three change-set arrays; -5. compute configChanged as providerChanged || !isDeepStrictEqual(previous, next); -6. apply provider unregister/register changes; -7. assign \_config = next and update the TTL; and -8. call onChange?.(this.\_config) exactly when configChanged is true. - -The malformed-config branch keeps the prior config and provider registry, updates only the retry TTL as it does today, and does not call the listener. TTL hits and semantically identical reloads do not call it. The callback observes the new config and an already-updated registry. - -### Session registration contract - -In src/index.ts, keep the existing store, registry, cache, and session lifecycle. Inside initializeSession: - -1. declare a no-op registerTools variable before constructing ConfigManager; -2. pass () => registerTools() as the listener after ctx.modelRegistry; -3. define all resolver functions and current-config getters without calling refresh from the registration closure; -4. assign one registerTools closure that creates and registers all seven tool definitions; and -5. call registerTools() once after assignment. - -The seven stable names are: - -web_search -web_fetch -web_read -code_search -web_docs_search -web_docs_fetch -web_research - -Docs and research resolvers still return no provider when the current config or registry makes them unavailable. Their definitions are nevertheless registered unconditionally so a later refresh can activate them without a restart. Guidance remains registration-time data; the listener replaces the definition when guidance changes. - -The registration closure must be side-effect free with respect to config refresh. It reads configManager.current; it must not call configManager.refresh() while constructing definitions, preventing callback recursion. - -Pass this refresh-aware getter to handleProviderRequest: - -() => { -configManager.refresh(); -return configManager.current; -} - -Do not add a second config-loading path in src/session.ts. - -### Current execution-config contracts - -In src/index.ts, provide: - -const getCombineConfig = () => { -configManager.refresh(); -return configManager.current.combine; -}; - -const getDeepResearchConfig = () => { -configManager.refresh(); -return configManager.current.deepResearch; -}; - -Keep resolver refreshes and strategy-aware registry selection as they are. The getters and resolvers may both refresh; the TTL makes the second call a no-op. - -In src/tools/web-search.ts, append getCombineConfig?: () => CombineConfig after the existing combineConfig argument. At execution start, use the getter when supplied and use the resulting config for: - -- default combine activation; -- fusion mode; -- target backend count; and -- RRF k. - -If no getter is supplied, preserve the existing captured combineConfig behavior for direct callers and tests. Explicit params.combine continues to override the config’s enabled value. - -In src/tools/web-research.ts, append getDeepResearchConfig?: () => DeepResearchConfig after the existing executionHooks argument. After resolving candidates, use the getter when supplied for: - -- the enabled defense-in-depth check; and -- modeDefaults passed to applyResearchMode. - -If no getter is supplied, preserve the existing captured config behavior. Keep the existing unavailable and disabled errors, query preparation, mode selection, provider request construction, report writing, session entry, and result rendering unchanged. - -### Test-host contract - -In tests/helpers.ts, make createMockPi().registerTool() replace an existing entry with the same tool.name and append only new names. Assert uniqueness after initial session start and after forced refreshes. This matches Pi’s runtime map behavior and exposes accidental duplicate registration. - -## Tasks - -### Task 1: Add precise ConfigManager change notification - -Files: - -- Modify: src/config-manager.ts -- Modify: tests/config-manager.test.ts - -- [ ] Step 1: Add the listener and semantic comparison. - -Add the constructor option and listener field without changing existing constructor call sites. Replace the provider-structure JSON.stringify comparison with isDeepStrictEqual, and use the same comparison for the complete config in refresh(). - -- [ ] Step 2: Notify only after a valid changed reload. - -Apply the provider change set before assigning the new config and invoking the callback. Preserve the malformed reload behavior and the existing TTL behavior. - -- [ ] Step 3: Add focused manager tests. - -Using expireTtlForTest() and the existing mocked loadMergedConfig, prove that: - -1. provider changes notify with the new config; -2. provider registry changes are visible before the listener runs; -3. guidance, default provider, selection strategy, combine, and deep-research changes notify even when providers are unchanged; -4. an environment-backed key transition not represented by a config-object change notifies and re-registers the provider; -5. reordered but semantically equal object keys do not notify or re-register; -6. an identical reload and a TTL hit do not notify; and -7. a thrown strict reload leaves the prior config/providers intact and does not notify. - -Run: - -pnpm exec vitest run tests/config-manager.test.ts - -Expected: focused ConfigManager tests pass. - -### Task 2: Make registration refreshable and optional tools stable - -Files: - -- Modify: src/index.ts -- Modify: src/tools/web-search.ts -- Modify: src/tools/web-research.ts -- Modify: tests/helpers.ts -- Modify: tests/index.test.ts -- Modify: tests/index-research.test.ts -- Modify: tests/tools/web-search.test.ts -- Modify: tests/tools/web-research.test.ts - -- [ ] Step 1: Implement one registration closure. - -Move the existing session-start registration calls into one closure. Register all seven stable names on every session, including docs and research. Keep provider resolvers dynamic and pass the two current-config getters to their factories. Pass the refresh-aware config getter to handleProviderRequest. - -- [ ] Step 2: Match Pi replacement in the mock host. - -Change createMockPi().registerTool() to replace by name. Add a small uniqueness assertion helper or inline assertions; do not add a production registry or test-only production accessor. - -- [ ] Step 3: Preserve direct factory compatibility. - -Append, rather than insert, the optional getter parameters. Existing direct factory tests and legacy callback arguments must continue to compile and retain their behavior when no getter is provided. - -- [ ] Step 4: Verify current combine and research settings. - -Add direct factory tests that change the getter result between executions: - -1. web_search uses newly enabled combine mode/targets/k and still honors explicit combine: false. -2. web_research uses newly changed mode defaults and rejects execution after the getter reports disabled. - -Run: - -pnpm exec vitest run tests/index.test.ts tests/index-research.test.ts tests/tools/web-search.test.ts tests/tools/web-research.test.ts - -Expected: stable registration, replacement, unavailable behavior, and current execution settings pass. - -### Task 3: Prove live transitions, guidance, strategy, native rewrite, and trust behavior - -Files: - -- Modify: tests/index.test.ts -- Modify: tests/index-research.test.ts -- Modify: tests/index-strategy.test.ts - -- [ ] Step 1: Update stable optional-tool expectations. - -Replace tests that expect missing docs/research definitions with tests that always find all seven names. With no Context7/Exa capability, execute the stable definitions and assert their existing unavailable/disabled messages. With trusted literal keys, assert the same names remain present and the configured providers become selectable after refresh. - -- [ ] Step 2: Test same-name replacement and guidance. - -Start with a custom guidance.web_search.promptSnippet, force a /tools reload to a config with a different snippet, and assert: - -1. the current web_search definition has the new snippet; -2. the tool name occurs exactly once; and -3. the old definition is not the active entry in MockPi.tools. - -Use the existing interactive command mock to trigger configManager.refresh(true); do not expose the manager solely for this test. - -- [ ] Step 3: Test current strategy and combine behavior. - -Start with selectionStrategy: auto and combine disabled. After a forced reload to best-performing with combine enabled, assert the next search uses the performance selector and current fusion configuration. Preserve explicit provider and explicit combine overrides. - -- [ ] Step 4: Test native OpenAI rewrite refresh. - -Start with openai-web-search enabled, force a reload with it disabled, and call the existing before_provider_request handler for an OpenAI model. Assert the first payload is rewritten and the post-reload payload is not. This proves the handler reads the manager rather than a stale snapshot. - -- [ ] Step 5: Preserve trust gating. - -For an untrusted project, project credentials, base URLs, browser-cookie settings, and SSRF ranges remain stripped. Stable optional definitions may exist, but their resolvers return unavailable and no sensitive provider is instantiated. Keep the existing trusted-project assertions for configured literal keys. - -Run: - -pnpm exec vitest run tests/index.test.ts tests/index-research.test.ts tests/index-strategy.test.ts - -Expected: runtime transitions, tool replacement, guidance, strategy, combine, native rewrite, and trust tests pass. - -### Task 4: Complete the phase gate and commit - -- [ ] Step 1: Check for stale conditional-registration and append-only patterns. - -Run: - -rg -n "registry\\.selectDocs\\(|researchEnabled|tools\\.push\\(|registerTool\\(" src/index.ts tests/helpers.ts tests/index.test.ts tests/index-research.test.ts - -Expected: optional definitions are registered inside the single session closure, MockPi replaces same-name entries, and no test relies on append-only registration. - -- [ ] Step 2: Run the complete phase gate. - - pnpm exec vitest run tests/config-manager.test.ts tests/index.test.ts tests/index-research.test.ts tests/index-strategy.test.ts tests/tools/web-search.test.ts tests/tools/web-research.test.ts tests/commands/tools.test.ts tests/commands/tools-actions.test.ts tests/commands/tools-dashboard.test.ts tests/session.test.ts - pnpm check - git diff --check - -Expected: focused tests and the full suite pass with only the documented pre-existing Biome and Node-engine warnings. - -- [ ] Step 3: Commit the atomic phase. - - git add src/config-manager.ts src/index.ts src/tools/web-search.ts src/tools/web-research.ts tests - git commit -m "refactor: align config refresh with tool lifecycle" - -The implementation commit must not include provider catalog, provider-operation-policy, extraction, dependency, or unrelated lint changes. - -## Phase completion gate - -- Config changes notify only after valid semantic changes, after provider state is applied. -- Environment-backed credential transitions are detected even when the config object is unchanged. -- Tool definitions are replaced by name, not accumulated. -- All seven tool names are stable at startup; optional tools become available or unavailable from current state without restart. -- Guidance, selection strategy, combine settings, deep-research mode defaults, native OpenAI rewrite behavior, trust, and error messages remain correct. -- Malformed reloads preserve the prior usable config, providers, and tool definitions. -- Focused tests, pnpm check, and git diff --check pass. diff --git a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-5-extraction-seam.md b/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-5-extraction-seam.md deleted file mode 100644 index bd708bb..0000000 --- a/docs/superpowers/plans/2026-08-12-architecture-deepening-phase-5-extraction-seam.md +++ /dev/null @@ -1,207 +0,0 @@ -# Pi Tools Phase 5: Single Content Extraction Seam Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -## Readiness decision - -The previous plan was not ready against the clean post-Phase-4 repository. The current branch is `20260814-architecture-deepening-phase-5-extraction-seam` at `74d988c`; `pnpm check` passes 89 test files and 1,465 tests. The environment still reports Node 23.11.0 versus the package requirement of Node >=24.15.0, and the repository has its existing Biome warnings. - -The gaps were: - -- `pipeline.ts` has no registered-provider options, while `web-fetch.ts` owns a second fallback path only for retryable transport failures. The plan did not specify how the shared adapter handles network/429/5xx failures without accidentally entering Gemini/raw extraction. -- The current pipeline wraps caller cancellation as `RetryableExtractionError`; the plan only described cancellation inside the new adapter. The GET, probe, Cloudflare retry, and transport-fallback boundaries need explicit cancellation checks. -- The draft chain behavior omitted the no-provider marker and claimed to record per-provider failure plus success labels even though `executeWithFallback` returns only the winning provider. Attempt details already belong in activity/metrics; the extraction chain will record only tier-level failure and the winning provider. -- Moving result types without updating type-only consumers would leave the new seam coupled to `pipeline.ts`. The type-only imports need to move while `pipeline.ts` keeps compatibility re-exports. -- Existing Jina tests use content shorter than the planned 100-character provider contract, and existing web-fetch fallback tests cover the old outer path rather than the new in-pipeline path. - -## Goal - -Make registered fetch providers part of one content-extraction fallback seam and remove the duplicate direct Jina Reader implementation. - -## Atomic result - -For a validated HTML response, the normal path is: - -```text -validated HTTP response - → Readability - → RSC - → registered fetch-provider fallback - → Gemini URL context, then Gemini Web - → raw text -``` - -For a retryable direct transport failure (network error, 429, or 5xx), the pipeline tries only the registered fetch-provider fallback and then preserves `RetryableExtractionError` if no provider succeeds. It never enters the Gemini/raw HTML path without a response body. - -Structured routes (GitHub, PDF, YouTube, local video, raw mode), SSRF validation, per-hop redirect validation, cancellation, cache/storage behavior, and sanitized tool errors remain intact. `src/extract/jina-reader.ts` and its duplicate tests are deleted; `src/providers/jina.ts` owns the only `r.jina.ai` request. - -## Locked contracts - -### Fallback seam - -Create `src/extract/fallbacks.ts` with the smallest reusable interface: - -```ts -export interface ExtractionFallback { - name: string; - run(): Promise; -} -``` - -`createFetchProviderFallback()` adapts the already selected, registry-wrapped `FetchProvider[]`. It must: - -- call providers in supplied order through `executeWithFallback({ operation: "fetch" })`; -- trim provider text and treat empty text as a failed attempt; -- return normalized `ExtractedContent` with `fetch-provider:`, the original URL, title, chars, and `truncated: false`; -- pass the supplied `AbortSignal` and `ExecutionHooks` to the shared executor; and -- keep provider errors out of the returned content and extraction chain. - -`runExtractionFallbacks()` checks the caller signal before and after each adapter. A caller-aborted signal is rethrown and never converted into a fallback failure or sent to a later adapter. Ordinary adapter errors advance the chain: - -- `null` result: `:fail`; -- thrown result: `:error`; and -- successful result: append the result’s first extraction-chain marker and return. - -When no fetch providers are supplied, `pipeline.ts` appends `fetch-provider:skipped` and does not create an adapter. When providers are supplied but all fail, the chain contains `fetch-provider:error`; individual attempt failures remain in shared activity/metrics. A first-provider failure followed by a successful second provider records only `fetch-provider:` in the extraction chain. - -Gemini URL context and Gemini Web are one `gemini-html` adapter that tries URL context first and Web second. This preserves the existing high-level markers: the successful adapter contributes `html:gemini-url-context` or `html:gemini-web`, and an all-null result contributes `gemini-html:fail`. - -### Pipeline and transport behavior - -In `extractContent()`: - -1. Check `signal?.throwIfAborted()` before routing and retain the existing frame, video, YouTube, original-URL SSRF, GitHub, probe, direct HTTP, Cloudflare retry, binary, PDF, raw, Readability, and RSC ordering. -2. After original `validateUrl()` succeeds, create the fetch adapter from `options.fetchProviders` but do not invoke it before validation. -3. On a network error from the GET or Cloudflare retry, check caller cancellation, then run only the fetch adapter. On 429/5xx, do the same after the status is known. A successful transport fallback returns its provider-normalized result with its provider-only chain marker, matching the former `web_fetch` fallback result. -4. If no provider succeeds, throw `RetryableExtractionError` based on the original transport failure. An optional provider summary must use `sanitizeError`; it must not expose credentials or raw provider errors. A provider budget rejection is not a failure metric and must not replace the original retryable classification. -5. For a successful non-PDF response whose Readability and RSC tiers fail, run the registered fetch adapter, then the combined Gemini adapter, then raw text. Do not invoke registered providers for raw mode, binary/oversized responses, PDFs, GitHub structured results, YouTube, or local video results. - -The direct GET/Cloudflare catches and the probe catch must rethrow caller cancellation before wrapping ordinary transport errors. SSRF errors remain `SSRFError`; redirect hops remain validated by `fetchValidated()`. - -### Type ownership - -`src/extract/types.ts` owns `ExtractedContent`, `VideoFrame`, and `ImageBlock`. `pipeline.ts` re-exports those types so existing imports remain valid. Type-only consumers move to the new module: `gemini-url-context.ts`, `cache.ts`, `youtube.ts`, `video.ts`, and `frames.ts`. `ExtractOptions` stays in `pipeline.ts` because it describes pipeline routing inputs rather than the extracted result contract. - -### Jina provider - -`JinaProvider.fetch()` remains the only `r.jina.ai` transport. It trims its response and throws `Jina reader returned insufficient content` when the result is shorter than 100 characters. Authorization and search behavior remain unchanged. - -## File map - -Create: - -- `src/extract/types.ts` — shared result types. -- `src/extract/fallbacks.ts` — registered-fetch adapter and ordered fallback runner. -- `tests/extract/fallbacks.test.ts` — seam ordering, normalization, cancellation, hooks, and budget behavior. - -Modify: - -- `src/extract/pipeline.ts` — shared types/options, transport fallback, ordered HTML seam, and cancellation checks. -- `src/extract/gemini-url-context.ts` — import result types from `types.ts`. -- `src/extract/youtube.ts` — import result types from `types.ts`; retain `ExtractOptions` from `pipeline.ts`. -- `src/extract/video.ts` — import result types from `types.ts`; retain `ExtractOptions` from `pipeline.ts`. -- `src/extract/frames.ts` — import `VideoFrame` from `types.ts`. -- `src/cache.ts` — import `ExtractedContent` from `extract/types.ts`. -- `src/providers/jina.ts` — normalize and reject insufficient fetch content. -- `src/tools/web-fetch.ts` — pass candidates/hooks into `extractContent` and remove the outer fallback. -- `tests/extract/pipeline.test.ts` — transport fallback, cancellation, and chain contracts. -- `tests/extract/pipeline-routing.test.ts` — registered-provider ordering before Gemini. -- `tests/extract/pipeline-ssrf.test.ts` — provider never runs before original URL validation. -- `tests/extract/cloudflare-retry.test.ts` — preserve challenge/retry and retryable fallback behavior. -- `tests/providers/jina.test.ts` — fetch success, short-content, non-2xx, and authorization behavior. -- `tests/tools/web-fetch.test.ts` — one pipeline fallback path, hooks, errors, cache, and multi-URL behavior. -- `tests/tools/web-fetch-video.test.ts` — preserve extraction options while passing seam inputs. - -Delete: - -- `src/extract/jina-reader.ts` — duplicate direct Jina transport. -- `tests/extract/jina-reader.test.ts` — tests for the deleted module. - -No changes are required in `src/index.ts` or `tests/helpers.ts`: Phase 3 already supplies registry-wrapped fetch candidates and hooks, and Phase 4’s mock host already replaces same-name tools. - -## Tasks - -### Task 1: Move result types and build the fallback seam - -Files: `src/extract/types.ts`, `src/extract/fallbacks.ts`, `tests/extract/fallbacks.test.ts`, plus the listed type-only consumers. - -- [ ] **Step 1: Move and re-export result types.** Create `types.ts` with the current `ExtractedContent`, `VideoFrame`, and `ImageBlock` definitions. Remove their definitions from `pipeline.ts`, import them there, and re-export them. Update the type-only consumers listed in the file map. -- [ ] **Step 2: Implement the registered-fetch adapter.** Use `executeWithFallback` for every registered attempt. Trim text, reject empty results, preserve title/URL/chars, and pass signal/hooks. -- [ ] **Step 3: Implement the ordered runner.** Enforce signal checks and the `fail`/`error`/success chain contract. Do not expose aggregate provider messages through extraction content. -- [ ] **Step 4: Add seam tests.** Prove supplied ordering, empty-result advancement, normalization, title/chars/url, second-provider success, all-provider failure, caller cancellation stopping later adapters, success/failure hooks, and `BudgetExceededError` exclusion from failure hooks. - -Run: - -```bash -pnpm exec vitest run tests/extract/fallbacks.test.ts -``` - -### Task 2: Wire the seam into the pipeline and remove duplicate Jina - -Files: `src/extract/pipeline.ts`, `src/providers/jina.ts`, `src/extract/jina-reader.ts`, `tests/extract/jina-reader.test.ts`, and the focused pipeline/provider tests. - -- [ ] **Step 1: Add options and a single fetch adapter instance.** Extend `ExtractOptions` with `fetchProviders?: readonly FetchProvider[]` and `executionHooks?: ExecutionHooks`. Create the adapter only after original URL validation so transport and HTML paths share the same registry-wrapped candidates. -- [ ] **Step 2: Preserve direct routes and validation.** Keep the existing route order, HEAD probe, binary/size checks, manual redirect validation, PDF/OCR, raw mode, and Cloudflare retry. Add caller-signal checks at pipeline entry, probe error handling, and GET/Cloudflare catches. -- [ ] **Step 3: Handle retryable transport failures.** On network errors, 429, and 5xx, invoke only the registered fetch adapter. Return its normalized result on success; otherwise preserve `RetryableExtractionError` based on the original failure and sanitize any provider summary. -- [ ] **Step 4: Replace direct Jina/Gemini calls for thin HTML.** After `rsc:no-match`, append `fetch-provider:skipped` when appropriate, run registered providers, then the combined Gemini adapter, then raw text. Remove the `jina-reader` import and markers. -- [ ] **Step 5: Move Jina ownership.** Delete the duplicate extraction module and test. Keep `JinaProvider.fetch()` as the only Reader request, trim successful content, and reject content shorter than 100 characters. - -### Task 3: Update extraction and tool tests around the new boundary - -Files: `tests/extract/pipeline.test.ts`, `tests/extract/pipeline-routing.test.ts`, `tests/extract/pipeline-ssrf.test.ts`, `tests/extract/cloudflare-retry.test.ts`, `tests/providers/jina.test.ts`, `tests/tools/web-fetch.test.ts`, and `tests/tools/web-fetch-video.test.ts`. - -- [ ] **Step 1: Test normal HTML ordering.** With thin HTML and mocked Gemini adapters, assert Readability/RSC precede registered providers, providers precede Gemini, a successful provider stops the chain, and all-failure markers are sanitized/high-level only. -- [ ] **Step 2: Test retryable transport behavior.** Cover network, 429, and 5xx success through a registered provider; all-provider failure as `RetryableExtractionError`; no Gemini/raw attempt without a response body; and budget rejection without a failure metric. -- [ ] **Step 3: Test cancellation.** Abort before and during direct transport and during provider fallback. Assert the caller’s abort is preserved, no later provider runs, and no provider failure hook records cancellation. -- [ ] **Step 4: Test security and structured routes.** Verify the provider is not called for an invalid original URL, redirect-to-private-IP, binary/oversized response, PDF, GitHub, YouTube, local video, or raw mode. Keep per-hop redirect validation assertions. -- [ ] **Step 5: Preserve web-fetch behavior.** Verify the single extraction call receives resolved candidates/hooks, provider fallback results use the same cache/content-store/rendering path, and multi-URL deduplication, concurrency, partial failures, fresh mode, image blocks, and sanitized errors remain unchanged. -- [ ] **Step 6: Update Jina tests.** Use a valid long fetch fixture, assert trim and title-independent text, short-content rejection, non-2xx errors, and authorization headers for both search and Reader fetches. - -Run: - -```bash -pnpm exec vitest run \ - tests/extract/fallbacks.test.ts \ - tests/extract/pipeline.test.ts \ - tests/extract/pipeline-routing.test.ts \ - tests/extract/pipeline-ssrf.test.ts \ - tests/extract/cloudflare-retry.test.ts \ - tests/providers/jina.test.ts \ - tests/tools/web-fetch.test.ts \ - tests/tools/web-fetch-video.test.ts -``` - -### Task 4: Complete the phase gate - -- [ ] **Step 1: Confirm no duplicate transport/outer fallback remains.** Search `src` and tests for `extractViaJinaReader`, `jina-reader`, `r.jina.ai`, and `executeWithFallback`. `r.jina.ai` must remain only in `src/providers/jina.ts` and its provider tests; `web-fetch.ts` must not execute a second fetch fallback. -- [ ] **Step 2: Run the full gate.** - -```bash -pnpm check -git diff --check -git status --short -``` - -Expected: the focused tests and full suite pass with only the documented Biome and Node-engine warnings. `git status --short` contains only the intended Phase 5 files. - -- [ ] **Step 3: Commit the atomic phase.** - -```bash -git add \ - src/cache.ts \ - src/extract/types.ts src/extract/fallbacks.ts src/extract/pipeline.ts \ - src/extract/gemini-url-context.ts src/extract/youtube.ts src/extract/video.ts src/extract/frames.ts \ - src/extract/jina-reader.ts src/providers/jina.ts src/tools/web-fetch.ts \ - tests/extract/fallbacks.test.ts tests/extract/pipeline.test.ts tests/extract/pipeline-routing.test.ts \ - tests/extract/pipeline-ssrf.test.ts tests/extract/cloudflare-retry.test.ts tests/extract/jina-reader.test.ts \ - tests/providers/jina.test.ts tests/tools/web-fetch.test.ts tests/tools/web-fetch-video.test.ts -git commit -m "refactor: unify content extraction fallbacks" -``` - -## Phase completion gate - -- Registered fetch providers are the only provider-backed extraction adapters. -- Direct Jina Reader transport is deleted from `src/extract`. -- Fallback order, retryable transport behavior, budgets, hooks, cancellation, SSRF, redirects, and structured routes are covered by tests. -- Cache, content storage, multi-URL, image, and rendering behavior remain intact. -- Focused tests, `pnpm check`, and `git diff --check` pass. diff --git a/docs/superpowers/plans/2026-08-12-architecture-deepening.md b/docs/superpowers/plans/2026-08-12-architecture-deepening.md deleted file mode 100644 index 25f361e..0000000 --- a/docs/superpowers/plans/2026-08-12-architecture-deepening.md +++ /dev/null @@ -1,99 +0,0 @@ -# Pi Tools Architecture Deepening Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Deepen all five reviewed architectural seams in `@pi-vault/pi-tools` through independently mergeable phases ordered from simplest to most complex. - -**Architecture:** Keep `src/extract/pipeline.ts` as the deep content-extraction module, but make its external fallback adapters explicit. Make provider metadata the catalog consumed by configuration, registration, and the dashboard. Concentrate capability selection, fallback/fusion execution, budgets, activity, and outcome recording behind the existing registry/execution seams, then align those seams with a refreshable session runtime. - -**Tech Stack:** TypeScript, Node native `fetch`, Pi extension APIs, TypeBox, Vitest, Biome, pnpm. - ---- - -## Execution order - -Implement and merge these phase plans in order: - -1. [Phase 1 — explicit search filter semantics](2026-08-12-architecture-deepening-phase-1-filters.md) -2. [Phase 2 — single provider catalog](2026-08-12-architecture-deepening-phase-2-catalog.md) -3. [Phase 3 — capability-aware provider operation policy](2026-08-12-architecture-deepening-phase-3-operation-policy.md) -4. [Phase 4 — aligned configuration and tool lifecycle](2026-08-12-architecture-deepening-phase-4-runtime-lifecycle.md) -5. [Phase 5 — single content extraction seam](2026-08-12-architecture-deepening-phase-5-extraction-seam.md) - -Each phase must be a separate mergeable change. A phase is complete only when its focused tests and `pnpm check` pass. Do not combine phase commits to hide a failing intermediate state. - -## Cross-phase invariants - -Keep these behaviors unchanged unless the phase plan explicitly names the change: - -- SSRF validation still runs before direct external HTTP fetches, every redirect hop is validated, and trust gating still strips sensitive project configuration. -- Hard-budget usage is reserved and persisted before provider delegation. A failed persistence write rolls usage back. `BudgetExceededError` is not a provider failure. -- Provider errors still use the existing fallback and `AggregateProviderError` semantics. -- Tool result error sanitization remains in place. -- `web_fetch` still supports one URL, multiple URLs, raw mode, PDFs, GitHub URLs, YouTube, local video, cache, session content storage, and `web_read` retrieval. -- No new package or SDK is added. Use existing TypeScript, native `fetch`, and test helpers. -- Do not split a file solely because it is long. A new file is justified only when it owns a concrete interface at a seam. -- Do not change the Node engine declaration or clean unrelated Biome warnings. - -## Baseline and verification - -Before each phase, run: - -```bash -pnpm check -``` - -Expected baseline in the current environment: - -- 87 test files pass. -- 1,410 tests pass. -- Biome reports pre-existing warnings. -- pnpm reports the environment is Node 23.11.0 while `package.json` declares Node `>=24.15.0`. - -The implementation environment should use Node `>=24.15.0` when available. If the environment remains on Node 23, record the engine warning but do not change the project requirement as part of this work. - -At every phase gate run the focused command listed in that phase plan, then: - -```bash -pnpm check -git diff --check -git status --short -``` - -`git status --short` must contain only intended phase files before committing. - -## Rollout and rollback - -Use one branch or worktree per phase. Merge only after the phase gate passes. If a phase must be reverted, revert its phase commit as a whole; do not manually undo only half of a seam change. The previous phase must continue to compile and pass its tests after a later phase is reverted. - -## Phase contracts - -### Phase 1 result - -Requested search domain/date filters are either enforced natively, applied through a safe local domain post-filter, or rejected with an explicit unsupported-filter error. A provider that silently ignores a requested filter is no longer eligible for that filtered call. Provider ordering is unchanged for unfiltered calls. - -### Phase 2 result - -Provider identity facts, fallback environment names, and built-in provider defaults have one catalog source. `config.ts`, `ConfigManager`, `allProviders`, and the dashboard consume that catalog. User config remains the higher-priority override. - -### Phase 3 result - -Search, fetch, code-search, docs, and research use coherent capability-aware selection/execution rules. Search `auto` retains tier ordering. Performance selection can be applied to every capability that has metrics. Fallback and fusion share activity and outcome recording. - -### Phase 4 result - -The session has one refreshable runtime source. Provider refresh and tool definitions use the same current config. Optional docs/research definitions are registered at session start so later key/config changes are usable; their execution checks current availability. Changed guidance is re-registered through Pi’s name-replacing `registerTool` path. - -### Phase 5 result - -`web_fetch` passes registered fetch adapters into one extraction fallback seam. The extraction order remains direct HTTP/structured extraction, registered fetch adapters, Gemini fallbacks, and raw text. The duplicate direct Jina implementation is removed; Jina transport is owned by the provider adapter. - -## Completion checklist - -- [ ] Phase 1 merged and its focused filter tests pass. -- [ ] Phase 2 merged and catalog/config/registration tests pass. -- [ ] Phase 3 merged and provider policy tests pass. -- [ ] Phase 4 merged and runtime refresh/tool registration tests pass. -- [ ] Phase 5 merged and extraction/fallback/SSRF tests pass. -- [ ] Final `pnpm check` passes with only documented pre-existing warnings. -- [ ] README or changelog updates are made only where a user-visible behavior or configuration contract changed; no architecture essay is added. diff --git a/docs/superpowers/plans/2026-08-12-tinyfish-provider.md b/docs/superpowers/plans/2026-08-12-tinyfish-provider.md deleted file mode 100644 index e8a587c..0000000 --- a/docs/superpowers/plans/2026-08-12-tinyfish-provider.md +++ /dev/null @@ -1,457 +0,0 @@ -# TinyFish Search + Fetch Provider Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Add TinyFish as a tier-2 provider for the existing `web_search` and `web_fetch` tools. - -**Architecture:** Add one dedicated `TinyFishProvider` using Node's native `fetch` and TinyFish's canonical Search and Fetch REST endpoints. Register it through pi-tools' existing `ProviderMeta`/`ProviderRegistry` seams so fallback, fusion, budgets, dashboard discovery, credentials, and automatic selection require no new infrastructure. This is not a Pi `ExtensionAPI.registerProvider()` model-provider integration; Pi continues to load the unchanged pi-tools extension and its existing `web_search`/`web_fetch` tools. - -**Tech Stack:** TypeScript, native `fetch`, Vitest, Biome, existing Pi Tools provider/configuration APIs. - ---- - -## File map - -- Create `src/providers/tinyfish.ts`: TinyFish Search/Fetch HTTP client and `ProviderMeta`. -- Create `tests/providers/tinyfish.test.ts`: mocked TinyFish request/response contract tests. -- Modify `src/providers/all.ts`: include TinyFish in the provider registry barrel. -- Modify `src/config.ts`: add the default provider entry and `TINYFISH_API_KEY` fallback mapping. -- Modify `tests/providers/all.test.ts`: update provider count/name expectations. -- Modify `tests/config.test.ts`: update built-in budget and environment mapping expectations. -- Modify `tests/config-manager.test.ts`: verify the resolved TinyFish key registers both capabilities through the real metadata path. -- Modify `README.md`: document TinyFish setup, capabilities, and configuration. -- Modify `CHANGELOG.md`: add the Unreleased provider entry. - -## Decisions locked by the design - -- V1 supports TinyFish Search and Fetch only; Agent and Browser APIs are out of scope. -- Provider name is `tinyfish`; display label is `TinyFish`. -- Tier is `2`, so configured TinyFish participates in automatic selection after tier-1 providers. -- API key is required and resolves through `TINYFISH_API_KEY`. -- Built-in budget is `{ mode: "unlimited" }`; TinyFish documents Search and Fetch as non-credit APIs, while upstream rate-limit errors remain ordinary provider failures. -- Search uses the documented first page and slices results locally to `maxResults`; no pagination changes are introduced. -- Existing single-URL `FetchProvider.fetch()` is retained even though TinyFish supports batch Fetch. -- No SDK, dependency, endpoint override, retry layer, new tool, generic adapter change, or Pi core change. - -### Task 1: Add failing TinyFish provider contract tests - -**Files:** - -- Create: `tests/providers/tinyfish.test.ts` - -- [ ] **Step 1: Add metadata and Search request tests** - -Use the existing `stubFetch` helper and assert the exact public contract: - -```ts -import { afterEach, beforeEach, describe, expect, it } from "vitest"; -import { - TinyFishProvider, - providerMeta, -} from "../../src/providers/tinyfish.ts"; -import { stubFetch } from "../helpers.ts"; - -describe("TinyFishProvider", () => { - let fetchStub: ReturnType; - - beforeEach(() => { - fetchStub = stubFetch(); - }); - - afterEach(() => { - fetchStub.restore(); - }); - - it("has the expected metadata", () => { - const provider = new TinyFishProvider("key"); - expect(provider.name).toBe("tinyfish"); - expect(provider.label).toBe("TinyFish"); - expect(providerMeta).toMatchObject({ - name: "tinyfish", - tier: 2, - requiresKey: true, - }); - expect(providerMeta.create("key").search).toBeDefined(); - expect(providerMeta.create("key").fetch).toBeDefined(); - }); - - it("sends the API key and maps Search filters", async () => { - fetchStub.addResponse("api.search.tinyfish.ai", { - body: { results: [] }, - }); - - await new TinyFishProvider("tiny-key").search("pi tools", 5, undefined, { - includeDomains: ["github.com", "docs.example.com"], - excludeDomains: ["spam.example.com"], - startDate: "2026-01-01", - endDate: "2026-08-12", - }); - - const [input, init] = (globalThis.fetch as any).mock.calls[0]; - const url = new URL(input as string); - expect(url.origin + url.pathname).toBe("https://api.search.tinyfish.ai/"); - expect(url.searchParams.get("query")).toBe("pi tools"); - expect(url.searchParams.get("include_domains")).toBe( - "github.com,docs.example.com", - ); - expect(url.searchParams.get("exclude_domains")).toBe("spam.example.com"); - expect(url.searchParams.get("after_date")).toBe("2026-01-01"); - expect(url.searchParams.get("before_date")).toBe("2026-08-12"); - expect(init.headers["X-API-Key"]).toBe("tiny-key"); - expect(init.headers.Accept).toBe("application/json"); - }); -}); -``` - -- [ ] **Step 2: Add Search normalization and failure tests** - -Add tests for the documented `{ results: [{ title, snippet, url }] }` response: - -- return normalized `SearchResult` values; -- discard non-HTTP URLs and cap output to `maxResults`; -- throw an error containing `TinyFish search error` for a 429 response. - -Use a fixture with two valid results and one `javascript:` result, call `search("test", 1)`, and assert exactly one valid result is returned. - -- [ ] **Step 3: Add Fetch request, response, and partial-error tests** - -Add tests asserting: - -```ts -fetchStub.addResponse("api.fetch.tinyfish.ai", { - body: { - results: [ - { - url: "https://example.com", - title: "Example", - text: "# Example\n\nFetched markdown", - format: "markdown", - }, - ], - errors: [], - }, -}); - -const result = await new TinyFishProvider("tiny-key").fetch( - "https://example.com", -); -expect(result).toEqual({ - text: "# Example\n\nFetched markdown", - title: "Example", - contentType: "text/markdown", -}); - -const [input, init] = (globalThis.fetch as any).mock.calls[0]; -expect(input).toBe("https://api.fetch.tinyfish.ai"); -expect(init.method).toBe("POST"); -expect(JSON.parse(init.body)).toEqual({ - urls: ["https://example.com"], - format: "markdown", -}); -expect(init.headers["X-API-Key"]).toBe("tiny-key"); -``` - -Also test that a response with no matching result and an `errors` entry throws an error containing that URL/error, and that a non-2xx response throws `TinyFish fetch error`. - -- [ ] **Step 4: Add abort-signal forwarding test** - -Call `search` with an `AbortController.signal`, inspect the mocked fetch call, and assert `init.signal` is the same signal object. - -- [ ] **Step 5: Run the focused tests and confirm they fail** - -Run: - -```bash -pnpm exec vitest run tests/providers/tinyfish.test.ts -``` - -Expected: collection/import failure because `src/providers/tinyfish.ts` does not yet exist. - -### Task 2: Implement the native TinyFish provider - -**Files:** - -- Create: `src/providers/tinyfish.ts` -- Test: `tests/providers/tinyfish.test.ts` - -- [ ] **Step 1: Implement response guards and URL normalization** - -Add local helpers without changing shared provider types: - -```ts -function isRecord(value: unknown): value is Record { - return typeof value === "object" && value !== null && !Array.isArray(value); -} - -function validHttpUrl(value: unknown): value is string { - if (typeof value !== "string") return false; - try { - const url = new URL(value); - return url.protocol === "http:" || url.protocol === "https:"; - } catch { - return false; - } -} -``` - -Use these helpers when converting unknown JSON into local provider results. Do not use `any` in production code. - -- [ ] **Step 2: Implement Search with native fetch** - -Build the request against `https://api.search.tinyfish.ai` using `URL`/`URLSearchParams`: - -```ts -const url = new URL("https://api.search.tinyfish.ai"); -url.searchParams.set("query", query); -if (filters?.includeDomains?.length) { - url.searchParams.set("include_domains", filters.includeDomains.join(",")); -} -if (filters?.excludeDomains?.length) { - url.searchParams.set("exclude_domains", filters.excludeDomains.join(",")); -} -if (filters?.startDate) url.searchParams.set("after_date", filters.startDate); -if (filters?.endDate) url.searchParams.set("before_date", filters.endDate); - -const response = await fetch(url, { - headers: { Accept: "application/json", "X-API-Key": this.apiKey }, - signal, -}); -if (!response.ok) { - throw new Error( - `TinyFish search error: ${response.status} ${response.statusText}`, - ); -} -``` - -Parse `results` only when it is an array. Map string title/snippet values to empty strings when absent, discard records whose URL is not HTTP(S), and return `.slice(0, maxResults)`. - -- [ ] **Step 3: Implement Fetch with native fetch** - -Send the documented Markdown request: - -```ts -const response = await fetch("https://api.fetch.tinyfish.ai", { - method: "POST", - headers: { - Accept: "application/json", - "Content-Type": "application/json", - "X-API-Key": this.apiKey, - }, - body: JSON.stringify({ urls: [url], format: "markdown" }), - signal, -}); -if (!response.ok) { - throw new Error( - `TinyFish fetch error: ${response.status} ${response.statusText}`, - ); -} -``` - -For the existing single-URL interface, use the first usable result. TinyFish documents `results[].url` as the original requested URL and `final_url` as the post-redirect URL, so no redirect matching or URL rewriting is needed in the local `FetchResult`. Return its string `text`, optional string `title`, and `contentType: "text/markdown"`. - -If there is no usable result, inspect `errors` for `errors[].url === url` and throw `TinyFish fetch error: `; otherwise throw `TinyFish fetch error: no result for `. Preserve the caller's signal and do not retry. - -- [ ] **Step 4: Add provider metadata** - -Export: - -```ts -export const providerMeta: ProviderMeta = { - name: "tinyfish", - tier: 2, - requiresKey: true, - create: (key) => { - const provider = new TinyFishProvider(key!); - return { search: provider, fetch: provider }; - }, -}; -``` - -- [ ] **Step 5: Run focused tests and typecheck** - -Run: - -```bash -pnpm exec vitest run tests/providers/tinyfish.test.ts -pnpm exec tsc --noEmit -``` - -Expected: both commands pass. - -### Task 3: Register TinyFish and add configuration defaults - -**Files:** - -- Modify: `src/providers/all.ts` -- Modify: `src/config.ts` -- Modify: `tests/providers/all.test.ts` -- Modify: `tests/config.test.ts` -- Modify: `tests/config-manager.test.ts` - -- [ ] **Step 1: Register the provider barrel entry** - -Add: - -```ts -import { providerMeta as tinyfish } from "./tinyfish.ts"; -``` - -and append `tinyfish` to `allProviders`. - -- [ ] **Step 2: Add credential fallback mapping** - -Add to `FALLBACK_ENV_MAP`: - -```ts -tinyfish: "TINYFISH_API_KEY", -``` - -This preserves the existing `resolveProviderKey()` fallback path. `ConfigManager` registers the provider from `DEFAULT_CONFIG.providers.tinyfish.apiKey` and resolves that configured environment-variable name through `resolveApiKey()`; the fallback map is kept consistent for callers that use the provider-name lookup path. - -- [ ] **Step 3: Add built-in provider defaults** - -Add to `DEFAULT_CONFIG.providers`: - -```ts -tinyfish: { - enabled: true, - budget: { mode: "unlimited" }, - apiKey: "TINYFISH_API_KEY", -}, -``` - -- [ ] **Step 4: Update provider barrel tests** - -Update `tests/providers/all.test.ts` to: - -- change the expected provider count from `22` to `23`; -- add `tinyfish` to the expected provider names; -- keep the custom-cost provider list unchanged because TinyFish uses the default unit cost behavior. - -- [ ] **Step 5: Update config tests** - -Update `tests/config.test.ts` to: - -- expect `tinyfish: { mode: "unlimited" }` in the built-in budget map; -- add `tinyfish` to the expected fallback provider list; -- assert `FALLBACK_ENV_MAP.tinyfish === "TINYFISH_API_KEY"`. - -- [ ] **Step 6: Cover ConfigManager registration with the real TinyFish metadata** - -Extend `tests/config-manager.test.ts` with a focused test that: - -- loads a config containing an enabled `tinyfish` entry whose `apiKey` is `TINYFISH_API_KEY`; -- stubs `resolveApiKey("TINYFISH_API_KEY")` to return a test key; -- constructs `ConfigManager` with the real TinyFish `providerMeta` and a fresh `ProviderRegistry`; -- asserts `registerProvider` receives the TinyFish name, tier-2 policy, and instances containing both `search` and `fetch`. - -This verifies the runtime registration path used during Pi session startup without calling TinyFish. - -- [ ] **Step 7: Run registration/config tests** - -Run: - -```bash -pnpm exec vitest run tests/providers/all.test.ts tests/config.test.ts tests/config-manager.test.ts tests/providers/tinyfish.test.ts -``` - -Expected: all tests pass. - -### Task 4: Document TinyFish setup and release notes - -**Files:** - -- Modify: `README.md` -- Modify: `CHANGELOG.md` - -- [ ] **Step 1: Add the provider capability row** - -Add to the README provider table: - -```markdown -| TinyFish | Search, fetch | `TINYFISH_API_KEY` | -``` - -- [ ] **Step 2: Add TinyFish to the configuration example** - -Add to the full configuration reference: - -```json -"tinyfish": { - "enabled": true, - "apiKey": "TINYFISH_API_KEY" -} -``` - -Add a short setup note near the provider documentation explaining that TinyFish uses the Search and Fetch REST APIs and requires an API key. - -- [ ] **Step 3: Add the Unreleased changelog entry** - -Under `## [Unreleased]`, add: - -```markdown -### Added - -- Added TinyFish Search and Fetch provider support through `TINYFISH_API_KEY`. -``` - -If an `### Added` subsection already exists, append to it instead of duplicating it. - -- [ ] **Step 4: Check documentation whitespace** - -Run: - -```bash -git diff --check -``` - -Expected: no whitespace errors. Markdown is intentionally excluded from this repository's Biome configuration; TypeScript remains covered by `pnpm check`. - -### Task 5: Run full verification and self-review - -**Files:** - -- Verify all files changed by Tasks 1–4. - -- [ ] **Step 1: Run the complete project check** - -Run: - -```bash -pnpm check -``` - -Expected: Biome lint, TypeScript, and all Vitest tests pass. - -- [ ] **Step 2: Verify the package contents without publishing** - -Run: - -```bash -pnpm pack:dry-run -``` - -Expected: the package includes `src/providers/tinyfish.ts`, tests are excluded according to the existing package configuration, and no new dependency is listed. - -- [ ] **Step 3: Review the final diff** - -Run: - -```bash -git diff --check -git status --short -git diff -- src/providers/tinyfish.ts src/providers/all.ts src/config.ts tests/providers/tinyfish.test.ts tests/providers/all.test.ts tests/config.test.ts tests/config-manager.test.ts README.md CHANGELOG.md -``` - -Confirm that only the planned provider, tests, configuration, and documentation files changed; there are no lockfile or dependency changes. - -- [ ] **Step 4: Complete the implementation handoff** - -Report the passing commands and note that live TinyFish API verification was not required. Do not claim live connectivity or upstream availability based solely on mocked tests. - -## Self-review result - -- Spec coverage: Search, Fetch, authentication, tier, budget, config, fallback, ConfigManager registration, normalization, errors, cancellation, tests, docs, and verification are each assigned to a task. -- Placeholder scan: no unfinished markers or unspecified implementation steps remain. -- Type consistency: `TinyFishProvider` implements the existing `SearchProvider` and `FetchProvider` methods; `providerMeta.create()` returns the capabilities expected by `ProviderMeta`. -- Scope check: one provider integration only; TinyFish Agent/Browser work is explicitly excluded. diff --git a/docs/superpowers/specs/2026-08-12-architecture-deepening-design.md b/docs/superpowers/specs/2026-08-12-architecture-deepening-design.md deleted file mode 100644 index 45f3fa1..0000000 --- a/docs/superpowers/specs/2026-08-12-architecture-deepening-design.md +++ /dev/null @@ -1,124 +0,0 @@ -# Pi Tools Architecture Deepening Design - -## Status - -Approved design for a phased implementation plan. The work covers all five candidates from the whole-repository architecture review, ordered from simplest to most complex. Each phase is independently mergeable, has a usable result, preserves unrelated public behavior, and leaves the full test suite passing. - -## Objective - -Improve the highest-leverage seams in `@pi-vault/pi-tools` without turning the review into a broad rewrite. The implementation will concentrate policy, remove duplicated provider knowledge, and make runtime behavior observable through stable interfaces. - -The work will use existing TypeScript types, native `fetch`, the current provider registry, and Vitest. It will not add dependencies or split modules solely because they are long. - -## Current friction - -The review found five related opportunities: - -1. `SearchProvider` accepts domain and date filters even though several adapters ignore them silently. -2. Provider identity is repeated across provider metadata, configuration defaults, environment mappings, and dashboard discovery. -3. Provider selection, fallback/fusion execution, budget reservation, activity logging, and outcome recording have different paths for different capabilities. -4. `ConfigManager` refreshes provider instances during a session while optional tool registration, guidance, and combine settings are captured at session start. -5. Content extraction calls Jina and Gemini fallbacks directly while `web_fetch` has a separate registered-provider fallback; Jina has duplicate implementations. - -The deletion test rules out splitting `src/extract/pipeline.ts` merely because it is 548 lines: its small interface hides substantial routing, validation, and fallback behavior. The improvement belongs at the external adapter seam, not in arbitrary file subdivision. - -## Target architecture - -The intended dependency direction is: - -```text -config - ↓ -provider catalog - ↓ -provider registry - ↓ -provider operation policy - ↓ -search / fetch / docs / research tools - -web_fetch - ↓ -content extraction module - ↓ -ordered extraction adapters - ↓ -provider operation policy -``` - -The provider catalog owns built-in provider facts. User configuration remains the override implementation. The provider registry continues to own budget reservation and registered adapters. The provider operation policy owns capability-aware selection and execution behavior. The extraction module remains deep: it owns routing order, validation, and result shaping while external extraction adapters cross one explicit seam. - -## Locked design decisions - -- Five phases are required; none may be skipped because a later phase appears more valuable. -- Phase order is dependency-first and simplest-to-most-complex: - 1. search filter semantics; - 2. provider catalog; - 3. provider operation policy; - 4. configuration/tool lifecycle; - 5. content extraction seam. -- Every phase is independently mergeable and rollbackable. -- Every phase has a focused test gate and runs the full existing test suite before completion. -- Phase 1 does not change provider ordering or selection strategy. -- Phase 2 keeps user overrides in configuration and moves only built-in provider facts into the catalog. -- Phase 3 preserves current tier ordering for `auto`; `best-performing` becomes capability-aware. Budget reservation still happens before provider delegation. -- Phase 4 aligns tool availability and guidance with current session state without assuming that the host can unregister a tool dynamically. -- Phase 5 preserves the existing extraction order and Gemini fallback behavior while giving external adapters one seam and removing duplicate Jina behavior. -- No new dependency, SDK, generic framework, or speculative abstraction is part of the work. - -## Phase contracts - -### Phase 1 — explicit search filter semantics - -Usable result: a caller can determine whether requested domain/date filters are enforced. Each adapter either honors the filter, uses a safe central post-filter, or reports the unsupported condition explicitly. Silent filter loss is removed. Existing provider ordering and fallback behavior stay unchanged. - -### Phase 2 — single provider catalog - -Usable result: adding a provider’s identity no longer requires editing several independent metadata lists. The catalog supplies identity, tier, credential mapping, and built-in defaults to configuration, registration, and the dashboard. Project and global user overrides remain authoritative. - -### Phase 3 — capability-aware provider operation policy - -Usable result: all capabilities use one coherent selection and execution policy. Search keeps tier ordering under `auto`; other capabilities no longer accidentally rely on registration order when the selected strategy requires more. Fallback and fusion share activity and outcome recording, and fetch outcomes can feed performance metrics. - -### Phase 4 — aligned configuration and tool lifecycle - -Usable result: a refreshed configuration has one runtime source of truth. Provider availability, guidance, combine settings, and optional docs/research tool behavior no longer depend on stale session-start snapshots. The implementation remains compatible with a host that only supports registration at session initialization. - -### Phase 5 — single content extraction seam - -Usable result: `web_fetch` still follows its established extraction order, but Jina, Gemini, and registered fetch adapters are selected through one explicit extraction policy. Jina transport and result shaping have one implementation. External fallback behavior is covered through the seam rather than through duplicated direct calls. - -## Data flow and error rules - -- Configuration errors retain the current safe behavior: malformed non-strict configuration falls back where it currently does; strict refresh keeps the prior usable configuration when refresh fails. -- Budget errors remain `BudgetExceededError` and are not recorded as provider failures. -- Provider failures continue through `AggregateProviderError` and existing fallback semantics. -- Unsupported search filters must be explicit and must not silently claim a constrained result set. -- Runtime refresh must not make an already running tool use a partially updated configuration. -- Extraction must preserve SSRF validation, binary/size checks, redirect validation, cancellation, and existing `RetryableExtractionError` behavior. -- No phase may weaken trust gating, credential handling, or error sanitization. - -## Testing strategy - -Each phase plan will use the existing test style and add the smallest regression surface that proves its seam: - -- contract tests for filter support and unsupported behavior; -- catalog/config/registry integration tests for a provider’s single source of identity; -- selection, fallback, fusion, budget, activity, and metrics tests through the operation seam; -- post-refresh runtime tests for tool availability and current guidance/configuration; -- extraction routing, Jina deduplication, Gemini fallback, cancellation, and SSRF regression tests through the extraction seam. - -The baseline command is `pnpm check`. The current repository passes 87 test files and 1,410 tests, with pre-existing lint warnings and a Node engine warning because the environment is Node 23.11.0 while the package declares Node >=24.15.0. The plan will not include unrelated lint cleanup or an engine change. - -## Planned documents - -After this spec is reviewed, the implementation plan will be split into one main plan and five phase plans: - -- `docs/superpowers/plans/2026-08-12-architecture-deepening.md` -- `docs/superpowers/plans/2026-08-12-architecture-deepening-phase-1-filters.md` -- `docs/superpowers/plans/2026-08-12-architecture-deepening-phase-2-catalog.md` -- `docs/superpowers/plans/2026-08-12-architecture-deepening-phase-3-operation-policy.md` -- `docs/superpowers/plans/2026-08-12-architecture-deepening-phase-4-runtime-lifecycle.md` -- `docs/superpowers/plans/2026-08-12-architecture-deepening-phase-5-extraction-seam.md` - -The main plan will define cross-phase invariants, sequencing, rollout, rollback, and verification. Each phase plan will define its file map, atomic result, implementation tasks, tests, error rules, and completion gate. diff --git a/docs/superpowers/specs/2026-08-12-architecture-deepening-phase-1-filters-design.md b/docs/superpowers/specs/2026-08-12-architecture-deepening-phase-1-filters-design.md deleted file mode 100644 index 910c35c..0000000 --- a/docs/superpowers/specs/2026-08-12-architecture-deepening-phase-1-filters-design.md +++ /dev/null @@ -1,166 +0,0 @@ -# Phase 1: Explicit Search Filter Semantics - -## Status - -Approved revised design after repository review. The existing Phase 1 implementation plan is close but must be revised with the decisions and gates below before implementation starts. - -## Review outcome - -The repository is clean on branch `20260812-architecture-deepening-phase-1-filters`. The baseline `pnpm check` passes with 87 test files and 1,410 tests; the existing Biome warnings and Node 23 engine warning remain documented pre-existing conditions. - -The current `SearchProvider` accepts domain and date filters without declaring whether an adapter enforces them. Several adapters ignore one or both groups. The original plan therefore addresses the right seam, but it was not implementation-ready because it did not specify the shared constant location, deterministic unsupported-group aggregation, cancellation at the early rejection path, runtime metadata-preservation tests, a production support-matrix test, or the user-facing README change. - -## Goal - -Prevent filtered searches from silently losing requested domain or date constraints. - -For every filtered search, each eligible provider must either enforce every requested filter natively, declare safe local domain post-filtering, or be excluded before execution. If no candidate can enforce the request, `web_search` returns an explicit structured error. - -## Non-goals - -- Change provider ordering, selection strategy, budgets, fallback order, or fusion behavior. -- Add result top-up requests after local post-filtering. -- Redesign provider registration or introduce a general policy framework. -- Change invalid-date handling already performed by `buildFilters`. -- Clean unrelated lint warnings or modify the Node engine requirement. - -## Filter-support contract - -`src/providers/types.ts` owns the contract and its shared values: - -```ts -export type SearchFilterMode = "native" | "post-filter" | "unsupported"; - -export interface SearchFilterSupport { - readonly domains: SearchFilterMode; - readonly dates: Exclude; -} - -export const NATIVE_DOMAIN_DATE_FILTERS: SearchFilterSupport = { - domains: "native", - dates: "native", -}; - -export const NATIVE_DOMAIN_FILTERS: SearchFilterSupport = { - domains: "native", - dates: "unsupported", -}; - -export const POST_FILTER_DOMAIN_FILTERS: SearchFilterSupport = { - domains: "post-filter", - dates: "unsupported", -}; - -export const UNSUPPORTED_SEARCH_FILTERS: SearchFilterSupport = { - domains: "unsupported", - dates: "unsupported", -}; -``` - -`SearchProvider` requires `readonly filterSupport: SearchFilterSupport`. This makes every production adapter and typed test double declare its filter policy. The HTTP adapter accepts optional metadata for configuration convenience and defaults to `UNSUPPORTED_SEARCH_FILTERS`, so an omitted declaration cannot silently become permissive. - -The registry copies `filterSupport` into its budget-wrapped search provider. It preserves the metadata but does not apply filter logic; `web_search` remains the policy boundary. - -## Production support matrix - -| Providers | Domains | Dates | -| --- | --- | --- | -| Brave Search, Exa, Serper, TinyFish | native | native | -| Tavily | native | unsupported | -| DuckDuckGo | post-filter | unsupported | -| Brave LLM, fastCRW, LangSearch, Linkup, Marginalia, Perplexity, WebSearchAPI, You.com, Firecrawl, Jina, Ollama, OpenAI Codex, OpenAI web search, Parallel, SearXNG, Sofya | unsupported | unsupported | - -Native means the provider adapter sends the provider-supported domain/date parameters or query syntax. `post-filter` means the provider may narrow its query, but returned URLs are centrally checked by exact host/subdomain matching before they are exposed. Unsupported means the candidate is excluded from a request that asks for that group. - -## Domain post-filter - -Add `filterResultsByDomains(results, filters)` and `unsupportedSearchFilters(filters, support)` to `src/utils/filters.ts`. - -Domain matching parses each result URL with the native `URL` class, lowercases the hostname, trims surrounding dots from the requested domain, and accepts only an exact host or a subdomain. Thus `docs.example.com` matches `example.com`, while `notexample.com` does not. Invalid result URLs do not match an include or exclude domain. - -The post-filter applies only when `support.domains === "post-filter"`. Native providers are not locally re-filtered, preserving their upstream ranking and semantics. - -## `web_search` data flow - -1. Resolve candidates through the existing selection and combine strategy. -2. If the original list is empty, preserve the existing `Search error: No search providers available` result. -3. Build the existing normalized filters. -4. Evaluate every candidate with `unsupportedSearchFilters`. Keep candidates with no unsupported groups in their original order. Collect the union of unsupported groups in deterministic `domains`, then `dates` order. -5. If no candidates remain, honor an already-aborted signal before returning an explicit unsupported-filter result. Do not call providers, reserve budget, create activity entries, or invoke success/failure/result callbacks. -6. Execute the eligible candidates through the existing fallback or fusion executor. Each provider call is wrapped so a post-filter-domain provider applies `filterResultsByDomains` to its returned results. -7. Preserve the existing executor behavior: ordinary fallback accepts a successful empty result, while fusion treats an empty result as unusable and continues. Do not make additional requests to refill results removed by local filtering. - -Unfiltered searches pass the original candidate list and follow the current execution and formatting paths unchanged. - -## Error behavior - -When the original candidate list is non-empty but every candidate is ineligible, return: - -```ts -{ - content: [{ - type: "text", - text: "Search error: no eligible provider supports requested filters: domains, dates", - }], - details: { - provider: "none", - resultCount: 0, - unsupportedFilters: ["domains", "dates"], - }, -} -``` - -The message lists only the requested groups that prevented eligibility. Unsupported candidates are not provider failures, so they do not affect activity, failure metrics, or fallback error aggregation. Provider failures, budget errors, cancellation handling, callbacks, and fusion metadata remain governed by the existing executors. - -## Test design - -- Contract tests cover exact host and subdomain inclusion, exclusion without lookalike matches, malformed URLs, unsupported-group detection, and native/post-filter combinations. -- A production support-matrix test asserts the declared policy for every registered search provider. -- HTTP-adapter tests cover custom metadata and the unsupported default. -- Registry tests assert that the budget wrapper preserves `filterSupport` at runtime. -- `web-search` tests cover unsupported providers not being called, explicit structured errors, partial eligibility, post-filter results, fusion candidate exclusion, early cancellation, and unchanged unfiltered behavior. -- All existing typed test fixtures update to provide `UNSUPPORTED_SEARCH_FILTERS` explicitly; no casts are added to bypass the contract. - -## Files in scope - -Modify the existing Phase 1 implementation file map: - -- `src/providers/types.ts` -- `src/utils/filters.ts` -- `src/providers/http-adapter.ts` -- `src/providers/http-providers.ts` -- `src/providers/duckduckgo.ts` -- `src/providers/exa.ts` -- `src/providers/tavily.ts` -- `src/providers/serper.ts` -- `src/providers/tinyfish.ts` -- `src/providers/firecrawl.ts` -- `src/providers/jina.ts` -- `src/providers/ollama.ts` -- `src/providers/openai-codex.ts` -- `src/providers/openai-web-search.ts` -- `src/providers/parallel.ts` -- `src/providers/searxng.ts` -- `src/providers/sofya.ts` -- `src/providers/registry.ts` -- `src/tools/web-search.ts` -- the existing typed search-provider fixtures and focused tests -- `README.md` - -Create: - -- `tests/providers/filter-support.test.ts` - -No Phase 2–5 files are in scope. - -## Verification gate - -Run the focused provider and tool tests, then: - -```bash -pnpm check -git diff --check -git status --short -``` - -The phase is ready to merge only when all focused tests and the full suite pass, the diff contains only intended Phase 1 changes, and the README accurately describes native, post-filtered, and explicitly unsupported filter behavior. diff --git a/package.json b/package.json index 164c304..5588610 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@pi-vault/pi-tools", - "version": "0.5.0", + "version": "0.5.1", "type": "module", "description": "Pi extension for web search, content extraction, documentation lookup, and deep research tools", "author": "Lanh Hoang ", diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 5dcd505..1a6b9d1 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -1,5 +1,3 @@ allowBuilds: "@google/genai": true protobufjs: true -minimumReleaseAgeExclude: - - typebox@1.3.13