[aryasaatvik]

Phase 1 of Tier-2 tool-catalog caching — stacked on #49.

Caches executor.tools.list (today a full tool scan + tool_policy read on every call) behind a content-derived revision key, so the list stays warm and only re-reads D1 when the catalog or policies actually change. No TTL.

Invalidation

• produceConnectionTools writes a per-connection toolset_revision = sha256 of the connection's manifest rows' (path, index_fingerprint), in plugin_storage (owner-scoped, no schema migration), at all 5 catalog-mutation sites (produce, both clear branches, connection-remove, integration-remove).
• catalogGen combines the in-scope connections' revisions; policyGen combines the tool_policy rows. Key = sha256(owner, integration, connection, includeBlocked, catalogGen, policyGen).
• Token rotation does not bust (never calls produceConnectionTools); add/remove/change a tool, or any policy edit, does.

Scope

• Cached value = the DB-derived, policy-filtered list. Static tools unioned live (never cached); query applied post-cache so distinct queries share one cached set.
• Hand-authored ToolListItem Schema (list projection only — no input/output schema); branded ids stored as strings, rebuilt on read.

Notes (intentional, single-tenant self-host)

• catalogGen/policyGen reads are tenant-wide → per-tenant-coarse invalidation (a catalog change re-warms all owners' list caches). Correct, just coarse.

Gate

tsgo (sdk) · oxlint --deny-warnings · oxfmt · vitest sdk 350/350 (new cache suite: add/remove/content invalidation, policy bust, query post-cache, static-union-on-hit).

Phase 2 (stacked) moves the UI list onto the compact manifest tier.

Stack

1. perf(sdk): precompile tool-schema TypeScript during indexing #49
2. perf(sdk): cache tools.list by catalog + policy revision #50 👈 current

[greptile-apps]

Greptile Summary

Adds a content-addressed KV cache for executor.tools.list to replace the per-call full tool scan. The cache key is derived from a policyGen (SHA-256 of all tool_policy rows) and catalogGen (SHA-256 of per-connection toolset_revision rows written to plugin_storage by produceConnectionTools), so no TTL is needed and the cache stays warm through token rotation. Static tools are always unioned live, and the query substring filter is applied post-cache so distinct queries share one cached entry.

• Cache invalidation is handled at all 5 mutation sites (produce, two clear branches, connection-remove, integration-remove); (owner, key) is included in the sort so the all-owners revision read is deterministic (addresses the previous cross-owner thread concern).
• Intentional coarseness: catalogGen and policyGen are tenant-wide reads, so any catalog or policy change busts all in-scope list caches — acknowledged in the PR notes. Cache entries are never evicted (no TTL), so stale orphans accumulate; the platform's KV eviction or a future Phase 2 cleanup would address this.

Confidence Score: 5/5

Safe to merge; the caching layer is best-effort (errors fall back to a live DB read) and the invalidation logic is transactional at every mutation site.

All five catalog-mutation sites correctly write or delete the plugin_storage revision row inside the same transaction as the tool writes. The policyGen/catalogGen computation is deterministic after the (owner, key) sort fix from the previous thread. Cache hits and misses both produce identical output for the same filter; the test suite covers add/remove invalidation, content-change invalidation, policy bust, query post-cache sharing, and static tool live union. No correctness issues found.

No files require special attention.

Important Files Changed

Filename	Overview
packages/core/sdk/src/tool-list-cache.ts	New module defining the schema, key derivation, and constants for the tools.list KV cache; version-locked schema, deterministic key via sha256, and well-named constants. No issues.
packages/core/sdk/src/executor.ts	Core caching logic added: revision computation in produceConnectionTools, revision cleanup at all 5 mutation sites, and the revamped toolsList that computes catalogGen/policyGen live then uses KV as a best-effort cache. Previous cross-owner and error-swallow concerns addressed (per thread replies).
packages/core/sdk/src/executor.test.ts	Adds 5 targeted cache tests: add/remove invalidation, content-change invalidation, policy bust, query post-cache sharing, and static-tool live union. All test the right invariants; no gaps in coverage for the stated scope.
.changeset/tools-list-revision-cache.md	Minor changeset for @executor-js/sdk accurately describing the caching behaviour, invalidation triggers, and what remains live.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Caller
    participant toolsList
    participant D1 as D1 (tool_policy / plugin_storage)
    participant KV as KV Cache

    Caller->>toolsList: tools.list(filter)
    toolsList->>D1: "findMany(tool_policy, {})"
    D1-->>toolsList: policyRows → policyGen (sha256)
    toolsList->>D1: "findMany(plugin_storage, {plugin_id=catalog-revision, owner?})"
    D1-->>toolsList: revisionRows → catalogGen (sha256)
    toolsList->>toolsList: "key = sha256(owner, integration, connection, includeBlocked, catalogGen, policyGen)"
    toolsList->>KV: get(key)

    alt Cache HIT
        KV-->>toolsList: ToolListCacheEntry
        toolsList->>toolsList: "dbTools = entry.tools.map(cachedItemToTool)"
    else Cache MISS
        KV-->>toolsList: Option.none
        toolsList->>D1: "findMany(tool, {integration, owner, connection})"
        D1-->>toolsList: toolRows
        toolsList->>toolsList: apply policy filter → dbTools
        toolsList->>KV: "set(key, {version, tools: dbTools}) [Effect.ignore]"
    end

    toolsList->>toolsList: union staticTools (live, no cache)
    toolsList->>toolsList: apply query filter post-cache
    toolsList-->>Caller: Tool[]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Caller
    participant toolsList
    participant D1 as D1 (tool_policy / plugin_storage)
    participant KV as KV Cache

    Caller->>toolsList: tools.list(filter)
    toolsList->>D1: "findMany(tool_policy, {})"
    D1-->>toolsList: policyRows → policyGen (sha256)
    toolsList->>D1: "findMany(plugin_storage, {plugin_id=catalog-revision, owner?})"
    D1-->>toolsList: revisionRows → catalogGen (sha256)
    toolsList->>toolsList: "key = sha256(owner, integration, connection, includeBlocked, catalogGen, policyGen)"
    toolsList->>KV: get(key)

    alt Cache HIT
        KV-->>toolsList: ToolListCacheEntry
        toolsList->>toolsList: "dbTools = entry.tools.map(cachedItemToTool)"
    else Cache MISS
        KV-->>toolsList: Option.none
        toolsList->>D1: "findMany(tool, {integration, owner, connection})"
        D1-->>toolsList: toolRows
        toolsList->>toolsList: apply policy filter → dbTools
        toolsList->>KV: "set(key, {version, tools: dbTools}) [Effect.ignore]"
    end

    toolsList->>toolsList: union staticTools (live, no cache)
    toolsList->>toolsList: apply query filter post-cache
    toolsList-->>Caller: Tool[]

Loading

_{Reviews (4): Last reviewed commit: "fix(sdk): scope tools.list catalogGen by..." | Re-trigger Greptile}