Skip to content

feat(s53): prompt-cache/memory program — fill the empty Cache tab + cache-stability fixes - #11

Open
drwhofan2k18-pixel wants to merge 4 commits into
TheArchitectit:masterfrom
drwhofan2k18-pixel:s53-cache-memory-completion
Open

feat(s53): prompt-cache/memory program — fill the empty Cache tab + cache-stability fixes#11
drwhofan2k18-pixel wants to merge 4 commits into
TheArchitectit:masterfrom
drwhofan2k18-pixel:s53-cache-memory-completion

Conversation

@drwhofan2k18-pixel

Copy link
Copy Markdown
Contributor

Summary

Completes the S53 prompt-cache/memory program: surfaces real provider prompt-cache metrics (filling the empty Cache tab), fixes the cache-stability bugs that suppress the hit rate, and lands the two open issues from the QA pass. The empty Cache tab on v0.11.0 is now fixed end-to-end.

This branch also carries the S53A/S53B work (prefix-break classification, embedder cache, memoryStats(), /api/memory-status + /api/provider-cache).

What this PR delivers

Issues (from the QA/burn pass)

S53 — Cache tab + memory (display + cost visibility)

  • Sub-sprint A ✅ — readProviderCacheStats() + /api/provider-cache endpoint (lifetime aggregates, not the 30-min rolling window).
  • Sub-sprint B ✅ — ProviderCacheCard + MemoryEffectivenessCard; pricing.ts computes $ saved from inputRatePerToken (cacheRead×0.9 save, cacheWrite×0.25 premium).
  • S53B memory ✅ — memoryStats() + /api/memory-status; TrigramEmbedder FIFO cache (256, MEGACOMPACT_EMBED_CACHE).
  • CacheTab.tsx now fetches /api/provider-cache alongside the dedup cards — fixes the empty-Cache-tab root cause (it previously read /api/snapshot = dedup stats = zeros).

S53-D — cache-STABILITY (the hit-rate fix) (9430368)

  • RECOMPACT_PCT_DELTA 10 → 50 (env-overridable: MEGACOMPACT_RECOMPACT_PCT_DELTA). The v0.8.6 value of 10 rebuilt the KV prefix on every 10% window growth — the root cause of the observed 55–60% hit rate. Replaying the cached trim view up to 50% growth stabilizes the prefix.
  • Skip-path fallback (D7): a skipped compaction now falls back to the cached trim replay when trimCache is valid — never to the full transcript (which was a guaranteed cache miss).

S53-C/D5 — MEGA CACHE flare gate (9430368)

  • widget.ts gated the dedup flare on cachePct >= 100. After S53-C, cachePct is the bounded 0–100 provider prompt-cache pct, which silently killed the flare. Both flare sites now gate on megaCacheFlarePct (the dedup hit rate that armed the flare, which legitimately exceeds 100%).

Docs

  • docs/BRANCH_GAP_ANALYSIS_REFRESH.md — refreshes the 07-29 branch analysis (only feature/promptcache-stats moved, with planning docs only), verifies all claims against current master, and assesses the S53 implementation.

Verification gate (all green)

npm test           → 925 pass / 0 fail across 93 files
npm run lint       → clean (tsc + guardrails + semantic)
regression_check   → clean
npm pack --dry-run → 939 kB (was 1.8 MB), 0 test files leaked

Notes

  • S53A/S53B were authored by opencode on this branch; S53-C/D5 + the two issues + the analysis are my additions on top.
  • The local implementation diverges slightly from the upstream spec naming (readProviderCacheStats vs spec's readProviderCacheLifetime; pricing.ts was struck in the spec but built anyway — functionally fine).
  • Still out of scope (per spec, future sprints): recall-injection relocation (systemPrompt → tail user-role), DB-mirror high-water mark (O(n²) hot path), PLAN_V2 Phases 2–4.

drwhofan2k18-pixel and others added 4 commits July 30, 2026 15:49
…ents, pricing module

- DuplicateTurnError contract: both SqliteTurnStore and InMemoryTurnStore
  throw DuplicateTurnError on (conversationId, turnIndex) collision
- Perf samples: samplingStrategy field + sampling-based perf tracking
- Runtime: new exports (newGameSession, getGameSessions, getGameState,
  loadFromSnapshot, getSystemPressure)
- Events: context-handler enhancements (getTokenCost, getSummarizationStats)
- Pricing module: token cost estimation with provider-specific rates
- Recall: quality gate option support
- Dedup L1: segment-level deduplication support
- CI: install-script-block guard step
- Dashboard client: expanded api/client exports

Co-Authored-By: opencode <noreply@opencode.ai>
…y-status API

S53A: Prompt-cache telemetry
- classifyPrefixBreak() labels breaks as recall|compaction|inject|other
  using ±2s tolerance against recall/compaction/inject timestamps
- /api/provider-cache accepts ?since=&until= query params
- CacheStatsResponseSchema includes prefixBreaks[] array

S53B: Embedder cache + memory status
- TrigramEmbedder FIFO cache (256 entries), defensive copies on hit
- MEGACOMPACT_EMBED_CACHE=0 disables cache
- memoryStats() computes totals, 30-day recall window, top-N stable
- /api/memory-status endpoint returns MemoryStatusResponse
- MemoryEffectivenessCard + ProviderCacheCard dashboard components
- Dashboard API client exports memoryStatus, providerCache, sendAction
- reset-runtime interface expanded with S53B/S54 properties
- perf-samples: samplingStrategy field + sampling-based tracking

Tests: 11 new tests (4 embedder-cache + 7 memoryStats + 3 endpoint +
  1 prefix-telemetry + 2 cache-server prefixBreaks), 904 total pass.

Co-Authored-By: opencode <noreply@opencode.ai>
…issue TheArchitectit#10)

The `files` allowlist globbed whole directories (dist/, extensions/, src/) that
contain compiled *.test.js + *.map, leaking ~180 test files + 15 dashboard
source maps into the published package (1.8 MB / 7.1 MB unpacked). Added `!`
negation patterns to `files` so the exclusions compose with the allowlist (npm
ignores `.npmignore` when `files` is set, so the exclusions must live here).

Verified via `npm pack --dry-run`:
- test files in pack: 186 → 0
- dashboard .map files: 15 → 0
- tarball: 1.8 MB → 939 kB; unpacked: 7.1 MB → 3.2 MB
- all runtime files (mega-compact entry, dashboard index.html, dist runtime)
  still present.

Closes TheArchitectit#10. Gate green: build + lint + 902 tests / 0 fail.
…y) + gap-analysis refresh

Closes the remaining S53 sub-sprints identified in the gap analysis
(docs/BRANCH_GAP_ANALYSIS_REFRESH.md). The Cache-tab display work (A/B) and
S53B memory were already on this branch; this finishes the cache-STABILITY
fixes that raise the provider prompt-cache hit rate, plus the flare regression
the earlier C work introduced.

S53-D — cache-stability (the hit-rate fix):
- RECOMPACT_PCT_DELTA 10 → 50, now config-driven + env-overridable
  (MEGACOMPACT_RECOMPACT_PCT_DELTA). The v0.8.6 value of 10 rebuilt the prefix
  on every 10% window growth, suppressing the hit rate toward the 55–60%
  observed. Replaying the cached trim view up to 50% growth keeps the KV prefix
  stable. (context-handler.ts + mega-config.ts.)
- Skip-path fallback (D7): when runCompact returns skipped, fall back to the
  cached trim replay when trimCache is still valid — never to the full
  transcript. A skip previously returned nothing → pi sent the untrimmed
  transcript → guaranteed cache miss. This is the v0.8.6 replay-cache design
  intent; the skip path was the one place defeating it.

S53-C/D5 — MEGA CACHE flare gate:
- widget.ts gated the dedup flare on cachePct >= 100. After S53-C, cachePct is
  the bounded 0–100 provider prompt-cache pct (practically never 100), silently
  killing the flare. Both flare sites (minimal + full TUI) now gate on
  megaCacheFlarePct (the dedup hit rate that armed the flare, which legitimately
  exceeds 100%). widget.test.ts fixture updated to arm megaCacheFlarePct.

docs: BRANCH_GAP_ANALYSIS_REFRESH.md — refreshes the 07-29 analysis (only
feature/promptcache-stats moved, with planning docs only), verifies all claims
against current master, and assesses the in-flight S53A/B/C/D implementation.

Gate green: 925 pass / 0 fail (93 files), build + lint + regression clean.
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant