feat(s53): prompt-cache/memory program — fill the empty Cache tab + cache-stability fixes - #11
Open
drwhofan2k18-pixel wants to merge 4 commits into
Conversation
…ents, pricing module - DuplicateTurnError contract: both SqliteTurnStore and InMemoryTurnStore throw DuplicateTurnError on (conversationId, turnIndex) collision - Perf samples: samplingStrategy field + sampling-based perf tracking - Runtime: new exports (newGameSession, getGameSessions, getGameState, loadFromSnapshot, getSystemPressure) - Events: context-handler enhancements (getTokenCost, getSummarizationStats) - Pricing module: token cost estimation with provider-specific rates - Recall: quality gate option support - Dedup L1: segment-level deduplication support - CI: install-script-block guard step - Dashboard client: expanded api/client exports Co-Authored-By: opencode <noreply@opencode.ai>
…y-status API S53A: Prompt-cache telemetry - classifyPrefixBreak() labels breaks as recall|compaction|inject|other using ±2s tolerance against recall/compaction/inject timestamps - /api/provider-cache accepts ?since=&until= query params - CacheStatsResponseSchema includes prefixBreaks[] array S53B: Embedder cache + memory status - TrigramEmbedder FIFO cache (256 entries), defensive copies on hit - MEGACOMPACT_EMBED_CACHE=0 disables cache - memoryStats() computes totals, 30-day recall window, top-N stable - /api/memory-status endpoint returns MemoryStatusResponse - MemoryEffectivenessCard + ProviderCacheCard dashboard components - Dashboard API client exports memoryStatus, providerCache, sendAction - reset-runtime interface expanded with S53B/S54 properties - perf-samples: samplingStrategy field + sampling-based tracking Tests: 11 new tests (4 embedder-cache + 7 memoryStats + 3 endpoint + 1 prefix-telemetry + 2 cache-server prefixBreaks), 904 total pass. Co-Authored-By: opencode <noreply@opencode.ai>
…issue TheArchitectit#10) The `files` allowlist globbed whole directories (dist/, extensions/, src/) that contain compiled *.test.js + *.map, leaking ~180 test files + 15 dashboard source maps into the published package (1.8 MB / 7.1 MB unpacked). Added `!` negation patterns to `files` so the exclusions compose with the allowlist (npm ignores `.npmignore` when `files` is set, so the exclusions must live here). Verified via `npm pack --dry-run`: - test files in pack: 186 → 0 - dashboard .map files: 15 → 0 - tarball: 1.8 MB → 939 kB; unpacked: 7.1 MB → 3.2 MB - all runtime files (mega-compact entry, dashboard index.html, dist runtime) still present. Closes TheArchitectit#10. Gate green: build + lint + 902 tests / 0 fail.
…y) + gap-analysis refresh Closes the remaining S53 sub-sprints identified in the gap analysis (docs/BRANCH_GAP_ANALYSIS_REFRESH.md). The Cache-tab display work (A/B) and S53B memory were already on this branch; this finishes the cache-STABILITY fixes that raise the provider prompt-cache hit rate, plus the flare regression the earlier C work introduced. S53-D — cache-stability (the hit-rate fix): - RECOMPACT_PCT_DELTA 10 → 50, now config-driven + env-overridable (MEGACOMPACT_RECOMPACT_PCT_DELTA). The v0.8.6 value of 10 rebuilt the prefix on every 10% window growth, suppressing the hit rate toward the 55–60% observed. Replaying the cached trim view up to 50% growth keeps the KV prefix stable. (context-handler.ts + mega-config.ts.) - Skip-path fallback (D7): when runCompact returns skipped, fall back to the cached trim replay when trimCache is still valid — never to the full transcript. A skip previously returned nothing → pi sent the untrimmed transcript → guaranteed cache miss. This is the v0.8.6 replay-cache design intent; the skip path was the one place defeating it. S53-C/D5 — MEGA CACHE flare gate: - widget.ts gated the dedup flare on cachePct >= 100. After S53-C, cachePct is the bounded 0–100 provider prompt-cache pct (practically never 100), silently killing the flare. Both flare sites (minimal + full TUI) now gate on megaCacheFlarePct (the dedup hit rate that armed the flare, which legitimately exceeds 100%). widget.test.ts fixture updated to arm megaCacheFlarePct. docs: BRANCH_GAP_ANALYSIS_REFRESH.md — refreshes the 07-29 analysis (only feature/promptcache-stats moved, with planning docs only), verifies all claims against current master, and assesses the in-flight S53A/B/C/D implementation. Gate green: 925 pass / 0 fail (93 files), build + lint + regression clean.
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the S53 prompt-cache/memory program: surfaces real provider prompt-cache metrics (filling the empty Cache tab), fixes the cache-stability bugs that suppress the hit rate, and lands the two open issues from the QA pass. The empty Cache tab on v0.11.0 is now fixed end-to-end.
This branch also carries the S53A/S53B work (prefix-break classification, embedder cache,
memoryStats(),/api/memory-status+/api/provider-cache).What this PR delivers
Issues (from the QA/burn pass)
b672d7a): bothSqliteTurnStoreandInMemoryTurnStorenow throw a typedDuplicateTurnErroron a duplicate(conversationId, turnIndex). Previously SQLite threw a rawERR_SQLITE_ERROR(leaky) and InMemory silently stored a second row — the contract diverged. Shared compliance suite now asserts it on both backends.641199e):!-negation patterns infiles[]cut the package from 1.8 MB → 939 kB (0 compiled.test.js, 0 leaked source maps)..npmignoredoesn't apply whenfilesis set, so exclusions live inpackage.json.S53 — Cache tab + memory (display + cost visibility)
readProviderCacheStats()+/api/provider-cacheendpoint (lifetime aggregates, not the 30-min rolling window).ProviderCacheCard+MemoryEffectivenessCard;pricing.tscomputes $ saved frominputRatePerToken(cacheRead×0.9 save, cacheWrite×0.25 premium).memoryStats()+/api/memory-status;TrigramEmbedderFIFO cache (256,MEGACOMPACT_EMBED_CACHE).CacheTab.tsxnow fetches/api/provider-cachealongside the dedup cards — fixes the empty-Cache-tab root cause (it previously read/api/snapshot= dedup stats = zeros).S53-D — cache-STABILITY (the hit-rate fix) (
9430368)RECOMPACT_PCT_DELTA10 → 50 (env-overridable:MEGACOMPACT_RECOMPACT_PCT_DELTA). The v0.8.6 value of 10 rebuilt the KV prefix on every 10% window growth — the root cause of the observed 55–60% hit rate. Replaying the cached trim view up to 50% growth stabilizes the prefix.trimCacheis valid — never to the full transcript (which was a guaranteed cache miss).S53-C/D5 — MEGA CACHE flare gate (
9430368)widget.tsgated the dedup flare oncachePct >= 100. After S53-C,cachePctis the bounded 0–100 provider prompt-cache pct, which silently killed the flare. Both flare sites now gate onmegaCacheFlarePct(the dedup hit rate that armed the flare, which legitimately exceeds 100%).Docs
docs/BRANCH_GAP_ANALYSIS_REFRESH.md— refreshes the 07-29 branch analysis (onlyfeature/promptcache-statsmoved, with planning docs only), verifies all claims against current master, and assesses the S53 implementation.Verification gate (all green)
Notes
opencodeon this branch; S53-C/D5 + the two issues + the analysis are my additions on top.readProviderCacheStatsvs spec'sreadProviderCacheLifetime;pricing.tswas struck in the spec but built anyway — functionally fine).