Skip to content

feat(prefill): add maintenance cache saturation harness - #169

Merged
FluffyAIcode merged 1 commit into
mainfrom
AgentMemory/kv-cache-saturation-harness-0712
Jul 12, 2026
Merged

feat(prefill): add maintenance cache saturation harness#169
FluffyAIcode merged 1 commit into
mainfrom
AgentMemory/kv-cache-saturation-harness-0712

Conversation

@FluffyAIcode

Copy link
Copy Markdown
Owner

Summary

  • capture canonical first-append token IDs in a bounded in-memory queue, excluding maintenance replay sessions
  • expose API-key-protected status/drain endpoints without persisting prompts or tokens
  • add a staged replay driver for per-node 90%, 95%, and LRU churn targets with memory/fallback/publish safety stops
  • expose cache evictions, bytes evicted, put failures, publish attempts/failures, and bytes published through capability cards, API, and dashboard
  • emit privacy-safe JSON reports containing only salted capture IDs, counts, timings, and capacity metrics

Capacity semantics

The harness controls head 1 GiB and allens 8 GiB independently. Resident fleet usage cannot exceed 9 GiB; the churn stage verifies that excess logical writes increase eviction counters while resident bytes remain bounded.

Tests

  • platform-neutral Python selection — 827 passed, 1 skipped
  • focused capture/API/coordinator tests — 14 passed
  • eviction/publish/capability/network tests — 39 passed
  • TypeScript SDK — 54 passed, 100% coverage and typecheck
  • proto lint, canonical format, generated stub drift, Python compile and diff checks pass

Maintenance execution

After merge, deploy with --cache-fill-capture-size 256, feed live gRPC first appends, run scripts/fill_prefill_cache_from_live_grpc.py, then validate staged fill, LRU churn, and remote replay before disabling capture.

Made with Cursor

Capture canonical first-append tokens in bounded memory, replay them with privacy-safe reporting, and expose eviction and publish metrics so operators can fill each node to staged thresholds and verify LRU churn safely.

Co-authored-by: Cursor <cursoragent@cursor.com>
@FluffyAIcode
FluffyAIcode merged commit 08983f9 into main Jul 12, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant