Skip to content

fix(agents): reuse deterministic interactive Prefill KV - #183

Merged
FluffyAIcode merged 1 commit into
mainfrom
AgentMemory/repl-prefill-reuse-0717
Jul 17, 2026
Merged

fix(agents): reuse deterministic interactive Prefill KV#183
FluffyAIcode merged 1 commit into
mainfrom
AgentMemory/repl-prefill-reuse-0717

Conversation

@FluffyAIcode

Copy link
Copy Markdown
Owner

Summary

  • remove random run nonce from interactive Generator and Critic model prompts
  • centralize deterministic prompt construction for exact KV identity across repeated tasks
  • compact system instructions while retaining open-problem honesty and evidence-window constraints
  • reduce default Critic evidence budget from 128 to 64 content tokens
  • retain random IDs only for benchmark telemetry, outside model tokenization

Measured prompt reduction

For 证明黎曼猜想 with a long Generator response:

  • Generator Prefill: 93 → 40 tokens (57% reduction)
  • Critic Prefill: 311 → 176 tokens (43% reduction)
  • repeated identical tasks become eligible for exact allens/Primary KV hits because tokenized prompts no longer contain random entropy

The first unseen prompt remains bounded by the 16GB allens Gemma Prefill rate; this change removes unnecessary repeated cold work without violating Primary decode-only mode.

Tests

  • deterministic prompt identity and retained safety instructions
  • Agent GAN targeted suite — 13 passed
  • full platform-neutral suite — 859 passed, 1 skipped
  • Python compile and diff checks pass

Made with Cursor

Remove per-run prompt entropy and compact Agent GAN templates so repeated tasks can hit allens KV while new Critic contexts require substantially fewer Prefill tokens.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursor

cursor Bot commented Jul 17, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@FluffyAIcode
FluffyAIcode merged commit e60e4ee into main Jul 17, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant