Skip to content

fix(core): mark prompt-cache breakpoints for Claude models - #349

Merged
davemooreuws merged 1 commit into
mainfrom
claude-prompt-caching
Oct 11, 2026
Merged

davemooreuws merged 1 commit into
mainfrom
claude-prompt-caching

Conversation

@tjholm

@tjholm tjholm commented Oct 9, 2026

Copy link
Copy Markdown
Member

Fixes SUGA-1955

Breakpoints

  • System text (covers the tools), the end of a turn's history, and each step's newest message
  • The history mark is PromptMessage.cacheBreakpoint, set by modelPrompt before the trailing [Turn] message; kept in the turn checkpoint
  • The step mark moves rather than accumulates, since the SDK carries prepareStep messages forward and Anthropic allows four

Providers

  • ProviderDialect.cacheBreakpoint: Anthropic always, OpenRouter for anthropic/* models
  • Everyone else's requests are unchanged

Checked locally (OpenRouter, claude-haiku-5.5)

  • Later turns read 95–99% of the prompt from cache; cost per turn about 1/6 of uncached
  • A turn after one with tool calls re-writes a few hundred tokens of history; left for a follow-up

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
sugabots-website f55bf08 Oct 09 2026, 02:34 AM

@tjholm
tjholm force-pushed the claude-prompt-caching branch from 18f9aab to f55bf08 Compare October 9, 2026 02:33
@davemooreuws
davemooreuws merged commit 863c3b6 into main Oct 11, 2026
7 checks passed
@davemooreuws
davemooreuws deleted the claude-prompt-caching branch October 11, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants