Cost-aware compaction: scheduled runs compact once the resent prompt exceeds a token budget - #1561
Merged
Merged
Conversation
…exceeds a token budget Compaction fired only on context-WINDOW pressure (90% of the model's window), and scheduled runs only warned unless FLEET_SCHEDULED_AUTO_COMPACT=1. A 1M-window model never reaches 90% in a 100-turn run, yet every one of those turns resends and pays for the whole transcript: the Reklaim health scan (6bd0c212) resent ~115K tokens per call for 99 calls at an 11% cache-hit rate — 11.35M prompt tokens, $8.29, mostly re-reading its own history. No compaction ever ran (#1534). Scheduled runs (the driver-supplied requireCompactionOptIn marks them) now also compact — the same oldest-half summary through the same governed summarizer — once the resent prompt (LastStepPromptTokens, the anti-spiral signal the window path already uses) exceeds FLEET_CONTEXT_RESEND_BUDGET_TOKENS (default 80K; 0 disables). This is a cost control, not a safety valve, so it is deliberately NOT behind the SCHEDULED_AUTO_COMPACT opt-in, which keeps guarding the window-pressure path; interactive chats are untouched. The fleet.context_compacted event and the [context_compacted] session breadcrumb carry trigger=resend_budget with the measured size and the budget, and the window path now labels its own compactions trigger=window. The per-result lever the issue also asked for already exists as the max_tool_output_bytes admin setting (default 64 KiB); the docs now point at it for jobs that keep rendering 40 KB documents into return_vars. TestRun_ContextPressure_ScheduledWarnsOnlyWithoutFlag pins the window contract and now disables the budget trigger explicitly; the budget trigger has its own tests. Closes #1534 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1534.
Problem
Compaction fired only on context-window pressure (90%), and scheduled runs only warned unless
FLEET_SCHEDULED_AUTO_COMPACT=1. A 1M-window model never reaches 90% in a 100-turn run, yet every turn resends the whole transcript and pays for it: Reklaim6bd0c212resent ~115K tokens per call for 99 calls at an 11% cache-hit rate — 11.35M prompt tokens, $8.29 — and no compaction ever ran.Change
checkContextPressure, scheduled engines only —requireCompactionOptInis the driver's scheduled marker): onceLastStepPromptTokens(the anti-spiral signal the window path already uses) exceedsFLEET_CONTEXT_RESEND_BUDGET_TOKENS(default 80 000;0disables), the same oldest-halfproactiveCompactruns through the same governed summarizer. It is a cost control, not a safety valve, so it is deliberately not behindFLEET_SCHEDULED_AUTO_COMPACT, which keeps guarding the window-pressure path. Interactive chats are untouched.fleet.context_compactednow carriestrigger(resend_budget|window) and, for the budget path,used_tokens+resend_budget_tokens; the session log gets a[context_compacted] trigger=resend_budget used=… budget=…breadcrumb.knobs.go(scopeExternal) and the env-file allowlist;AGENT-RUNTIME.mdcompaction table + a paragraph. The per-result lever the issue also asked for already exists (max_tool_output_bytes, default 64 KiB); the docs point at it.Contract change, on purpose
TestRun_ContextPressure_ScheduledWarnsOnlyWithoutFlag("a scheduled run must not silently rewrite its transcript without the opt-in") now disables the budget trigger explicitly: that test pins the window-pressure contract, while the cost trigger is default-on for scheduled runs — with #1518's completed-step checkpoints making re-drives safe, and the breadcrumb + event making it not silent. Expected effect on the Reklaim job: ~50K resent tokens per turn instead of ~115K.Tests:
TestRun_ResendBudget_ScheduledCompactsWithoutOptIn(run-loop, event payload + breadcrumb),TestCheckContextPressure_ResendBudgetIsScheduledOnly(interactive ignores; scheduled compacts; 0 disables),TestCheckContextPressure_WindowCompactionNamesTrigger,TestContextResendBudgetTokensKnob; existing context/compaction suites green; vet + golangci-lint clean.🤖 Generated with Claude Code