feat(sessions): detect an unhonoured context window, and hand off before filling it - #91
feat(sessions): detect an unhonoured context window, and hand off before filling it#91juyoungk23 wants to merge 2 commits into
Conversation
…configured A roster entry can declare a context window the engine does not honour, and nothing notices. Claude Code treats "opus" and "opus[1m]" as different models — only the latter gets the 1M window — so a roster that declares 1M for the bare alias silently runs at the standard window. Combined with Jinn's injected persona/org/tool context, that floor is a large fraction of the smaller window, so sessions compact early and can thrash until the turn dies. Observed on one install before the cause was known: 63 compaction boundaries in a single transcript, context pinned at a 191,584-token ceiling against a declared 1,000,000, while a correctly-configured model in the same transcript reached 640,447. Adds a detector that reports the mismatch instead of leaving it invisible. A peak that stops growing is a ceiling: track the highest context each model reaches and how long that peak holds, and warn once per model when it has held across many turns, sits above an evidence floor, and never approaches the declared window. Uses only what a turn already produced — no catalog lookup, no network — so it works on subscription auth. Two earlier designs were falsified by replaying a real transcript and both are documented in the module: classifying individual compaction events by the level they fire at does not separate healthy from broken (the ranges overlap), and comparing a running max warns on healthy models before they have climbed. Also: - setup template offers the [1m] variants as selectable options. Defaults are unchanged — the 1M variants may carry different billing on some plans, so this surfaces the choice rather than making it. - setup template ids containing "[" are quoted. Unquoted, they are a YAML parse error in flow style, which would break `jinn setup` outright; a new test parses DEFAULT_CONFIG and guards this. - contextWindowForModel gains an opt-in `exact` mode. It falls back to the engine default for unlisted models, which is right for display and wrong for comparisons — reporting model Y's window for model X would produce false warnings. Existing callers are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VQ5j8zHtB9hmNxbJycQQUN
…xt window A long-lived worker re-sends its whole accumulated context every turn. Measured on a real fleet: worker sessions averaged ~337K tokens/turn, of which only ~9% was conversation — the rest tool results, tool inputs, and thinking — and sessions ran for days across thousands of turns. Neither obvious remedy is right. `/clear` destroys working state with nothing externalised. `/compact` is already automatic, and is lossy in a way the *model* chooses, leaving what survives trapped in a transcript. Instead this fires the delegation contract the org already runs on: at 70% of the context window a worker is asked to write its state to the work item and end with DONE or BLOCKED, so a fresh session resumes from the ledger. A session that needs compaction is evidence its state was never externalised. Delivery is passive by design. The nudge is queued into transportMeta at turn end and prepended to the next real prompt, so it costs no extra turn and is dropped if the session never runs again. It escalates once at 85% and never repeats a level. Operator chat (no employee) is excluded — it has no one to hand off to, and belongs in the UI instead. The transportMeta transitions live in shared/context-pressure.ts rather than inline, so manager.ts and the tests exercise the same code; an earlier version open-coded them and the tests silently validated a copy of the logic instead of the logic. The keys manager preserves across connector merges are derived from that module for the same reason. Also adds an opt-in `exact` mode to contextWindowForModel: it falls back to the engine default for unlisted models, which is right for display and wrong for a threshold comparison. Existing callers are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VQ5j8zHtB9hmNxbJycQQUN
|
Thanks for this, and for the write-up — the investigation that produced it was sound. Unfortunately it's been overtaken by events: the compaction thrash you were chasing was root-caused and fixed in v0.28.5, by a different mechanism, before this landed. I'd like to reject it as submitted and cherry-pick the one piece that's still additive. The diagnosis is supersededYour stated cause is that I also queried the catalog discovery reads: Specific problemsThe The ceiling detector false-positives. The handoff nudge is silently dropped on engine-switch turns. It's prepended at Semantic hazard: the nudge tells the worker to end with What I'd like insteadThe context-pressure handoff is a genuinely new capability with nothing equivalent on Also happy to take |
Two changes for long-running sessions.
Background
Every model has a context limit, and the config declares what that limit is for each one. When the declared number is larger than the limit actually being enforced, Jinn thinks a session has room it doesn't have — so the engine compacts (summarises the conversation and drops history) far earlier and far more often than expected. Sessions get slower and lossier, and nothing points at why.
This is easy to hit by accident. Configure a model at 1M, but hand the engine a bare model alias instead of the explicit large-window variant, and the real ceiling can be a fraction of that. Nothing compares the two numbers, so the mismatch just quietly costs you.
How I found this bug
I had Opus configured at 1M, but my sessions were compacting constantly — 63 compaction boundaries in a single transcript. The cause was the model id: the Claude CLI treats
opusandopus[1m]as two different models, and the bare alias enforces 200K no matter what the config claims. Correcting my own model ids fixed my case, and since that's just a config change it isn't part of this PR. What is here is the general problem it exposed — nothing in Jinn was comparing the declared window against the one actually being enforced, so the mismatch stayed invisible for weeks.1. Detect a window that isn't being honoured
You can't ask the engine what limit it's enforcing, but you can watch for it. This tracks the highest context a model has ever reached and how long that high-water mark has held. Once it stops rising and parks well below the declared number across many turns, the enforced ceiling is lower than configured — and it now says so, instead of someone working it out much later.
It also adds the explicit large-window model entries to the generated config template, with a note that they're separate model ids rather than a flag — which is the mistake that causes this in the first place.
2. Hand a worker off before it runs out of room
A session that fills its window compacts mid-task, which is the worst moment to lose detail. This watches context pressure and nudges a handoff while there's still room to write a clean summary and start fresh.
Also
contextWindowForModelgains anexactoption. By default, asking about a model that isn't in the config falls back to the engine's default model — fine when you're displaying a number, wrong when you're comparing declared against observed, because it would answer with a different model's window.Relationship to 0.28.4
CLAUDE_CODE_AUTO_COMPACT_WINDOWraises the compaction trigger so long sessions don't compact more often than the model requires. This is the other half: catching the case where the window was never what the config claimed.