Skip to content

feat(sessions): detect an unhonoured context window, and hand off before filling it - #91

Open
juyoungk23 wants to merge 2 commits into
hristo2612:mainfrom
juyoungk23:pr/context-ceiling
Open

feat(sessions): detect an unhonoured context window, and hand off before filling it#91
juyoungk23 wants to merge 2 commits into
hristo2612:mainfrom
juyoungk23:pr/context-ceiling

Conversation

@juyoungk23

@juyoungk23 juyoungk23 commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Two changes for long-running sessions.

Background

Every model has a context limit, and the config declares what that limit is for each one. When the declared number is larger than the limit actually being enforced, Jinn thinks a session has room it doesn't have — so the engine compacts (summarises the conversation and drops history) far earlier and far more often than expected. Sessions get slower and lossier, and nothing points at why.

This is easy to hit by accident. Configure a model at 1M, but hand the engine a bare model alias instead of the explicit large-window variant, and the real ceiling can be a fraction of that. Nothing compares the two numbers, so the mismatch just quietly costs you.

How I found this bug

I had Opus configured at 1M, but my sessions were compacting constantly — 63 compaction boundaries in a single transcript. The cause was the model id: the Claude CLI treats opus and opus[1m] as two different models, and the bare alias enforces 200K no matter what the config claims. Correcting my own model ids fixed my case, and since that's just a config change it isn't part of this PR. What is here is the general problem it exposed — nothing in Jinn was comparing the declared window against the one actually being enforced, so the mismatch stayed invisible for weeks.

1. Detect a window that isn't being honoured

You can't ask the engine what limit it's enforcing, but you can watch for it. This tracks the highest context a model has ever reached and how long that high-water mark has held. Once it stops rising and parks well below the declared number across many turns, the enforced ceiling is lower than configured — and it now says so, instead of someone working it out much later.

It also adds the explicit large-window model entries to the generated config template, with a note that they're separate model ids rather than a flag — which is the mistake that causes this in the first place.

2. Hand a worker off before it runs out of room

A session that fills its window compacts mid-task, which is the worst moment to lose detail. This watches context pressure and nudges a handoff while there's still room to write a clean summary and start fresh.

Also

contextWindowForModel gains an exact option. By default, asking about a model that isn't in the config falls back to the engine's default model — fine when you're displaying a number, wrong when you're comparing declared against observed, because it would answer with a different model's window.

Relationship to 0.28.4

CLAUDE_CODE_AUTO_COMPACT_WINDOW raises the compaction trigger so long sessions don't compact more often than the model requires. This is the other half: catching the case where the window was never what the config claimed.

juyoungk23 and others added 2 commits July 25, 2026 08:56
…configured

A roster entry can declare a context window the engine does not honour, and
nothing notices. Claude Code treats "opus" and "opus[1m]" as different models —
only the latter gets the 1M window — so a roster that declares 1M for the bare
alias silently runs at the standard window. Combined with Jinn's injected
persona/org/tool context, that floor is a large fraction of the smaller window,
so sessions compact early and can thrash until the turn dies.

Observed on one install before the cause was known: 63 compaction boundaries in
a single transcript, context pinned at a 191,584-token ceiling against a
declared 1,000,000, while a correctly-configured model in the same transcript
reached 640,447.

Adds a detector that reports the mismatch instead of leaving it invisible. A
peak that stops growing is a ceiling: track the highest context each model
reaches and how long that peak holds, and warn once per model when it has held
across many turns, sits above an evidence floor, and never approaches the
declared window. Uses only what a turn already produced — no catalog lookup, no
network — so it works on subscription auth.

Two earlier designs were falsified by replaying a real transcript and both are
documented in the module: classifying individual compaction events by the level
they fire at does not separate healthy from broken (the ranges overlap), and
comparing a running max warns on healthy models before they have climbed.

Also:
- setup template offers the [1m] variants as selectable options. Defaults are
  unchanged — the 1M variants may carry different billing on some plans, so
  this surfaces the choice rather than making it.
- setup template ids containing "[" are quoted. Unquoted, they are a YAML
  parse error in flow style, which would break `jinn setup` outright; a new
  test parses DEFAULT_CONFIG and guards this.
- contextWindowForModel gains an opt-in `exact` mode. It falls back to the
  engine default for unlisted models, which is right for display and wrong for
  comparisons — reporting model Y's window for model X would produce false
  warnings. Existing callers are unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ5j8zHtB9hmNxbJycQQUN
…xt window

A long-lived worker re-sends its whole accumulated context every turn. Measured
on a real fleet: worker sessions averaged ~337K tokens/turn, of which only ~9%
was conversation — the rest tool results, tool inputs, and thinking — and
sessions ran for days across thousands of turns.

Neither obvious remedy is right. `/clear` destroys working state with nothing
externalised. `/compact` is already automatic, and is lossy in a way the *model*
chooses, leaving what survives trapped in a transcript.

Instead this fires the delegation contract the org already runs on: at 70% of
the context window a worker is asked to write its state to the work item and
end with DONE or BLOCKED, so a fresh session resumes from the ledger. A session
that needs compaction is evidence its state was never externalised.

Delivery is passive by design. The nudge is queued into transportMeta at turn
end and prepended to the next real prompt, so it costs no extra turn and is
dropped if the session never runs again. It escalates once at 85% and never
repeats a level. Operator chat (no employee) is excluded — it has no one to
hand off to, and belongs in the UI instead.

The transportMeta transitions live in shared/context-pressure.ts rather than
inline, so manager.ts and the tests exercise the same code; an earlier version
open-coded them and the tests silently validated a copy of the logic instead of
the logic. The keys manager preserves across connector merges are derived from
that module for the same reason.

Also adds an opt-in `exact` mode to contextWindowForModel: it falls back to the
engine default for unlisted models, which is right for display and wrong for a
threshold comparison. Existing callers are unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ5j8zHtB9hmNxbJycQQUN
@juyoungk23
juyoungk23 marked this pull request as draft July 25, 2026 16:08
@juyoungk23
juyoungk23 marked this pull request as ready for review July 25, 2026 16:13
@hristo2612

Copy link
Copy Markdown
Owner

Thanks for this, and for the write-up — the investigation that produced it was sound. Unfortunately it's been overtaken by events: the compaction thrash you were chasing was root-caused and fixed in v0.28.5, by a different mechanism, before this landed. I'd like to reject it as submitted and cherry-pick the one piece that's still additive.

The diagnosis is superseded

Your stated cause is that opus and opus[1m] are different models and the bare alias enforces 200K regardless of config. The actual cause was elsewhere: jinn routes every Claude session through a per-session loopback SSE proxy, so ANTHROPIC_BASE_URL points at 127.0.0.1. Claude Code decides first-party status by matching that host literally, the loopback address failed the check, and the session silently fell back to a 200K window — which also neutered v0.28.4's CLAUDE_CODE_AUTO_COMPACT_WINDOW pin, so that change could never have taken effect. 7e2303df asserts first-party for jinn's own proxy; a verified session then reached 323,429 tokens in a single request with zero compactions, on the bare alias.

I also queried the catalog discovery reads: claude-opus-5, claude-sonnet-5, claude-fable-5, claude-opus-4-8 all report max_input_tokens: 1000000. Declared and enforced now agree, so there's no mismatch left for the detector to find on this path.

Specific problems

The cli/setup.ts template change should not merge. It teaches every new install that [1m] variants are the answer. Those ids aren't in the Anthropic catalog so discovery can't back them, they carry no contextWindow in the template — which makes your own detector and nudge inert for exactly the entries it adds — and they collide with the model-resolution work in v0.28.6. Post-7e2303df this documents the wrong root cause.

The ceiling detector false-positives. evidence/warned in context-ceiling.ts are process-global per engine/model, so turns from all concurrent sessions pool into one high-water mark and one turnsSincePeak. The 79-vs-4-turn plateau your thresholds are calibrated on came from a single session's turn sequence. I reproduced a false positive: one legitimate 150K peak on a genuine 1M window, then 25 ordinary short turns from other sessions on the same model → warns that the operator's 1M config is a lie. That's trivially reachable on any real gateway. If this is kept, evidence must be keyed per session, not per process/model.

The handoff nudge is silently dropped on engine-switch turns. It's prepended at promptToRun = msg.text, but the sync path then does promptToRun = [intro, transcript, currentPrompt].join(…), discarding it — while markNudgeDelivered still runs at turn end, so the level is recorded as delivered and never fires again.

Semantic hazard: the nudge tells the worker to end with DONE or BLOCKED: <need>, but delegation-completion-contract.ts's TERMINAL_SIGNAL matches done/hand-off. A context-pressure handoff would be indistinguishable at the parent from genuine task completion, so a parent could relay an unfinished delegation as finished.

What I'd like instead

The context-pressure handoff is a genuinely new capability with nothing equivalent on main, and its instinct — externalise to the Todo rather than /clear//compact — matches how the system is meant to work. It has no dependency on the context-window bug at all; it's a feat bundled under a fix framing. Please resubmit it on its own with the sync-path drop fixed and a completion token the delegation contract can distinguish from DONE.

Also happy to take contextWindowForModel's exact option as a small standalone — it's correct. (For the record it currently has zero callers on main; this PR is its first consumer, so it's new surface rather than hardening of an existing path.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants