Keep single-shot prompts intact when max_tokens eats the context window

[Gunterlie]

Linked issue

Closes #4110

Why this change

A user reported Noodle generating posts with no character cards and no lorebook entries. Their captured request to the model explains why — the prompt body was never sent:

"messages": [
  { "role": "system", "content": "You write a fake social media timeline..." },
  { "role": "user",   "content": "# Content Uniqueness ... # JSON Output Format ..." }
],
"max_tokens": 32768

buildRefreshPrompt returns three messages: system rules, the context body (# Active Noodle Accounts, # Character Profiles, # World / Lore, the timeline), then the JSON-format instruction (noodle-public-prompt.service.ts:621). Only two arrived. The entire context body was deleted in transit, so the model received rules and an output schema with nothing to write about — and invented accounts (user_123, Kaelen_Storm) to fill the void. Those invented handles then failed validateNoodleGeneratedRefresh, so the refresh died two steps downstream with a "no usable response" error that pointed nowhere near the real problem.

The deletion happens in fitMessagesToContext:

• findOldestRemovableConversationBlock (base-provider.ts:332) skips system messages and protects only the final message (index < messages.length - 1). Everything else is fair game.
• When no message carries contextKind: "history", the fallback pass at :460 removes unmarked non-system messages anyway. Noodle sets contextKind nowhere, so its context body is the first thing deleted and the trailing instruction is the thing kept.
• max_tokens is only reduced when the input budget has already fallen to the 128-token floor (:422). Between those two regimes there is a window where the budget is too small for the prompt but too large to trigger the reduction — so the output budget keeps its full value while the prompt is thrown away.

Concretely, a connection storing max_tokens: 32768 against a ~36k context window leaves ~3.3k tokens of input budget. A ~4k-token prompt exceeds it, so the body is deleted and max_tokens stays at 32768 — matching the captured request exactly, including the 1210 prompt tokens the local backend reported.

This is not Noodle-specific. Any single-shot prompt builder — summarizers, one-off generators — has no history to annotate and is exposed the same way.

Why it surfaced now (it is not a recent regression)

This looked like fallout from the recent Noodle work. It isn't — both code ingredients are old:

• The trimmer preferring prompt deletion over shrinking max_tokens — a747a0cf7 (2026-05-07), "Prefer trimming old history over collapsing completion budget", which introduced the exact inputBudget <= reservedInputFloor gate this PR relaxes.
• Noodle inheriting the connection's stored max_tokens rather than its own computed budget — 958e5534f (2026-07-16) and the service extraction in 0029d9551 (2026-07-18).

Neither matches the "worked in the morning, broken by evening" report, because the third ingredient is not code — it is prompt size. The body is deleted as soon as context tokens > usableWindow − max_tokens. With max_tokens: 32768 against a ~36k window that threshold sits near 3.3k tokens, and the Noodle context message grows on its own with normal use: the 48h recent-post window (up to 100 posts), recalled older memories, the number of invited characters, and lorebook context all add to it. No deploy is needed to cross the line — only continued use. It also explains the intermittency the reporter saw ("I refreshed a few times"): as the 48h window rolls, the prompt shrinks and grows, so the failure can appear to resolve itself and return.

This is why the bug is invisible on a typical dev setup. A connection with no stored max_tokens override, or a large context window, leaves an input budget of tens of thousands of tokens and never approaches the threshold. The trap is config-shaped, not version-shaped.

Worth stating plainly for planning purposes: the recent Noodle features (recalled memories, character schedules, enhanced timeline writing, lorebook context) are not buggy, but each one makes the prompt richer and therefore moves affected users closer to a threshold that already existed. That argues for the "fail loudly when the prompt cannot fit" follow-up rather than against the features.

What changed

• base-provider.ts — a prompt with no contextKind: "history" message has nothing that is safe to drop, so the output budget yields instead: the existing max_tokens reduction now also applies when there is no removable history, before any message is deleted. The later fallback passes are untouched, so a prompt that genuinely cannot fit is still handled.
• Conversations are unaffected. Chat, Game, generate, and Professor Mari all annotate their turns with contextKind: "history", so history trimming still runs first and a large valid response budget does not collapse to the floor.
• scripts/regressions/context-fit.regression.ts (new, wired into pnpm regression:prompt) — asserts a Noodle-shaped single-shot prompt survives a 36864/32768 squeeze with its body intact and a usable reduced max_tokens; that a roomy window changes nothing; that an annotated conversation still trims history; and that an unconfigured context window touches nothing.

Validation

• pnpm check passes locally
• Container (Docker / Podman) built and ran without issue
• Ran the app, clicked through the changes manually
• Checked edge cases (light + dark mode, mobile viewport, empty states, error paths)
• Above manual verification completed (describe below)
• Read and followed CONTRIBUTING.md

Manual verification notes

• Manually verify the reported case: on a connection with a large stored max_tokens (e.g. 32768) and a context window only slightly larger, run a Noodle timeline refresh and confirm the outgoing request now contains three messages, with # Character Profiles and # World / Lore present.
• Manually verify the generated posts reflect character cards and lorebook entries again, instead of generic invented content.
• Manually verify a long chat conversation still trims oldest history as before and does not lose its system prompt or latest turn.
• Manually verify a chat with a large max_tokens still returns full-length replies (the response budget should not collapse).
• Manually verify the server log no longer shows [LLM context] Trimmed prompt ... for a normal Noodle refresh on that connection.

Automated, run locally:

• pnpm --filter @marinara-engine/server exec tsx ../../scripts/regressions/context-fit.regression.ts — passes with this change, fails without it (verified by reverting base-provider.ts; it fails on "no message may be dropped from a single-shot prompt").
• pnpm check — passes (one pre-existing unrelated client lint warning, 0 errors).

Docs and release impact

• No docs changes needed
• Updated docs (README / CONTRIBUTING / android/README / CHANGELOG) as needed
• Translated docs on the docs-i18n branch updated to match, or a [docs-i18n] follow-up issue opened (see CONTRIBUTING.md § Translated documentation)
• Version/release files updated (only if this PR includes a version bump)

UI evidence (if applicable)

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8b586613-3449-413b-9309-80651c611907

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/context-fit-drops-single-shot-prompt

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Gunterlie]

@SpicyMarinara could you take a look at this one before it goes in? I'm not confident it's the right shape of fix for the engine as a whole, even though it should fix the reported bug.

Two things I found after opening it:

1. The same bug is still live for regular chat. The last-resort loop at base-provider.ts:489 calls findOldestRemovableConversationBlock with no preferredKind, so it deletes any non-system message regardless of contextKind. With this PR applied, a chat whose character card is marked contextKind: "prompt" still loses that card once history runs out:

fitMessagesToContext(msgs, { maxContext: 2048, maxTokens: 1024 })
kept: [ 'system/prompt:...', 'user/history:latest user turn' ]
card survived: false

2. This PR protects Noodle by accident rather than by intent. It keys off "this prompt contains no history at all". Noodle sets contextKind nowhere, so it qualifies today — but if anyone later annotates part of the Noodle prompt as history (the recent timeline is history-shaped), hasRemovableHistory flips true and the body becomes deletable again.

The assembler already has the right taxonomy (prompt vs history, assembler.ts:475,715); the trimmer just ignores it in the fallback passes. A fix that respects it would cover both cases:

• only history (and unannotated legacy) is removable, never prompt
• annotate Noodle's context message as prompt so it's protected by declaration
• when nothing legal can be dropped and the output budget is already at the floor, raise a clear "prompt does not fit context window" error instead of silently sending a mutilated prompt

I kept this PR narrow deliberately since it touches core trimming, but that also means it's a patch on a symptom. One more known edge: this change can push max_tokens toward the 128 floor for a genuinely oversized single-shot prompt, where before it kept a large output budget by deleting the prompt — for JSON-mode features that turns empty-context garbage into truncated invalid JSON.

Heads up that I won't be available today, so please don't wait on me for this one. If you think the taxonomy fix is the right call, feel free to take it over, expand this PR in place, or close it in favour of your own — no need to check with me first. If you'd rather leave it, it's fine sitting here until I'm back.

[Gunterlie]

Also flagging separately: I'm gone for the day, so I won't be responding to anything here.

Context for anyone picking it up while I'm out:

• Keep single-shot prompts intact when max_tokens eats the context window #4124 (this PR) — fixes the actual reported bug: the Noodle prompt body being deleted before the request is sent. Reviewed caveats in the comment above.
• Skip Noodle accounts whose character card no longer exists #4111 — stale Noodle accounts. Real bug, reproduced, but not the cause of the report it's linked to.
• Stop one invalid field from resetting all Noodle settings #4109 — Noodle settings silently resetting to defaults on one invalid field. Real bug, also not the cause of that report.

#4111 and #4109 stand on their own merits; neither should be merged as "the fix" for #4110. #4124 is the one that addresses it.