Skip to content

Compatible runtime budgets reserve output beyond the actual request cap #386

Description

@Calmingstorm

Confirmed conservative budget mismatch

Reviewed master at 886c36d8ebe861aa987059a1744d45b78797baae (v4.7.0). Suggested priority: P3, capacity utilization rather than overflow.

Compatible-model eligibility uses the effective request output ceiling, but runtime chat/agent budget snapshots omit it. The actual request caps output to 32,768 tokens for configured profiles, while the runtime budget may reserve the profile's much larger maximum output capacity.

Sources: cap-aware eligibility/helper, agent snapshot call, chat snapshot, request cap.

Reproduction using a shipped profile

total window: 1,048,576
profile output maximum: 393,216
actual request output cap: 32,768
runtime working budget: 491,520
cap-aligned working budget: 761,856

The default utilization factor still applies in both calculations. Verified by three reviewers with source and arithmetic; no live provider request or measured production compaction frequency is claimed.

Expected behavior / acceptance criteria

  • Use the same effective output reserve in eligibility and runtime snapshots, pinned to the generation's request plan.
  • If extra conservative reserve is intentional, express and document it separately from the physical output cap rather than silently disagreeing with preflight.
  • Test chat and agent paths with large profile output maxima and changed request caps.
  • Preserve configured utilization/headroom policy and overflow safety.

Behavior change: potentially later compaction/more retained history. This does not prove context overflow or corruption; it wastes usable capacity. No source changes were made.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions