fix(agent): adaptive default for agent_input_token_budget (closes #1170)#1190
fix(agent): adaptive default for agent_input_token_budget (closes #1170)#1190nickorlabs wants to merge 1 commit into
Conversation
|
This is closer because it adds a hard ceiling, but I would not merge this shape yet. Two blockers:
Suggested semantics:
Also please add/adjust tests for the explicit |
…sseus-dev#1170) The flat agent_input_token_budget=6000 default soft-capped every agent chat at 6000 input tokens regardless of model capability. Combined with the (now-fixed) Ollama num_ctx silent 2K cap, this meant users on long-context models (kimi-k2:1t, minimax-m3 at 1M, gpt-oss at 128K) silently lost most of their advertised context window — and the pasted-message head+tail truncation in src/context_compactor.py was the visible symptom. Semantics (revised per odysseus-dev#1190 review feedback from @pewdiepie-archdaemon): unset / None / empty / "auto" -> adaptive default, capped at the agent_input_token_hard_max setting (default ceiling: 200_000) 0 -> explicit "no soft trim" — preserves the existing escape-hatch for deployments that turned trimming off >0 -> explicit cap; still bounded by context_length when known so the budget never exceeds the real window negative / malformed string -> treated as unset (auto) Configurable hard max: The 200_000 ceiling is now a module-level DEFAULT_INPUT_TOKEN_HARD_MAX constant AND a tunable agent_input_token_hard_max setting. Admins can raise it (premium-API users with very large context) or lower it (cost-paranoid deployments) without code changes. Default chosen as "roomy but not runaway" for premium APIs. Three new symbols: - DEFAULT_INPUT_TOKEN_HARD_MAX module-level constant - _default_input_budget(ctx, hm) pure helper: ctx -> min(0.85*ctx, hm) - _resolve_input_token_budget(ctx) full semantic resolver Tests: - 74 total, all passing (49 pre-existing + 8 helper + 17 resolver) - explicit 0 case has 3 tests (int 0, string "0", and 0 with unknown ctx) to lock in the no-soft-trim escape hatch - the agent_input_token_hard_max setting has 4 tests (raise, lower, 0 falls back to default, malformed falls back to default) - a regression test pinning that an explicitly-stored agent_input_token_budget=6000 still yields exactly 6000 (existing user configs are preserved verbatim) Empirical evidence from a real Linux/Docker test deployment that motivated this: src.model_context - INFO - Context length for minimax-m3: 1000000 src.context_compactor - INFO - Trimming messages: 51967 > 4976 budget (ctx=6000) src.agent_loop - INFO - [agent] soft-trimmed context: 51967 -> 7123 tokens (budget=6000, reserve=1024) ctx=1M was discovered correctly, then min(1M, 6000) = 6000 kicked in. With this PR's adaptive default, the same setup yields a 200_000 budget, which the user can raise via agent_input_token_hard_max if they want the full window.
84bb2b2 to
0646490
Compare
|
Thanks for the careful read — both blockers were real. Pushed a revision (force-with-lease) that implements your suggested semantics. What changed
Configurable hard max
Three symbols now
The split lets the resolver be tested with Tests49 pre-existing + 8 helper + 17 resolver = 74 total, all pass. Explicit-0 case has three pinned tests ( Ready for another look when you have a minute. |
|
The revised resolver is much better and preserves the
Please add the setting to |
…sseus-dev#1170) Replaces the flat agent_input_token_budget=6000 default with adaptive semantics so users on long-context models (kimi-k2:1t, minimax-m3, gpt-oss at 128K, etc.) actually use the context window they have. Combined with the (now-merged) odysseus-dev#1056 Ollama num_ctx fix, this is the second half of the end-to-end long-context UX: odysseus-dev#1056 made Ollama honor the budget, this PR makes the budget meaningful. ## Semantics (per maintainer review on odysseus-dev#1190) unset / None / empty / "auto" -> adaptive default: ~85% of context_length, capped at the agent_input_token_hard_max setting 0 -> explicit "no soft trim" — preserves the existing escape hatch for deployments that turned trimming off >0 -> explicit cap, bounded by context_length when known negative / malformed string -> treated as unset (auto) ## What's in this PR 1. src/agent_loop.py - DEFAULT_INPUT_TOKEN_HARD_MAX = 200_000 (module-level, importable) - _default_input_budget(ctx, hard_max) — pure helper - _resolve_input_token_budget(ctx) — full semantic resolver - call site swapped to use the resolver 2. src/settings.py — both keys registered in DEFAULT_SETTINGS so the normal settings-management paths (/api/auth/settings, manage_settings tool) can persist them: - agent_input_token_budget default changed from 6000 to "auto" - agent_input_token_hard_max added with default 200_000 Each entry has a multi-line comment explaining the semantics inline. 3. src/tool_implementations.py — manage_settings friendly aliases: - existing 'token budget' -> agent_input_token_budget (kept) - new 'input budget' -> agent_input_token_budget - new 'hard max' -> agent_input_token_hard_max - new 'token budget cap' / 'input budget cap' -> agent_input_token_hard_max ## Backwards compatibility Users with an explicitly-persisted agent_input_token_budget=6000 in their data/settings.json keep exactly 6000 (the resolver treats it as an explicit cap). Pinned by a regression test. Fresh installs and users who never touched the setting get the adaptive default — that's the UX win. Users who explicitly stored 0 keep 0 (no soft trim), also pinned by tests. ## Tests 83 total, all passing (49 pre-existing + 8 helper + 17 resolver + 6 DEFAULT_SETTINGS-registration + 3 alias-registration). The DEFAULT_SETTINGS tests use real load_settings against a temp file so the end-to-end auto-default-when-unset path is exercised, not just the mocked-resolver case. python -m pytest tests/test_agent_loop.py -v ======================== 83 passed in 0.18s ======================== ## Empirical evidence (motivation) src.model_context - INFO - Context length for minimax-m3: 1000000 src.context_compactor - INFO - Trimming messages: 51967 > 4976 budget (ctx=6000) src.agent_loop - INFO - [agent] soft-trimmed context: 51967 -> 7123 tokens (budget=6000, reserve=1024) ctx=1M was discovered correctly, then min(1M, 6000) = 6000 dominated. With this PR + default 'auto' + hard_max=200_000, the same setup yields a 200K budget out of the box; admins who want more raise the hard_max.
0646490 to
ee57701
Compare
|
Good catch — What changed in this revision
Anything else you'd like me to adjust? |
|
The updated semantics and settings registration address my earlier blockers: But the branch is stale relative to current Please rebase/update so the two-dot diff contains only the intended files:
Once clean, this is ready for focused test review with |
…_token_hard_max setting Completes the reviewer requirement from PR odysseus-dev#1190 review that was carried over but not implemented in odysseus-dev#1230: > "The hard max is a function-local constant. For this setting, the ceiling > should be configurable or at least represented as a named setting/default > with tests." — review on odysseus-dev#1190 odysseus-dev#1230 shipped the adaptive auto-derivation but left `DEFAULT_HARD_MAX = 200_000` as a hardcoded module constant in src/context_budget.py. Admins on premium APIs with large context windows (kimi-k2 / minimax-m3 at 1M, etc.) can use their full window today only by setting `agent_input_token_budget` explicitly — which then takes them off the adaptive auto-path entirely. ## What this PR changes - src/settings.py: register `agent_input_token_hard_max` in DEFAULT_SETTINGS, default 200_000 (matches `DEFAULT_HARD_MAX`). Inline comment documents the no-op semantics in the explicit branch. - src/agent_loop.py: read the setting at the call site and pass it as the `hard_max` kwarg of `compute_input_token_budget`. Defensive parsing — missing / non-int / zero values fall back to `DEFAULT_HARD_MAX`, so a misconfig cannot silently zero the budget. - src/tool_implementations.py: three friendly aliases for `manage_settings`: - "hard max" -> agent_input_token_hard_max - "token budget cap" -> agent_input_token_hard_max - "input budget cap" -> agent_input_token_hard_max Plus the existing "token budget" -> agent_input_token_budget keeps a matching shorter alias "input budget". - tests/test_context_budget.py: 6 new tests on top of the existing 6: - hard_max raises the auto ceiling (1M ctx + raised cap -> 85% of ctx) - hard_max lowers the auto ceiling (128K ctx + 50K cap -> 50K) - hard_max has no effect on the explicit branch - DEFAULT_SETTINGS contains the new key - manage_settings aliases are registered - the live get_setting path returns the override value, and malformed values fall back per the agent_loop defensive parsing 12 passed in 0.04s. No changes to the pure helper signature or semantics; odysseus-dev#1230's behavior is the default when the new setting is unset. ## How it lets users drop the explicit override Before this PR, on a 1M-context model: agent_input_token_budget = 900_000 (explicit) -> 900K [user override] agent_input_token_budget = <unset> (auto) -> 200K [HARD_MAX] After this PR, same model: agent_input_token_budget = <unset> agent_input_token_hard_max = 900_000 -> min(1M * 0.85, 900K) = 850K [auto, no override needed] The explicit-override path keeps working unchanged for users who prefer it.
…_token_hard_max setting Completes the reviewer requirement from PR odysseus-dev#1190 review that was carried over but not implemented in odysseus-dev#1230: > "The hard max is a function-local constant. For this setting, the ceiling > should be configurable or at least represented as a named setting/default > with tests." — review on odysseus-dev#1190 odysseus-dev#1230 shipped the adaptive auto-derivation but left `DEFAULT_HARD_MAX = 200_000` as a hardcoded module constant in src/context_budget.py. Admins on premium APIs with large context windows (kimi-k2 / minimax-m3 at 1M, etc.) can use their full window today only by setting `agent_input_token_budget` explicitly — which then takes them off the adaptive auto-path entirely. ## What this PR changes - src/settings.py: register `agent_input_token_hard_max` in DEFAULT_SETTINGS, default 200_000 (matches `DEFAULT_HARD_MAX`). Inline comment documents the no-op semantics in the explicit branch. - src/agent_loop.py: read the setting at the call site and pass it as the `hard_max` kwarg of `compute_input_token_budget`. Defensive parsing — missing / non-int / zero values fall back to `DEFAULT_HARD_MAX`, so a misconfig cannot silently zero the budget. - src/tool_implementations.py: three friendly aliases for `manage_settings`: - "hard max" -> agent_input_token_hard_max - "token budget cap" -> agent_input_token_hard_max - "input budget cap" -> agent_input_token_hard_max Plus the existing "token budget" -> agent_input_token_budget keeps a matching shorter alias "input budget". - tests/test_context_budget.py: 6 new tests on top of the existing 6: - hard_max raises the auto ceiling (1M ctx + raised cap -> 85% of ctx) - hard_max lowers the auto ceiling (128K ctx + 50K cap -> 50K) - hard_max has no effect on the explicit branch - DEFAULT_SETTINGS contains the new key - manage_settings aliases are registered - the live get_setting path returns the override value, and malformed values fall back per the agent_loop defensive parsing 12 passed in 0.04s. No changes to the pure helper signature or semantics; odysseus-dev#1230's behavior is the default when the new setting is unset. ## How it lets users drop the explicit override Before this PR, on a 1M-context model: agent_input_token_budget = 900_000 (explicit) -> 900K [user override] agent_input_token_budget = <unset> (auto) -> 200K [HARD_MAX] After this PR, same model: agent_input_token_budget = <unset> agent_input_token_hard_max = 900_000 -> min(1M * 0.85, 900K) = 850K [auto, no override needed] The explicit-override path keeps working unchanged for users who prefer it.
…_token_hard_max setting (#1273) Completes the reviewer requirement from PR #1190 review that was carried over but not implemented in #1230: > "The hard max is a function-local constant. For this setting, the ceiling > should be configurable or at least represented as a named setting/default > with tests." — review on #1190 #1230 shipped the adaptive auto-derivation but left `DEFAULT_HARD_MAX = 200_000` as a hardcoded module constant in src/context_budget.py. Admins on premium APIs with large context windows (kimi-k2 / minimax-m3 at 1M, etc.) can use their full window today only by setting `agent_input_token_budget` explicitly — which then takes them off the adaptive auto-path entirely. ## What this PR changes - src/settings.py: register `agent_input_token_hard_max` in DEFAULT_SETTINGS, default 200_000 (matches `DEFAULT_HARD_MAX`). Inline comment documents the no-op semantics in the explicit branch. - src/agent_loop.py: read the setting at the call site and pass it as the `hard_max` kwarg of `compute_input_token_budget`. Defensive parsing — missing / non-int / zero values fall back to `DEFAULT_HARD_MAX`, so a misconfig cannot silently zero the budget. - src/tool_implementations.py: three friendly aliases for `manage_settings`: - "hard max" -> agent_input_token_hard_max - "token budget cap" -> agent_input_token_hard_max - "input budget cap" -> agent_input_token_hard_max Plus the existing "token budget" -> agent_input_token_budget keeps a matching shorter alias "input budget". - tests/test_context_budget.py: 6 new tests on top of the existing 6: - hard_max raises the auto ceiling (1M ctx + raised cap -> 85% of ctx) - hard_max lowers the auto ceiling (128K ctx + 50K cap -> 50K) - hard_max has no effect on the explicit branch - DEFAULT_SETTINGS contains the new key - manage_settings aliases are registered - the live get_setting path returns the override value, and malformed values fall back per the agent_loop defensive parsing 12 passed in 0.04s. No changes to the pure helper signature or semantics; #1230's behavior is the default when the new setting is unset. ## How it lets users drop the explicit override Before this PR, on a 1M-context model: agent_input_token_budget = 900_000 (explicit) -> 900K [user override] agent_input_token_budget = <unset> (auto) -> 200K [HARD_MAX] After this PR, same model: agent_input_token_budget = <unset> agent_input_token_hard_max = 900_000 -> min(1M * 0.85, 900K) = 850K [auto, no override needed] The explicit-override path keeps working unchanged for users who prefer it.
…_token_hard_max setting (odysseus-dev#1273) Completes the reviewer requirement from PR odysseus-dev#1190 review that was carried over but not implemented in odysseus-dev#1230: > "The hard max is a function-local constant. For this setting, the ceiling > should be configurable or at least represented as a named setting/default > with tests." — review on odysseus-dev#1190 odysseus-dev#1230 shipped the adaptive auto-derivation but left `DEFAULT_HARD_MAX = 200_000` as a hardcoded module constant in src/context_budget.py. Admins on premium APIs with large context windows (kimi-k2 / minimax-m3 at 1M, etc.) can use their full window today only by setting `agent_input_token_budget` explicitly — which then takes them off the adaptive auto-path entirely. ## What this PR changes - src/settings.py: register `agent_input_token_hard_max` in DEFAULT_SETTINGS, default 200_000 (matches `DEFAULT_HARD_MAX`). Inline comment documents the no-op semantics in the explicit branch. - src/agent_loop.py: read the setting at the call site and pass it as the `hard_max` kwarg of `compute_input_token_budget`. Defensive parsing — missing / non-int / zero values fall back to `DEFAULT_HARD_MAX`, so a misconfig cannot silently zero the budget. - src/tool_implementations.py: three friendly aliases for `manage_settings`: - "hard max" -> agent_input_token_hard_max - "token budget cap" -> agent_input_token_hard_max - "input budget cap" -> agent_input_token_hard_max Plus the existing "token budget" -> agent_input_token_budget keeps a matching shorter alias "input budget". - tests/test_context_budget.py: 6 new tests on top of the existing 6: - hard_max raises the auto ceiling (1M ctx + raised cap -> 85% of ctx) - hard_max lowers the auto ceiling (128K ctx + 50K cap -> 50K) - hard_max has no effect on the explicit branch - DEFAULT_SETTINGS contains the new key - manage_settings aliases are registered - the live get_setting path returns the override value, and malformed values fall back per the agent_loop defensive parsing 12 passed in 0.04s. No changes to the pure helper signature or semantics; odysseus-dev#1230's behavior is the default when the new setting is unset. ## How it lets users drop the explicit override Before this PR, on a 1M-context model: agent_input_token_budget = 900_000 (explicit) -> 900K [user override] agent_input_token_budget = <unset> (auto) -> 200K [HARD_MAX] After this PR, same model: agent_input_token_budget = <unset> agent_input_token_hard_max = 900_000 -> min(1M * 0.85, 900K) = 850K [auto, no override needed] The explicit-override path keeps working unchanged for users who prefer it.
Address RaresKeY's review: P2 (explicitness): is_setting_customized treated a saved value equal to the default as "not explicit", which ALSO blocked a user from deliberately pinning the default budget. Reframe the default value itself as the AUTO sentinel — agent_input_token_budget == DEFAULT_BUDGET means "scale to the model's context window", any other value is an explicit cap. A materialized default still reads as auto (fixing the original regression), and any non-default value the user chooses is now honoured. Drop the now-unused is_setting_customized helper. P2 (fallback context): auto-scaling trusted get_context_length() even when it returned only the bare DEFAULT_CONTEXT fallback (no endpoint-reported / known window), over-allocating on self-hosted/proxy setups. Add get_context_length_known() (also returns whether the window was actually discovered); the budget block passes 0 when unknown so auto-scaling stays conservative instead of inflating to an unproven window. hard_max stays auto-only — a deliberate explicit budget wins (odysseus-dev#1190); kept that contract and answered the reviewer's question rather than silently reversing it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…iew odysseus-dev#4122) - settings.py: an explicit budget is clamped to the window only — hard_max is auto-only (odysseus-dev#1190); drop the incorrect "and to hard_max". - is_setting_overridden docstring: drop the stale "adaptive budgets" example; point value-sensitive callers at context_budget.budget_is_explicit. - Tighten the budget-block comments to the contract (default = auto sentinel, non-default = explicit cap, hard_max = auto-only ceiling). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
odysseus-dev#1230/odysseus-dev#1273) The context-budget contract (auto-sentinel, explicit budgets honoured, hard_max auto-only) merged via odysseus-dev#1230 — odysseus-dev#1190 was the earlier, closed, superseded PR. Re-point the contract comments at odysseus-dev#1230 (the live source, already cited for the auto-sentinel two lines up in settings.py). The configurable hard_max setting (`agent_input_token_hard_max`) was a reviewer requirement first raised on odysseus-dev#1190, omitted from the merged odysseus-dev#1230, and actually added in odysseus-dev#1273 — credit odysseus-dev#1273 for it and correct the test comment's history (it previously implied this PR completed the requirement). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ndow scaling (#4122) * fix(agent): don't let a materialized default budget defeat context scaling #1230 scales agent_input_token_budget to the model's context window unless the user explicitly set a budget, detected via is_setting_overridden(). But the settings-save path materializes every DEFAULT_SETTINGS key into settings.json (load_settings merges defaults; handlers persist the merged dict), so the persisted default 6000 reads as "overridden" and the budget code takes the min(6000, ctx) branch — silently re-capping long-context models at 6000 for anyone who has ever saved a setting. This reintroduces the exact regression #1170/#1230 set out to fix. Add is_setting_customized() (saved value != default) and gate the scaling on it instead of mere presence. A persisted default is not a user choice. is_setting_overridden has exactly one consumer (this budget path), so the change is contained. Tests cover the materialized-default regression, a deliberately-chosen budget still being honoured, and the absent-key case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): rework context-budget fix per review (#4122) Address RaresKeY's review: P2 (explicitness): is_setting_customized treated a saved value equal to the default as "not explicit", which ALSO blocked a user from deliberately pinning the default budget. Reframe the default value itself as the AUTO sentinel — agent_input_token_budget == DEFAULT_BUDGET means "scale to the model's context window", any other value is an explicit cap. A materialized default still reads as auto (fixing the original regression), and any non-default value the user chooses is now honoured. Drop the now-unused is_setting_customized helper. P2 (fallback context): auto-scaling trusted get_context_length() even when it returned only the bare DEFAULT_CONTEXT fallback (no endpoint-reported / known window), over-allocating on self-hosted/proxy setups. Add get_context_length_known() (also returns whether the window was actually discovered); the budget block passes 0 when unknown so auto-scaling stays conservative instead of inflating to an unproven window. hard_max stays auto-only — a deliberate explicit budget wins (#1190); kept that contract and answered the reviewer's question rather than silently reversing it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(agent): lock the materialized-default budget regression (review on #4121) Per WGlynn's review on the issue: add an end-to-end regression that saves an UNRELATED setting (which makes the settings-save path materialize the budget default into settings.json) and asserts the budget still auto-scales rather than re-reading as an explicit 6000 cap — locking the exact reopening shut. To make the test bite the production decision (not just re-derive it), extract `budget_is_explicit()` into src/context_budget.py and use it from the agent loop. It keys off value-vs-default (the default is the auto sentinel), NOT settings presence — which is the whole point, since the save path materializes defaults. Note: after this PR's rework, is_setting_overridden has ZERO production callers, so the merged-dict materialization smell can't reach any setting through a presence check today (WGlynn's durability concern). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): bind the budget context window to its own provenance (review #4122) RaresKeY caught a correctness bug in the fallback-context guard: stream_agent_loop kept only the `known` flag from get_context_length_known() and budgeted off the passed-in `context_length`, which can come from a *different* lookup. Two failures: - local endpoints are re-queried, so the passed value can be a stale DEFAULT_CONTEXT fallback while the fresh probe proves the real (smaller) served context — we'd scale off the stale value; - callers that don't pass context_length (scheduled tasks, teacher escalation, skill test runs, bg_monitor) were capped at 6000 even when a long window is discoverable. Extract budget_context_for_model() which returns the freshly-probed window when known else 0, binding the flag to the value it proves; the agent loop uses it. Regression tests cover the stale-fallback, no-arg-caller, and probe-error paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agent): fix stale budget comments + tighten to the contract (review #4122) - settings.py: an explicit budget is clamped to the window only — hard_max is auto-only (#1190); drop the incorrect "and to hard_max". - is_setting_overridden docstring: drop the stale "adaptive budgets" example; point value-sensitive callers at context_budget.budget_is_explicit. - Tighten the budget-block comments to the contract (default = auto sentinel, non-default = explicit cap, hard_max = auto-only ceiling). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agent): correct budget issue citations (#1190 → merged #1230/#1273) The context-budget contract (auto-sentinel, explicit budgets honoured, hard_max auto-only) merged via #1230 — #1190 was the earlier, closed, superseded PR. Re-point the contract comments at #1230 (the live source, already cited for the auto-sentinel two lines up in settings.py). The configurable hard_max setting (`agent_input_token_hard_max`) was a reviewer requirement first raised on #1190, omitted from the merged #1230, and actually added in #1273 — credit #1273 for it and correct the test comment's history (it previously implied this PR completed the requirement). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ndow scaling (odysseus-dev#4122) * fix(agent): don't let a materialized default budget defeat context scaling odysseus-dev#1230 scales agent_input_token_budget to the model's context window unless the user explicitly set a budget, detected via is_setting_overridden(). But the settings-save path materializes every DEFAULT_SETTINGS key into settings.json (load_settings merges defaults; handlers persist the merged dict), so the persisted default 6000 reads as "overridden" and the budget code takes the min(6000, ctx) branch — silently re-capping long-context models at 6000 for anyone who has ever saved a setting. This reintroduces the exact regression odysseus-dev#1170/odysseus-dev#1230 set out to fix. Add is_setting_customized() (saved value != default) and gate the scaling on it instead of mere presence. A persisted default is not a user choice. is_setting_overridden has exactly one consumer (this budget path), so the change is contained. Tests cover the materialized-default regression, a deliberately-chosen budget still being honoured, and the absent-key case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): rework context-budget fix per review (odysseus-dev#4122) Address RaresKeY's review: P2 (explicitness): is_setting_customized treated a saved value equal to the default as "not explicit", which ALSO blocked a user from deliberately pinning the default budget. Reframe the default value itself as the AUTO sentinel — agent_input_token_budget == DEFAULT_BUDGET means "scale to the model's context window", any other value is an explicit cap. A materialized default still reads as auto (fixing the original regression), and any non-default value the user chooses is now honoured. Drop the now-unused is_setting_customized helper. P2 (fallback context): auto-scaling trusted get_context_length() even when it returned only the bare DEFAULT_CONTEXT fallback (no endpoint-reported / known window), over-allocating on self-hosted/proxy setups. Add get_context_length_known() (also returns whether the window was actually discovered); the budget block passes 0 when unknown so auto-scaling stays conservative instead of inflating to an unproven window. hard_max stays auto-only — a deliberate explicit budget wins (odysseus-dev#1190); kept that contract and answered the reviewer's question rather than silently reversing it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(agent): lock the materialized-default budget regression (review on odysseus-dev#4121) Per WGlynn's review on the issue: add an end-to-end regression that saves an UNRELATED setting (which makes the settings-save path materialize the budget default into settings.json) and asserts the budget still auto-scales rather than re-reading as an explicit 6000 cap — locking the exact reopening shut. To make the test bite the production decision (not just re-derive it), extract `budget_is_explicit()` into src/context_budget.py and use it from the agent loop. It keys off value-vs-default (the default is the auto sentinel), NOT settings presence — which is the whole point, since the save path materializes defaults. Note: after this PR's rework, is_setting_overridden has ZERO production callers, so the merged-dict materialization smell can't reach any setting through a presence check today (WGlynn's durability concern). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): bind the budget context window to its own provenance (review odysseus-dev#4122) RaresKeY caught a correctness bug in the fallback-context guard: stream_agent_loop kept only the `known` flag from get_context_length_known() and budgeted off the passed-in `context_length`, which can come from a *different* lookup. Two failures: - local endpoints are re-queried, so the passed value can be a stale DEFAULT_CONTEXT fallback while the fresh probe proves the real (smaller) served context — we'd scale off the stale value; - callers that don't pass context_length (scheduled tasks, teacher escalation, skill test runs, bg_monitor) were capped at 6000 even when a long window is discoverable. Extract budget_context_for_model() which returns the freshly-probed window when known else 0, binding the flag to the value it proves; the agent loop uses it. Regression tests cover the stale-fallback, no-arg-caller, and probe-error paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agent): fix stale budget comments + tighten to the contract (review odysseus-dev#4122) - settings.py: an explicit budget is clamped to the window only — hard_max is auto-only (odysseus-dev#1190); drop the incorrect "and to hard_max". - is_setting_overridden docstring: drop the stale "adaptive budgets" example; point value-sensitive callers at context_budget.budget_is_explicit. - Tighten the budget-block comments to the contract (default = auto sentinel, non-default = explicit cap, hard_max = auto-only ceiling). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agent): correct budget issue citations (odysseus-dev#1190 → merged odysseus-dev#1230/odysseus-dev#1273) The context-budget contract (auto-sentinel, explicit budgets honoured, hard_max auto-only) merged via odysseus-dev#1230 — odysseus-dev#1190 was the earlier, closed, superseded PR. Re-point the contract comments at odysseus-dev#1230 (the live source, already cited for the auto-sentinel two lines up in settings.py). The configurable hard_max setting (`agent_input_token_hard_max`) was a reviewer requirement first raised on odysseus-dev#1190, omitted from the merged odysseus-dev#1230, and actually added in odysseus-dev#1273 — credit odysseus-dev#1273 for it and correct the test comment's history (it previously implied this PR completed the requirement). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…_token_hard_max setting (odysseus-dev#1273) Completes the reviewer requirement from PR odysseus-dev#1190 review that was carried over but not implemented in odysseus-dev#1230: > "The hard max is a function-local constant. For this setting, the ceiling > should be configurable or at least represented as a named setting/default > with tests." — review on odysseus-dev#1190 odysseus-dev#1230 shipped the adaptive auto-derivation but left `DEFAULT_HARD_MAX = 200_000` as a hardcoded module constant in src/context_budget.py. Admins on premium APIs with large context windows (kimi-k2 / minimax-m3 at 1M, etc.) can use their full window today only by setting `agent_input_token_budget` explicitly — which then takes them off the adaptive auto-path entirely. ## What this PR changes - src/settings.py: register `agent_input_token_hard_max` in DEFAULT_SETTINGS, default 200_000 (matches `DEFAULT_HARD_MAX`). Inline comment documents the no-op semantics in the explicit branch. - src/agent_loop.py: read the setting at the call site and pass it as the `hard_max` kwarg of `compute_input_token_budget`. Defensive parsing — missing / non-int / zero values fall back to `DEFAULT_HARD_MAX`, so a misconfig cannot silently zero the budget. - src/tool_implementations.py: three friendly aliases for `manage_settings`: - "hard max" -> agent_input_token_hard_max - "token budget cap" -> agent_input_token_hard_max - "input budget cap" -> agent_input_token_hard_max Plus the existing "token budget" -> agent_input_token_budget keeps a matching shorter alias "input budget". - tests/test_context_budget.py: 6 new tests on top of the existing 6: - hard_max raises the auto ceiling (1M ctx + raised cap -> 85% of ctx) - hard_max lowers the auto ceiling (128K ctx + 50K cap -> 50K) - hard_max has no effect on the explicit branch - DEFAULT_SETTINGS contains the new key - manage_settings aliases are registered - the live get_setting path returns the override value, and malformed values fall back per the agent_loop defensive parsing 12 passed in 0.04s. No changes to the pure helper signature or semantics; odysseus-dev#1230's behavior is the default when the new setting is unset. ## How it lets users drop the explicit override Before this PR, on a 1M-context model: agent_input_token_budget = 900_000 (explicit) -> 900K [user override] agent_input_token_budget = <unset> (auto) -> 200K [HARD_MAX] After this PR, same model: agent_input_token_budget = <unset> agent_input_token_hard_max = 900_000 -> min(1M * 0.85, 900K) = 850K [auto, no override needed] The explicit-override path keeps working unchanged for users who prefer it.
…ndow scaling (odysseus-dev#4122) * fix(agent): don't let a materialized default budget defeat context scaling odysseus-dev#1230 scales agent_input_token_budget to the model's context window unless the user explicitly set a budget, detected via is_setting_overridden(). But the settings-save path materializes every DEFAULT_SETTINGS key into settings.json (load_settings merges defaults; handlers persist the merged dict), so the persisted default 6000 reads as "overridden" and the budget code takes the min(6000, ctx) branch — silently re-capping long-context models at 6000 for anyone who has ever saved a setting. This reintroduces the exact regression odysseus-dev#1170/odysseus-dev#1230 set out to fix. Add is_setting_customized() (saved value != default) and gate the scaling on it instead of mere presence. A persisted default is not a user choice. is_setting_overridden has exactly one consumer (this budget path), so the change is contained. Tests cover the materialized-default regression, a deliberately-chosen budget still being honoured, and the absent-key case. * fix(agent): rework context-budget fix per review (odysseus-dev#4122) Address RaresKeY's review: P2 (explicitness): is_setting_customized treated a saved value equal to the default as "not explicit", which ALSO blocked a user from deliberately pinning the default budget. Reframe the default value itself as the AUTO sentinel — agent_input_token_budget == DEFAULT_BUDGET means "scale to the model's context window", any other value is an explicit cap. A materialized default still reads as auto (fixing the original regression), and any non-default value the user chooses is now honoured. Drop the now-unused is_setting_customized helper. P2 (fallback context): auto-scaling trusted get_context_length() even when it returned only the bare DEFAULT_CONTEXT fallback (no endpoint-reported / known window), over-allocating on self-hosted/proxy setups. Add get_context_length_known() (also returns whether the window was actually discovered); the budget block passes 0 when unknown so auto-scaling stays conservative instead of inflating to an unproven window. hard_max stays auto-only — a deliberate explicit budget wins (odysseus-dev#1190); kept that contract and answered the reviewer's question rather than silently reversing it. * test(agent): lock the materialized-default budget regression (review on odysseus-dev#4121) Per WGlynn's review on the issue: add an end-to-end regression that saves an UNRELATED setting (which makes the settings-save path materialize the budget default into settings.json) and asserts the budget still auto-scales rather than re-reading as an explicit 6000 cap — locking the exact reopening shut. To make the test bite the production decision (not just re-derive it), extract `budget_is_explicit()` into src/context_budget.py and use it from the agent loop. It keys off value-vs-default (the default is the auto sentinel), NOT settings presence — which is the whole point, since the save path materializes defaults. Note: after this PR's rework, is_setting_overridden has ZERO production callers, so the merged-dict materialization smell can't reach any setting through a presence check today (WGlynn's durability concern). * fix(agent): bind the budget context window to its own provenance (review odysseus-dev#4122) RaresKeY caught a correctness bug in the fallback-context guard: stream_agent_loop kept only the `known` flag from get_context_length_known() and budgeted off the passed-in `context_length`, which can come from a *different* lookup. Two failures: - local endpoints are re-queried, so the passed value can be a stale DEFAULT_CONTEXT fallback while the fresh probe proves the real (smaller) served context — we'd scale off the stale value; - callers that don't pass context_length (scheduled tasks, teacher escalation, skill test runs, bg_monitor) were capped at 6000 even when a long window is discoverable. Extract budget_context_for_model() which returns the freshly-probed window when known else 0, binding the flag to the value it proves; the agent loop uses it. Regression tests cover the stale-fallback, no-arg-caller, and probe-error paths. * docs(agent): fix stale budget comments + tighten to the contract (review odysseus-dev#4122) - settings.py: an explicit budget is clamped to the window only — hard_max is auto-only (odysseus-dev#1190); drop the incorrect "and to hard_max". - is_setting_overridden docstring: drop the stale "adaptive budgets" example; point value-sensitive callers at context_budget.budget_is_explicit. - Tighten the budget-block comments to the contract (default = auto sentinel, non-default = explicit cap, hard_max = auto-only ceiling). Comment/docstring-only; no behaviour change. * docs(agent): correct budget issue citations (odysseus-dev#1190 → merged odysseus-dev#1230/odysseus-dev#1273) The context-budget contract (auto-sentinel, explicit budgets honoured, hard_max auto-only) merged via odysseus-dev#1230 — odysseus-dev#1190 was the earlier, closed, superseded PR. Re-point the contract comments at odysseus-dev#1230 (the live source, already cited for the auto-sentinel two lines up in settings.py). The configurable hard_max setting (`agent_input_token_hard_max`) was a reviewer requirement first raised on odysseus-dev#1190, omitted from the merged odysseus-dev#1230, and actually added in odysseus-dev#1273 — credit odysseus-dev#1273 for it and correct the test comment's history (it previously implied this PR completed the requirement). Comment/docstring-only; no behaviour change. ---------
…ndow scaling (odysseus-dev#4122) * fix(agent): don't let a materialized default budget defeat context scaling odysseus-dev#1230 scales agent_input_token_budget to the model's context window unless the user explicitly set a budget, detected via is_setting_overridden(). But the settings-save path materializes every DEFAULT_SETTINGS key into settings.json (load_settings merges defaults; handlers persist the merged dict), so the persisted default 6000 reads as "overridden" and the budget code takes the min(6000, ctx) branch — silently re-capping long-context models at 6000 for anyone who has ever saved a setting. This reintroduces the exact regression odysseus-dev#1170/odysseus-dev#1230 set out to fix. Add is_setting_customized() (saved value != default) and gate the scaling on it instead of mere presence. A persisted default is not a user choice. is_setting_overridden has exactly one consumer (this budget path), so the change is contained. Tests cover the materialized-default regression, a deliberately-chosen budget still being honoured, and the absent-key case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): rework context-budget fix per review (odysseus-dev#4122) Address RaresKeY's review: P2 (explicitness): is_setting_customized treated a saved value equal to the default as "not explicit", which ALSO blocked a user from deliberately pinning the default budget. Reframe the default value itself as the AUTO sentinel — agent_input_token_budget == DEFAULT_BUDGET means "scale to the model's context window", any other value is an explicit cap. A materialized default still reads as auto (fixing the original regression), and any non-default value the user chooses is now honoured. Drop the now-unused is_setting_customized helper. P2 (fallback context): auto-scaling trusted get_context_length() even when it returned only the bare DEFAULT_CONTEXT fallback (no endpoint-reported / known window), over-allocating on self-hosted/proxy setups. Add get_context_length_known() (also returns whether the window was actually discovered); the budget block passes 0 when unknown so auto-scaling stays conservative instead of inflating to an unproven window. hard_max stays auto-only — a deliberate explicit budget wins (odysseus-dev#1190); kept that contract and answered the reviewer's question rather than silently reversing it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(agent): lock the materialized-default budget regression (review on odysseus-dev#4121) Per WGlynn's review on the issue: add an end-to-end regression that saves an UNRELATED setting (which makes the settings-save path materialize the budget default into settings.json) and asserts the budget still auto-scales rather than re-reading as an explicit 6000 cap — locking the exact reopening shut. To make the test bite the production decision (not just re-derive it), extract `budget_is_explicit()` into src/context_budget.py and use it from the agent loop. It keys off value-vs-default (the default is the auto sentinel), NOT settings presence — which is the whole point, since the save path materializes defaults. Note: after this PR's rework, is_setting_overridden has ZERO production callers, so the merged-dict materialization smell can't reach any setting through a presence check today (WGlynn's durability concern). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): bind the budget context window to its own provenance (review odysseus-dev#4122) RaresKeY caught a correctness bug in the fallback-context guard: stream_agent_loop kept only the `known` flag from get_context_length_known() and budgeted off the passed-in `context_length`, which can come from a *different* lookup. Two failures: - local endpoints are re-queried, so the passed value can be a stale DEFAULT_CONTEXT fallback while the fresh probe proves the real (smaller) served context — we'd scale off the stale value; - callers that don't pass context_length (scheduled tasks, teacher escalation, skill test runs, bg_monitor) were capped at 6000 even when a long window is discoverable. Extract budget_context_for_model() which returns the freshly-probed window when known else 0, binding the flag to the value it proves; the agent loop uses it. Regression tests cover the stale-fallback, no-arg-caller, and probe-error paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agent): fix stale budget comments + tighten to the contract (review odysseus-dev#4122) - settings.py: an explicit budget is clamped to the window only — hard_max is auto-only (odysseus-dev#1190); drop the incorrect "and to hard_max". - is_setting_overridden docstring: drop the stale "adaptive budgets" example; point value-sensitive callers at context_budget.budget_is_explicit. - Tighten the budget-block comments to the contract (default = auto sentinel, non-default = explicit cap, hard_max = auto-only ceiling). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agent): correct budget issue citations (odysseus-dev#1190 → merged odysseus-dev#1230/odysseus-dev#1273) The context-budget contract (auto-sentinel, explicit budgets honoured, hard_max auto-only) merged via odysseus-dev#1230 — odysseus-dev#1190 was the earlier, closed, superseded PR. Re-point the contract comments at odysseus-dev#1230 (the live source, already cited for the auto-sentinel two lines up in settings.py). The configurable hard_max setting (`agent_input_token_hard_max`) was a reviewer requirement first raised on odysseus-dev#1190, omitted from the merged odysseus-dev#1230, and actually added in odysseus-dev#1273 — credit odysseus-dev#1273 for it and correct the test comment's history (it previously implied this PR completed the requirement). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
Closes #1170.
Replaces the flat
agent_input_token_budget = 6000default with adaptive semantics so users on long-context models (kimi-k2:1t,minimax-m3,gpt-oss128K, etc.) actually use the context window they have. Sibling to the now-merged #1056 — that PR made Ollama honornum_ctx; this one makes the app-side budget meaningful in the first place.Semantics
agent_input_token_budgetvalueNone/""/"auto"(the new default)min(context_length * 0.85, agent_input_token_hard_max)0>0context_lengthwhen knownagent_input_token_hard_max(new, default200_000) caps the auto branch only — explicit positive budgets are not capped by it (user knows what they want).What this PR changes
src/agent_loop.pyDEFAULT_INPUT_TOKEN_HARD_MAX = 200_000(module-level, importable)_default_input_budget(ctx, hard_max)— pure helper_resolve_input_token_budget(ctx)— full semantic resolversrc/settings.py— both keys registered inDEFAULT_SETTINGSso the normal settings paths (/api/auth/settings, themanage_settingstool) can read/persist them:agent_input_token_budgetdefault6000→"auto"agent_input_token_hard_maxadded with default200_000src/tool_implementations.py—manage_settingsfriendly aliases extended:"token budget"→agent_input_token_budget(kept)"input budget"→agent_input_token_budget"hard max","token budget cap","input budget cap"→agent_input_token_hard_maxBackwards compatibility
data/settings.jsonhas noagent_input_token_budgetkey"auto"fromDEFAULT_SETTINGS→ adaptive default kicks indata/settings.jsonhas explicit6000(or any positive int)data/settings.jsonhas explicit0data/settings.jsonhas explicit"auto"Pinned by regression tests for the
6000-preserved and0-preserved cases.Tests
83 total, all passing (49 pre-existing + 8 pure-helper + 17 resolver + 6
DEFAULT_SETTINGS-registration + 3 alias-registration).The
TestDefaultSettingsRegistrationgroup exercises the end-to-end auto-default-when-unset path against the realload_settings(usingtmp_pathso it's hermetic), not just the mocked resolver case. The alias tests grepsrc/tool_implementations.pydirectly to avoid pulling the full app stack but still pin that the friendly names exist.Diff
Adjacent context
num_ctxnow correctly sent to Ollama; combined, the full long-context pipeline workscontext_length; orthogonal to this PR