fix(agent): don't let a materialized default budget defeat context-window scaling

[nsgds]

Summary

#1230 made agent_input_token_budget adapt to the model's context window unless the user set an explicit budget. But the settings-save path materializes every DEFAULT_SETTINGS key into data/settings.json (load_settings() returns {**DEFAULT_SETTINGS, **saved} and the save handlers persist that merged dict), so once a user changes any setting the default budget (6000) is written to disk. Detecting "explicit" via is_setting_overridden (presence) then reads that materialized default as a deliberate choice → min(6000, context_length) = 6000 → long-context models are silently re-capped at 6000 input tokens (the exact regression #1170/#1230 set out to fix; symptom: the agent loses earlier conversation past ~6000 tokens).

Fix — the default value is the "auto" sentinel. agent_input_token_budget == DEFAULT_BUDGET (6000) means "scale to the model's window"; any other value is an explicit cap (clamped to the window). A materialized default still reads as auto, and a non-default value the user chooses is honoured. (A deliberate 6000 can't be told from a materialized one by value, so the default reads as auto — pin a cap with a nearby value, e.g. 5999.) The decision is centralised in budget_is_explicit() (src/context_budget.py); is_setting_overridden no longer has any production caller.

Two related correctness guards from review:

• Don't scale off an unproven context window. get_context_length_known() reports whether the window was actually discovered (endpoint-reported / known table) vs a bare DEFAULT_CONTEXT fallback; budget_context_for_model() binds that flag to the value it proves, so a fallback 128K (or a caller that doesn't pass a context length) doesn't inflate the budget.
• agent_input_token_hard_max stays auto-only — it caps the auto-derived budget; a deliberate explicit budget is honoured (#1230).

Target branch

• This PR targets dev, not main.

Linked Issue

Fixes #4121.

Type of Change

• Bug fix (non-breaking — fixes a confirmed regression)

Checklist

• I searched open issues and open PRs — this is not a duplicate.
• This PR targets dev.
• My changes are limited to the scope described above.
• I actually ran the app and verified the change end-to-end.

How to Test

1. On dev, with a long-context model: change one setting in the Settings panel (materializes agent_input_token_budget: 6000 into data/settings.json). Before this PR a conversation past ~6000 input tokens logs [agent] soft-trimmed context: N -> ~878 tokens (capped); with this PR it logs the scaled budget (~111k for a 131072 window) and history is retained.
2. Set agent_input_token_budget to a non-default value (e.g. 20000) — it's honoured as an explicit cap (clamped to the window).
3. python -m pytest tests/test_budget_auto_sentinel.py tests/test_context_budget.py tests/test_model_context.py -q — covers the auto sentinel, the materialized-default-after-saving-an-unrelated-setting regression, the fallback-context guard, and the provenance binding (no-arg caller / stale local re-query).

Visual / UI changes

None — backend only (src/agent_loop.py, src/context_budget.py, src/model_context.py, src/settings.py, tests).

[RaresKeY]

I think this is fixing a real regression, but there are two places where the new routing still needs tighter semantics before this is safe to rely on broadly.

Findings

issue: Explicitly setting the default budget no longer pins the agent budget

• Problem: is_setting_customized("agent_input_token_budget") treats a saved value equal to DEFAULT_SETTINGS["agent_input_token_budget"] as not explicit. That fixes the materialized-default case, but it also makes a deliberate user setting of 6000 indistinguishable from the default.

• Impact: Users lose a valid way to cap agent input at the documented/default budget. This matters for cost control and smaller/self-hosted deployments where 6000 is an intentional ceiling, not just a materialized default.

• Ask: Preserve explicitness separately from value equality, or make 6000 semantically “auto/default” and update the settings contract/tests accordingly.

• Location: src/settings.py:255

issue: Auto budget can still scale from fallback context instead of real endpoint capability

• Problem: This change sends materialized defaults down the auto-budget path, but that path trusts context_length as if it came from the endpoint/model. Today get_context_length() can still return the global DEFAULT_CONTEXT fallback (128000) when no provider-native or endpoint-reported context was found. In that case the agent can auto-scale to about 108800 input tokens even though the provider never proved that window.

• Impact: This is especially risky for self-hosted and OpenAI-compatible proxy deployments. Some endpoints expose real served context, some expose only model inventory, and local runners can have memory-dependent context limits. Treating the fallback as capability evidence can cause oversized prompts, context-limit failures, or local memory pressure.

• Ask: Keep the fix narrow, but do not auto-scale from an unknown/fallback context. If context discovery only reached the global fallback, retain the conservative/default budget or use an explicit unknown-context budget. Full provider capability metadata can be handled later.

• Location: src/agent_loop.py:2026

Open Questions

• question (scope): Since feat(models): define capability schema and readers #2739 is only the first schema/scaffolding step for model capabilities, should this PR add just a minimal trusted-vs-fallback guard now, and leave richer provider capability metadata to follow-ups? That later bucket likely covers determining endpoint/provider facts such as auto input budget, max tokens, and served context.

• question (budget semantics): Is agent_input_token_hard_max intended to be a global safety ceiling or only an auto-budget ceiling? The current code makes it auto-only, so explicit budgets can exceed it while only being clamped to model context.

Validation

Ran:

• PYTHONDONTWRITEBYTECODE=1 venv/bin/python -m pytest tests/test_setting_customized_budget.py tests/test_context_budget.py tests/test_manage_settings_token_budget.py -q
• Result: 18 passed, 1 warning

Not run:

• Live Ollama/LM Studio/vLLM self-hosted probe.
• GPU/VRAM OOM validation.

[nsgds]

Thanks @RaresKeY — reworked in 3c6b3f0 (supersedes the original is_setting_customized approach).

P2 (explicitness). You're right that is_setting_customized couldn't tell a deliberate 6000 from the materialized default. I took your second option — make the default value the auto sentinel: agent_input_token_budget == DEFAULT_BUDGET means "scale to the model's window", any other value is an explicit cap (clamped to the window). A materialized default still reads as auto (the original regression stays fixed), and any non-default value the user chooses is now honoured.

The one thing this gives up is pinning exactly the default — to cap near 6000 you'd set e.g. 5999. I did weigh the fuller "explicitness regardless of value" route (your option 1): it needs either a one-time migration to relabel already-materialized 6000s, or a separate write-time "explicit" flag that every setter (incl. the whole-form UI save) has to maintain — both meaningfully more surface than this fix, for a limitation whose practical cost is ~0 (on a 128K model the choice is 6000 vs 5999 → min(5999, window)). Glad to build the flag-based version if you'd rather have true explicitness — just laying out the tradeoff.

P2 (fallback context). Good catch — auto-scaling was trusting get_context_length() even when it returned only the bare DEFAULT_CONTEXT fallback. Added get_context_length_known(), which also reports whether the window was actually discovered (endpoint-reported or in the known-models table) vs a fallback; the budget block now passes 0 when it's unknown, so auto-scaling stays conservative (the default budget) instead of inflating to an unproven 128K.

This is intentionally the minimal trusted-vs-fallback guard — which is also my answer to your scope question: the authoritative "real served context / auto input budget / max tokens" belongs in the #2739 capability metadata, so I'd defer the richer provider-capability version to that bucket rather than grow it here.

On hard_max. Left it auto-only. #1190 deliberately made it an auto-budget ceiling that a deliberate explicit budget overrides (there's a test pinning exactly that — "the user's choice wins"), so I didn't want to silently reverse that contract on the back of this PR. If you think it should be a global safety ceiling that caps explicit budgets too, I'm happy to make it so — it's a one-liner — but it felt like its own decision rather than part of this fix.

[RaresKeY]

I found one current-head correctness issue in the new fallback-context guard.

P2 issue: The known-context guard can still budget from the stale fallback value

stream_agent_loop() now calls get_context_length_known(endpoint_url, model), but it discards the returned context length and keeps only _ctx_known:

_, _ctx_known = get_context_length_known(endpoint_url, model)
ctx_for_budget = context_length if _ctx_known else 0

That splits the provenance bit from the value it proves. The earlier context_length argument can come from a different lookup (maybe_compact() calls plain get_context_length()), and local endpoints are intentionally re-queried. If the first local probe falls back to DEFAULT_CONTEXT and the second probe succeeds with the real served context, this still computes the auto budget from the stale fallback. In a focused probe, first lookup (128000, False) followed by second lookup (4096, True) made current PR logic compute 108800; using the fresh proven 4096 gives 3481.

The same wiring also leaves direct/background stream_agent_loop() callers that do not pass context_length capped at 6000 even when get_context_length_known() discovers a long context. Real call sites include skill test runs, scheduled tasks, and teacher escalation.

Please bind the known flag to the same context value used for budget computation, either by using the returned value from get_context_length_known() or by propagating (context_length, known) from the initial compaction lookup. A focused stream_agent_loop regression should cover both the unknown-fallback case and the known-context/no-arg caller case.

Small cleanup: comments/docs still need to match the new contract. DEFAULT_BUDGET/6000 is now the auto sentinel, non-default values are explicit caps, and agent_input_token_hard_max only caps auto-derived budgets; some comments and the PR How-to-Test still describe older names/semantics.

[nsgds]

Good catch, thanks — fixed in 12cd741.

You're right that the guard split the provenance bit from the value it proves. I extracted budget_context_for_model(endpoint_url, model, fallback=...), which returns the freshly-probed window only when it was actually proven, else 0 — so the known flag is always bound to the value it justifies. The agent loop uses that instead of the passed-in context_length, which fixes both cases you found:

• the stale local re-query (first probe (128000, False), second (4096, True)) now budgets off the proven 4096;
• the no-arg callers (scheduled tasks, teacher escalation, skill runs, bg_monitor) now scale off the discovered window instead of being pinned at 6000.

Regression tests in tests/test_budget_auto_sentinel.py cover the unknown-fallback → 0, the no-arg-caller → discovered window, the fresh-value-beats-stale-fallback, and the probe-error → caller-fallback paths.

Also did the cleanup: refreshed the budget-block comments to the new contract (default value = auto sentinel, non-default = explicit cap, hard_max auto-only) and rewrote the PR description's Summary / How-to-Test to match (it still referenced the old is_setting_customized approach and test file).

[RaresKeY]

Runtime fix looks good on the paths I checked. One P3 cleanup: src/settings.py still says non-default explicit budgets are clamped to agent_input_token_hard_max, but the current contract/tests keep hard_max auto-only. The is_setting_overridden() docstring also still mentions adaptive budgets, even though agent budgeting now uses value-based budget_is_explicit().

Nit while touching those comments: the new budget comments are pretty verbose. Could you tighten them to the contract only: default 6000 = auto sentinel, non-default positive value = explicit cap, hard_max = auto-only ceiling?

[nsgds]

Thanks — all three addressed (comment-only), plus one I caught while in there.

• settings.py: the explicit-cap line now reads "clamped to the window only — hard_max does not apply to explicit budgets"; dropped the stale "and to hard_max".
• is_setting_overridden docstring: dropped the "adaptive budgets" example; it now notes a materialized default also reads as "present" and points value-sensitive callers to context_budget.budget_is_explicit.
• Tightened the budget comments to the contract only: default 6000 = auto sentinel, non-default positive value = explicit cap, hard_max = auto-only ceiling.

While tightening I noticed the comments were citing #1190 for the contract — but #1190 is the earlier, closed/superseded PR; the behaviour actually merged in #1230 (and settings.py already cited #1230 for the auto sentinel two lines up). The configurable agent_input_token_hard_max setting wasn't in merged #1230 either — it was a reviewer requirement first raised on #1190 and added in #1273. Re-pointed the comments accordingly (#1230 for the contract, #1273 for the configurable ceiling) and corrected the test comment's history.

Will keep comments tight to the contract and the issue refs pointing at the merged source going forward — appreciate the nudges.

[RaresKeY]

LGTM.

This is in my local merge queue. There’s an unrelated merge blocker right now, so I’ll merge it later once that clears.

[pewdiepie-archdaemon]

Looks good. I rechecked the final version after the review fixes: the default budget is now treated as the auto sentinel, fallback context no longer inflates the auto budget, and the known-context helper binds provenance to the value used for budgeting. Focused local suite passed: tests/test_budget_auto_sentinel.py tests/test_context_budget.py tests/test_model_context.py tests/test_context_cache_per_endpoint.py tests/test_llama_server_models_url.py (55 passed).