fix(model_context): query Ollama /api/show for actual serving context by MAK-9 · Pull Request #753 · odysseus-dev/odysseus

MAK-9 · 2026-06-01T20:38:33Z

Summary

Add a POST /api/show probe to _query_context_length() for local endpoints
Runs after the llama.cpp /slots attempt and before /v1/models
Reads model_info.{arch}.context_length — the value Ollama uses as its default num_ctx
Silently no-ops on non-Ollama local servers (404 or connection error → falls through)
Remote/cloud endpoints are not affected (probe is local-only)
Add TestQueryContextLength class with 6 tests covering all probe paths

Why

Ollama doesn't expose context length via /slots (llama.cpp-only) or /v1/models (no context_length field). The code fell through to KNOWN_CONTEXT_WINDOWS, returning 131072 for qwen3 models even when Ollama was actually serving them at 40960 tokens (the GGUF metadata value). Context compaction fired far too late and Ollama silently truncated long conversations.

/api/show is Ollama's native introspection endpoint and returns the correct value.

Test plan

python -m pytest tests/test_model_context.py -v — 28 tests pass
Live: get_context_length("http://localhost:11434/v1/chat/completions", "qwen3:14b") returns 40960 (was 131072)
Non-Ollama local server: /api/show returns 404 → code still resolves via /v1/models or known fallback

🤖 Generated with Claude Code

…outs

Odysseus resolves context window size via /slots (llama.cpp) then /v1/models. Ollama exposes neither with a context_length field, so the code fell through to the hardcoded KNOWN_CONTEXT_WINDOWS map — returning 131072 for qwen3 even when Ollama was serving the model at 40960 tokens (the value in the GGUF metadata). This caused context compaction to fire far too late, letting Ollama silently truncate conversations. Add a POST /api/show probe between the /slots and /v1/models attempts. Ollama returns model_info with a {arch}.context_length field that matches what it actually uses as the default num_ctx. The probe is local-only and fails silently on non-Ollama servers (e.g. llama.cpp, vLLM). Add TestQueryContextLength covering: /api/show hit, 404 fallthrough, connection error fallthrough, remote endpoint skip, known-model fallback, and unknown-model DEFAULT_CONTEXT fallback. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

pewdiepie-archdaemon · 2026-06-02T01:25:38Z

The Ollama /api/show context probe is a good idea and likely useful for preventing over-large prompts, but this PR currently carries unrelated Deep Research/settings/time-limit changes and is conflicting.

Please split this down to just src/model_context.py + tests/test_model_context.py for the Ollama context-length probe. Requirements for that focused version:

only probe /api/show for local/Ollama-looking endpoints
non-Ollama OpenAI-compatible servers should fall through cleanly
no extra latency on cloud endpoints
keep the live check optional/manual

Once narrowed, this should be much easier to review.

pewdiepie-archdaemon · 2026-06-02T03:47:30Z

The Ollama /api/show context probe is a good target fix, but this PR also changes Deep Research time limits/UI and is currently conflicting. Please split: one small PR for src/model_context.py + tests/test_model_context.py only, and a separate research-time-limit PR if still needed. The Ollama context piece should be much easier to merge once isolated.

_build_ollama_payload sends options.temperature and options.num_predict to /api/chat, but never options.num_ctx. Ollama defaults num_ctx to 2048 when the option is omitted, so prompts going to any Ollama backend are silently truncated there regardless of the model's actual capability. Thread the discovered context length through the three call sites (llm_call, llm_call_async, stream_llm) and emit options.num_ctx when it is known and larger than 2048. The builder filters out the DEFAULT_CONTEXT fallback (128000) so we don't lie to Ollama about models whose window we couldn't actually discover. Sister fix to PR odysseus-dev#753 — that PR teaches the compactor the right budget, this one tells Ollama to actually use that budget on the way in.

_build_ollama_payload sends options.temperature and options.num_predict to /api/chat, but never options.num_ctx. Ollama defaults num_ctx to 2048 when the option is omitted, so prompts going to any Ollama backend are silently truncated there regardless of the model's actual capability. Thread the discovered context length through the three call sites (llm_call, llm_call_async, stream_llm) and emit options.num_ctx when it is known and positive. The builder filters out the DEFAULT_CONTEXT fallback (128000) so we don't lie to Ollama about models whose window we couldn't actually discover. The issue's literal 'when > 2048' heuristic is dropped: a model with a real context smaller than 2048 would OOM if Ollama used its default, so we pass the real value regardless of size. Matches how src/context_compactor.py uses the same helper. Sister fix to PR odysseus-dev#753 — that PR teaches the compactor the right budget, this one tells Ollama to actually use that budget on the way in.

_build_ollama_payload sends options.temperature and options.num_predict to /api/chat, but never options.num_ctx. Ollama defaults num_ctx to 2048 when the option is omitted, so prompts going to any Ollama backend are silently truncated there regardless of the model's actual capability. Thread the discovered context length through the three call sites (llm_call, llm_call_async, stream_llm) and emit options.num_ctx when it is known and positive. The builder filters out the DEFAULT_CONTEXT fallback (128000) so we don't lie to Ollama about models whose window we couldn't actually discover. The issue's literal 'when > 2048' heuristic is dropped: a model with a real context smaller than 2048 would OOM if Ollama used its default, so we pass the real value regardless of size. Matches how src/context_compactor.py uses the same helper. Sister fix to PR #753 — that PR teaches the compactor the right budget, this one tells Ollama to actually use that budget on the way in.

pewdiepie-archdaemon · 2026-06-02T12:01:19Z

The Ollama /api/show context probing is useful, but this PR is bundled with already-merged Deep Research timeout changes and extraction timeout changes. Please split/rebase it into a focused PR that only adds Ollama context detection in src/model_context.py plus the relevant tests. That smaller PR should be straightforward to review.

_build_ollama_payload sends options.temperature and options.num_predict to /api/chat, but never options.num_ctx. Ollama defaults num_ctx to 2048 when the option is omitted, so prompts going to any Ollama backend are silently truncated there regardless of the model's actual capability. Thread the discovered context length through the three call sites (llm_call, llm_call_async, stream_llm) and emit options.num_ctx when it is known and positive. The builder filters out the DEFAULT_CONTEXT fallback (128000) so we don't lie to Ollama about models whose window we couldn't actually discover. The issue's literal 'when > 2048' heuristic is dropped: a model with a real context smaller than 2048 would OOM if Ollama used its default, so we pass the real value regardless of size. Matches how src/context_compactor.py uses the same helper. Sister fix to PR odysseus-dev#753 — that PR teaches the compactor the right budget, this one tells Ollama to actually use that budget on the way in.

github-actions · 2026-06-04T23:03:18Z

⚠️ PR description — action needed

The following required sections are missing or incomplete. Please update the PR description to address them:

Linked Issue — add a reference like Fixes #NNN, a bare #NNN, or a link to the issue.
Type of Change — check at least one box.
Checklist — check the duplicate-search box to confirm you searched existing issues and PRs.
How to Test — explain how a reviewer can verify this change. Numbered steps, the commands you ran, or a short code block all work — give a sentence or two of real detail (not just "tested locally").

This comment is deleted automatically once all sections are complete.

_build_ollama_payload sends options.temperature and options.num_predict to /api/chat, but never options.num_ctx. Ollama defaults num_ctx to 2048 when the option is omitted, so prompts going to any Ollama backend are silently truncated there regardless of the model's actual capability. Thread the discovered context length through the three call sites (llm_call, llm_call_async, stream_llm) and emit options.num_ctx when it is known and positive. The builder filters out the DEFAULT_CONTEXT fallback (128000) so we don't lie to Ollama about models whose window we couldn't actually discover. The issue's literal 'when > 2048' heuristic is dropped: a model with a real context smaller than 2048 would OOM if Ollama used its default, so we pass the real value regardless of size. Matches how src/context_compactor.py uses the same helper. Sister fix to PR odysseus-dev#753 — that PR teaches the compactor the right budget, this one tells Ollama to actually use that budget on the way in.

RaresKeY · 2026-07-20T13:03:32Z

Thanks for working on this. I’m closing this version because it remains conflicting and still bundles unrelated Deep Research changes after several requests to split it.

The provider-metadata side is now covered by the capability work in #2737, through merged #2739 and the continuing work in #5576. However, /api/show.model_info.*.context_length is model metadata, not the context currently allocated by the server, so it should not be treated as the “actual serving context.” Ollama reports the loaded allocation through /api/ps.context_length, while num_ctx can be configured separately.

If runtime prompt budgeting still needs a fix, it should return as a current, focused PR limited to src/model_context.py and its tests, keeping model limits and effective runtime allocation distinct.

redpersongpt and others added 4 commits June 1, 2026 16:59

Honor Deep Research time limits

3f3b2df

Allow longer deep research extraction timeouts

5f75484

Merge PR odysseus-dev#651: Allow longer Deep Research extraction time…

ffc55d3

…outs

This was referenced Jun 2, 2026

Ollama requests omit num_ctx, silently truncating prompts at 2048 tokens regardless of model #909

Closed

fix(llm): pass num_ctx to Ollama so prompts are not silently truncated at 2048 (fixes #909) #911

Closed

adityaT39 mentioned this pull request Jun 2, 2026

fix(ollama): pass num_ctx to prevent silent 2048-token truncation #960

Closed

This was referenced Jun 2, 2026

fix(llm): thread num_ctx into Ollama requests (#909) #1049

Closed

fix(llm): thread num_ctx into Ollama requests (#909) #1056

Merged

This was referenced Jun 2, 2026

Proposal: make agent_input_token_budget adapt to model context window (default 6000 silently caps long-context models) #1170

Closed

fix(agent): adaptive default for agent_input_token_budget (closes #1170) #1190

Closed

alteixeira20 added bug Something isn't working stale pr This pr need a merge from main, it is out of sync labels Jun 3, 2026

alteixeira20 changed the base branch from main to dev June 4, 2026 23:03

github-actions Bot added the needs work PR description incomplete — please update before review label Jun 4, 2026

RaresKeY mentioned this pull request Jun 5, 2026

feat(models): define capability schema and readers #2739

Merged

15 tasks

maxmilian mentioned this pull request Jul 6, 2026

Static context-window table overrides the endpoint's actual serving context — silent truncation in long agent sessions #5193

Open

3 tasks

RaresKeY closed this Jul 20, 2026

RaresKeY mentioned this pull request Jul 20, 2026

models: normalize loaded runtime context allocation evidence #5628

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(model_context): query Ollama /api/show for actual serving context#753

fix(model_context): query Ollama /api/show for actual serving context#753
MAK-9 wants to merge 4 commits into
odysseus-dev:devfrom
MAK-9:fix/ollama-api-show-context-length

MAK-9 commented Jun 1, 2026

Uh oh!

pewdiepie-archdaemon commented Jun 2, 2026

Uh oh!

pewdiepie-archdaemon commented Jun 2, 2026

Uh oh!

pewdiepie-archdaemon commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026

Uh oh!

RaresKeY commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

MAK-9 commented Jun 1, 2026

Summary

Why

Test plan

Uh oh!

pewdiepie-archdaemon commented Jun 2, 2026

Uh oh!

pewdiepie-archdaemon commented Jun 2, 2026

Uh oh!

pewdiepie-archdaemon commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 4, 2026

Uh oh!

RaresKeY commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants