Prerequisites
Install Method
Docker
Operating System
macOS (Apple Silicon), Docker Desktop
Steps to Reproduce
- Serve a local model with a context window smaller than its advertised maximum — e.g. a model advertising 262144 served at
num_ctx=32768.
- Hold a long conversation and watch the Context Window meter in the chat UI.
Expected Behaviour
The meter measures usage against the window the model is actually being served with, and says something when history is compacted.
Actual Behaviour
1. The meter measures against the wrong number. It shows usage against the model's advertised maximum, not the served window. Here: the meter read 25 963 / 128 000 — "20%" — while the real window was 30 000, i.e. ~87% full plus system prompt, tools and reasoning overhead. We were at the ceiling while the meter reported plenty of room.
The symptom at the ceiling is empty replies on near-zero latency, which reads as a crash rather than a full window. (The underlying error is additionally swallowed by the streamingTTS ReferenceError, reported separately — together they make a full context window nearly undiagnosable from the UI.)
2. Compaction is silent. When trim_for_context compacts history, the used-token count simply drops. From the user's side, part of the conversation has been removed with no notice. Our first read of it was "something cleared my context and I don't know what" — which is exactly the wrong thing for a user to have to guess about.
Additional Information
Two asks:
- Measure against the effective window. For Ollama the served window is available (
/api/ps reports the loaded window; a model's own num_ctx parameter overrides the environment default). Where the served window cannot be determined, saying so beats displaying a number derived from a different quantity.
- Mark compaction visibly. A marker in the transcript, or a note next to the meter, when history has been trimmed. Silent truncation of a conversation is a surprising thing for a UI to do without a word.
The general principle: a meter that reports a percentage of the wrong denominator is worse than no meter, because it is trusted.
Are you willing to submit a fix?
Yes — the fix described above is running in production here and I can open a PR.
Prerequisites
devbranch.Install Method
Docker
Operating System
macOS (Apple Silicon), Docker Desktop
Steps to Reproduce
num_ctx=32768.Expected Behaviour
The meter measures usage against the window the model is actually being served with, and says something when history is compacted.
Actual Behaviour
1. The meter measures against the wrong number. It shows usage against the model's advertised maximum, not the served window. Here: the meter read
25 963 / 128 000— "20%" — while the real window was 30 000, i.e. ~87% full plus system prompt, tools and reasoning overhead. We were at the ceiling while the meter reported plenty of room.The symptom at the ceiling is empty replies on near-zero latency, which reads as a crash rather than a full window. (The underlying error is additionally swallowed by the
streamingTTSReferenceError, reported separately — together they make a full context window nearly undiagnosable from the UI.)2. Compaction is silent. When
trim_for_contextcompacts history, the used-token count simply drops. From the user's side, part of the conversation has been removed with no notice. Our first read of it was "something cleared my context and I don't know what" — which is exactly the wrong thing for a user to have to guess about.Additional Information
Two asks:
/api/psreports the loaded window; a model's ownnum_ctxparameter overrides the environment default). Where the served window cannot be determined, saying so beats displaying a number derived from a different quantity.The general principle: a meter that reports a percentage of the wrong denominator is worse than no meter, because it is trusted.
Are you willing to submit a fix?
Yes — the fix described above is running in production here and I can open a PR.