fix(scheduler): clamp admission to the actual KV pool so oversized prompts fail loudly

[ascorb12]

Addresses the silent-hang half of #111 (the --moe-cache-auto budget-solve policy is deliberately left alone).

Problem

Scheduler._process_one_msg validates an incoming prompt against
self.engine.max_seq_len — the model's advertised context — but never against
the KV pool that was actually allocated (engine.num_pages * page_size). Any
prompt that fits the model's max but exceeds the pool passes admission, is
handed to prefill_manager.add_one_req, and can never be scheduled: no prefill
batch, no log line, no HTTP error. The client waits until its own timeout while
the engine sits idle. #111 demonstrates the boundary tracking num_pages
exactly (8,011 tokens passes, 8,760 hangs, pool 8,209) on a 96 GB RTX PRO 6000,
and with --moe-cache-auto the pool routinely lands near the 8,192
kv_reserve_tokens floor — so any agent-style workload (system prompt + tools
routinely >8k tokens) hits this at defaults.

Fix

Clamp the existing admission bound to the pool and reuse the error path that is
already there (ErrorReplyMsg, code="context_length_exceeded", the
"prompt is too long: N tokens > M" phrasing clients already match on):

pool_tokens = self.engine.num_pages * self.config.page_size
effective_max = min(max_seq_len, pool_tokens)
max_output_len = effective_max - input_len

Three lines plus the two message sites switching {max_seq_len} →
{effective_max}. No new config, no behavior change for prompts that fit.

Caveats considered

• Radix prefix sharing. With cache_type=radix, a prompt slightly over the
  pool could in principle still fit if it shares cached prefix pages. This clamp
  rejects it anyway. That is deliberate: admission cannot see prefix matches
  (matching happens at prefill), the reject is conservative and loud, and the
  alternative is the silent hang this PR removes. If prefix-aware admission is
  wanted later, it belongs in the prefill manager.
• SWA pools. For sliding-window hybrids the binding constraint for a long
  prompt is the full-attention pool, which is what engine.num_pages describes;
  window pools are per-layer bounded and smaller per token. The clamp uses the
  full pool, matching the observed hang boundary in Requests longer than the KV pool are queued forever with no error — and --moe-cache-auto leaves only ~8k tokens of KV on a 96GB GPU #111.
• The second half of Requests longer than the KV pool are queued forever with no error — and --moe-cache-auto leaves only ~8k tokens of KV on a 96GB GPU #111 (the --moe-cache-auto budget solve starving KV) is
  not addressed here — that is a policy question. This PR only makes the
  resulting limit visible instead of a hang.

Tested

• Hardware: Windows 11, RTX 4090 24 GB, 63 GB RAM, driver 610.88.
• Build: v0.1.2 wheel with this patch applied to scheduler.py from the v0.1.2
  tag (byte-identical to main in this region), shadowing the compiled module.
• Primary proof: openai/gpt-oss-20b, --num-tokens 8192, no
  --max-seq-len-override — so the server advertised context_length=131072
  while the pool held 8,192. This is exactly the configuration Requests longer than the KV pool are queued forever with no error — and --moe-cache-auto leaves only ~8k tokens of KV on a 96GB GPU #111 hangs on.
  • Oversized prompt (9,568 tokens): rejected in <0.1 s, HTTP 400,
    {"message":"prompt is too long: 9568 tokens > 8192 maximum (prompt + generation); shorten the prompt or increase the KV cache budget", "code":"context_length_exceeded"}. The 8192 in that message can only
    come from the new clamp — stock code compares against the advertised
    131,072 and admits the prompt into the queue it can never leave.
  • Large-but-fitting prompt (6,083 tokens): completed normally in 4.4 s.
  • No regression on normal prompts: 299-token generation at 129/95 tok/s
    (cold/warm), identical behavior to the unpatched module for anything
    that fits.

Per the AI policy in CONTRIBUTING: this change was produced with AI assistance;
the submitter has reviewed and tested everything in it and takes responsibility
for it.

🤖 Generated with Claude Code