server: remove two host side stalls in the decode loop at high concurrency

[danielhanchen]

Two host side costs in llama-server that stall the decode loop at high concurrency, found by
timing every phase of a step rather than by guessing. No backend, RPC or CUDA file is touched:
the change is measured on the CUDA backend and again on a -DGGML_CUDA=OFF -DGGML_RPC=OFF
build, and it does not depend on anything specific to this fork.

Where the host time goes

One DGX Spark, Qwen3.8-27B UD-Q4_K_XL, --parallel 32, 32 concurrent requests, 128 prompt and
256 generated tokens, greedy. Per decode iteration, milliseconds, and per call in
microseconds. This is host time inside the step, i.e. time in which the GPU has nothing to do.

span	before ms/iter	after ms/iter	before us/call	after us/call
post_decode (total)	13.004	7.000	13004	7000
common_sampler_sample	7.023	6.594	222.9	209.3
result path	5.739	0.257	182.1	8.1
send_partial_response	5.503	0.128	174.6	4.0
queue_results.send()	5.370	0.086	170.4	2.7
add_token	0.006	0.001	0.2	0.0
batch build (decode step)	0.013	0.011	12.9	11.0

queue_results.send() is 63 times cheaper per call and the whole result path 22 times cheaper.
The step's host stall drops from 13.0 ms to 7.0 ms, and what is left is almost entirely
common_sampler_sample, which is O(vocabulary) by construction and is not touched here.

The tracer's own summary of the same two cells, per step over 526 steps:

	step ms	build	submit	sync	post	sampling	send	GPU busy	idle, neither GPU busy
before	309.7	7.7	25.8	262.9	12.9	7.0	5.7	93.1%	21.3 ms
after	302.8	7.5	25.8	262.0	7.0	6.6	0.3	94.9%	15.3 ms

Whole cell throughput

Same node, tracing off, base and new bracketed base / new / base, one server load per arm, all
three concurrencies against each load. Aggregate tokens per second over the closed loop:

concurrency	base	new	base	new vs base bracket
1	11.63	11.45	11.44	inside the bracket, no change
8	53.24	53.22	53.42	inside the bracket, no change
32	93.31	98.99	98.02	+3.5% on the bracket mean, +1.0% on the nearer base

That is what the phase table predicts: 6.0 ms saved out of a 303 ms step is 2%, and there is
nothing to save at 1 or 8 slots because the result path is a per slot cost. The gain grows with
the slot count, because the old path was O(slots^2) per step.

The changes

1. server_response: deliver a result to the one thread waiting for it. Every pending
result lived in one vector behind one condition variable. A send walked the waiting id set
linearly, pushed, then notify_all woke every waiting HTTP thread, each of which re-took the
same mutex and scanned the whole vector before going back to sleep. At N slots that is N
wakeups and N scans per token, N^2 per decode step, all contending for the mutex the decode
thread needs to send the next token. Results are now queued on a per reader waiter: ids
registered together share one waiter, so a send is an O(1) lookup, a push and one wakeup. FIFO
order per reader is preserved, which is what scanning the shared vector from the front did.

2. Only keep per token probabilities when the request asked for them.
server_slot::generated_token_probs is read in exactly one place, send_final_response(), and
only under n_probs > 0. Every other request still pushed a completion_token_output per
token, each with a heap allocated string, into a list that grows for the whole generation and
is then discarded.

Correctness

Greedy, temperature 0, top_k 1, seed 42, cache_prompt false, eight requests (four
prompts, streamed and non streamed), one slot and one request in flight so the batch
composition is fixed, md5 of the concatenated output:

27B on CUDA,  base against base (control)  e2515bd5d500bc6aa47a695daa843b0c / e2515bd5d500bc6aa47a695daa843b0c
27B on CUDA,  base against this branch     e2515bd5d500bc6aa47a695daa843b0c / e2515bd5d500bc6aa47a695daa843b0c
CUDA=OFF RPC=OFF, master against master    000872d79ef3a4e2c1b736d3dacbbed6 / 000872d79ef3a4e2c1b736d3dacbbed6
CUDA=OFF RPC=OFF, master against branch    000872d79ef3a4e2c1b736d3dacbbed6 / 000872d79ef3a4e2c1b736d3dacbbed6

and with four slots and two of the requests streamed concurrently, which is the path the first
change touches, on the same CUDA=OFF RPC=OFF builds:

master  657e06f5c0d3cb5b30688771644b72db
branch  657e06f5c0d3cb5b30688771644b72db

The base against base control is there because with several requests decoded in one batch this
model is not run to run reproducible, so a concurrent harness cannot be used as a correctness
control: it produces two different md5s from the same binary.

KV cache and prompt cache behaviour is untouched. Nothing here goes near pre_decode(), the
common prefix match, the cache reuse path or llama_memory_*.

Build coverage

-DGGML_CUDA=OFF -DGGML_RPC=OFF -DLLAMA_OPENSSL=OFF configures and builds clean, which is the
build the second md5 above was produced with. The CUDA build used for the tables is the
ordinary one.

What is deliberately not in here

The largest single host stall on this workload is not in the decode step at all: it is
create_checkpoint() during prefill. On the same cell it is 331 of the 392 ms that a prefill
iteration spends building its batch, 66 checkpoints of 149 MiB each for 32 prompts, 50.2 ms per
checkpoint, and the longest single batch build in the cell is 1.03 s, which is a direct time to
first token cost. Of those 50.2 ms, 43.7 are the std::vector<uint8_t>::resize() that zero
fills the buffer and only 6.5 are the state copy that follows.

Removing the zero fill looks obvious and is wrong: it made the copy 140 times slower, 6.5 ms to
917 ms per checkpoint, and cost 24% of whole cell throughput. The memset is doing real work, it
faults the destination pages in before the device to host copy touches them. The fix therefore
has to be reuse of an already resident buffer rather than removal of the memset, and that
carries a memory policy decision, so it is left out of this PR. For scale, running the same cell
with --ctx-checkpoints 0 moves median TTFT at 32 concurrent from 6239 ms to 3688 ms at
unchanged throughput.