[sfxnz]

Summary

Concurrency streams (LLM inference). While a run is going, every request in flight is a live card: its text types out with the request's state, TTFT, tok/s and token progress. Above the cards: aggregate output with a trend line, a dot per stream and a level rail. A new Sweep concurrency option runs 1, 2, 4 … N streams in turn. After the run: per-level charts and a table (output tok/s, per-request decode, TTFT p50/p95, scaling efficiency), a waiting-vs-decoding timeline for every request, and each stream's final output.

Tool-call eval (new benchmark). Runs tool-eval-bench 2.8.0 in-process inside a payload, so scores, fingerprints and per-scenario verdicts match its CLI. A tool-calling preflight stops runs against servers without tool parsing (otherwise vLLM without --enable-auto-tool-choice still scores "★★★★ Good" with 93 of 97 scenarios silently excluded), and scores always carry their completion rate. Live scenario board; afterwards the score and rating, safety gate, categories and capabilities, a searchable scenario list with upstream's pass/fail criteria and traces, and the upstream result JSON as a download.

Plumbing.

• NodeBenchState.live (replaced) and .detail (merged), with size and nesting guards; persisted with the run, left out of run lists.
• A 32 MiB per-browser byte budget in the console hub.
• The agent drains a payload's output after terminating it, so final snapshots survive a cancel.
• The LLM payload exits on SIGTERM in milliseconds.

Behaviour changes

• Every LLM level now sends at least one request per stream (requests per level = max(requests, level)). Baselines key on the requests actually sent, so older runs with fewer requests than streams aren't compared with new ones. All other existing baselines still match; the new sweep param is left out of the key while off.
• New dependency: tool-eval-bench==2.8.0 plus rich, pygments, markdown-it-py and mdurl, all hash-pinned from PyPI. Existing pins are unchanged.

Test plan

• Server: ruff, ruff format, mypy, pytest -q (333 tests). test_builtin_io_random_workers_do_not_share_a_gil failed once under parallel load; it passes alone and also failed once before this change.
• Web: typecheck, lint, 628 unit tests, production build. Startup JS is 154.2 KB gzip (budget 200 KB). The benchmarks lazy chunk grew from 31.5 to 55.0 KB gzip.
• e2e: all 38 Playwright tests against the demo, including the new sweep, cancel, History and tool-eval flows and axe checks in both themes.
• End to end: a real dev agent and console against fake model servers. At 32 streams, live updates are about 17 KB at 2 Hz and payload CPU is about 4%; rates match the fake's timing. tool-eval fault modes covered: HTTP 400 without parser, 503 exclusions, <think> in answers, unsafe TC-34.
• Not yet run against a real model endpoint. The nodes were busy with measured workloads, so only fakes and the demo were used.

🤖 Generated with Claude Code