Skip to content

Update runtime targets to vLLM 0.32 and GuideLLM 0.8.1 - #39

Merged
brtydse100 merged 3 commits into
mainfrom
chore/runtime-vllm-032
Nov 17, 2026
Merged

brtydse100 merged 3 commits into
mainfrom
chore/runtime-vllm-032

Conversation

@brtydse100

@brtydse100 brtydse100 commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Split out of #34. Moves the runtime extra from vLLM 0.30.x / GuideLLM 0.7.x to vllm>=0.32,<0.33 and guidellm>=0.8.1,<0.9. No source changes were needed.

Changes

  • pyproject.toml: the runtime extra.
  • docs/reference/compatibility.md: target versions and a dated matrix row.
  • CHANGELOG.md: Unreleased entry.

Validation at a91d73b

This head includes main with the objective feature from #38.

  • Private and public suites on vLLM 0.32.0 / GuideLLM 0.8.1 in WSL: 1,636 passed, 7 failed, 2 collection errors. The 7 failures and 2 errors also occur on main; they come from hardcoded local paths and a missing helper module. Five private auto-mode test files were excluded because that code is not on this branch.
  • mkdocs build --strict passes.

Real GPU

RTX 3080 10 GB, WSL2, driver 621.01, CUDA 13.0, torch 2.13.0, vLLM 0.32.0, GuideLLM 0.8.1, local OPT-125M FP16. Manual space max-num-seqs in [2, 4] with baseline 8, TPE with 2 trials, 2 repeats, 32 input / 16 output tokens, temperature 0.

Throughput objective with a P99 TTFT target, 32 requests per run, at b76f75b (before merging main):

Backend Baseline (tok/s) Tuned trials (tok/s) Requests on selected config
vLLM Bench Serve 510.85 546.36, 302.82 64 of 64
GuideLLM 173.37 174.87, 171.32 64 of 64

Latency-percentile objectives, 16 requests per run, at a91d73b:

Case Baseline (ms) Tuned trials (ms) Selected
vLLM Bench Serve, P55 end-to-end 95.44 94.17, 99.80 94.17
vLLM Bench Serve, P99 end-to-end 342.56 294.80, 322.82 294.80
GuideLLM, P99 end-to-end 290.41 311.34, 320.20 baseline

No trial failed. JSON, CSV, and HTML were written; report regeneration, reclassification, and retry planning succeeded; the server port was released after each case.

These are short-context compatibility checks on a small model, not performance results. Evidence is private at vTune-private-tests/runtime032-main-evidence/b76f75b/ and vTune-private-tests/objective-percentiles-evidence/a91d73b-runtime032/.

Limitations

  • vLLM 0.30.x and GuideLLM 0.7.x fall outside the new range.
  • Native Linux, multi-GPU, and long-context runs were not exercised.

🤖 Generated with Claude Code

brtydse100 and others added 3 commits November 17, 2026 08:29
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@brtydse100
brtydse100 marked this pull request as ready for review November 17, 2026 06:52
@brtydse100
brtydse100 merged commit 49ba9c2 into main Nov 17, 2026
5 checks passed
brtydse100 added a commit that referenced this pull request Nov 17, 2026
Objectives (#38) and the runtime update (#39) are now on main. Take main's
versions of that code, keep only the auto-mode modules and hooks, drop the
planning document, move the two auto tests to their own file, and mark the
guide experimental.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant