Repository navigation
Update runtime targets to vLLM 0.32 and GuideLLM 0.8.1 - #39
Merged
Merged
Conversation
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
brtydse100
marked this pull request as ready for review
November 17, 2026 06:52
brtydse100
added a commit
that referenced
this pull request
Nov 17, 2026
Objectives (#38) and the runtime update (#39) are now on main. Take main's versions of that code, keep only the auto-mode modules and hooks, drop the planning document, move the two auto tests to their own file, and mark the guide experimental. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #34. Moves the
runtimeextra from vLLM 0.30.x / GuideLLM 0.7.x tovllm>=0.32,<0.33andguidellm>=0.8.1,<0.9. No source changes were needed.Changes
pyproject.toml: the runtime extra.docs/reference/compatibility.md: target versions and a dated matrix row.CHANGELOG.md: Unreleased entry.Validation at a91d73b
This head includes
mainwith the objective feature from #38.main; they come from hardcoded local paths and a missing helper module. Five private auto-mode test files were excluded because that code is not on this branch.mkdocs build --strictpasses.Real GPU
RTX 3080 10 GB, WSL2, driver 621.01, CUDA 13.0, torch 2.13.0, vLLM 0.32.0, GuideLLM 0.8.1, local OPT-125M FP16. Manual space
max-num-seqsin [2, 4] with baseline 8, TPE with 2 trials, 2 repeats, 32 input / 16 output tokens, temperature 0.Throughput objective with a P99 TTFT target, 32 requests per run, at b76f75b (before merging
main):Latency-percentile objectives, 16 requests per run, at a91d73b:
No trial failed. JSON, CSV, and HTML were written; report regeneration, reclassification, and retry planning succeeded; the server port was released after each case.
These are short-context compatibility checks on a small model, not performance results. Evidence is private at
vTune-private-tests/runtime032-main-evidence/b76f75b/andvTune-private-tests/objective-percentiles-evidence/a91d73b-runtime032/.Limitations
🤖 Generated with Claude Code