Repository navigation
Add selectable latency and throughput objectives for manual search spaces - #38
Merged
Merged
Conversation
…aces Split from the automatic-tuning work so the objective syntax can ship on its own. `optimization.objective` minimizes a latency mean or percentile, or maximizes a throughput metric; direction is carried through search, ranking, finalist selection, persistence, reports, retries, reclassification, and redaction. - Move the percentile-request logic into config/objective_percentiles.py and merge into whichever argument spelling the user wrote. - Reject latency statistic keys under `optimization.maximize`, which the sampler would maximize while ranking minimized. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Address independent review findings on the objective change: - Mean extraction reuses the earlier order (outer average, successful mean, outer mean) so runs stored by older versions regenerate with the same score. - Percentile argument merging treats `false` as an omitted flag and rejects duplicate aliases and non-boolean inline values, matching command building. - A zero improvement is reported as 0.0 rather than -0.0. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
brtydse100
marked this pull request as ready for review
November 17, 2026 06:41
brtydse100
added a commit
that referenced
this pull request
Nov 17, 2026
Objectives (#38) and the runtime update (#39) are now on main. Take main's versions of that code, keep only the auto-mode modules and hooks, drop the planning document, move the two auto tests to their own file, and mark the guide experimental. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #34 so the objective syntax can ship without auto mode. It works with existing manual
tunespaces.Behaviour
metric-percentiles. GuideLLM supports the percentiles it emits; others fail validation.optimization.maximizekeeps its behaviour. Latency statistic keys such asend_to_end_latency_ms.p99are now rejected undermaximizewith a pointer toobjective; the sampler would have maximized them while ranking minimized them.Differences from the code in #34
config/objective_percentiles.py. It now merges into whichever argument spelling was written, treatsfalseas an omitted flag, and rejects duplicate aliases and non-boolean inline values.Validation at 88c9e6e
mainat e02d8d9; they come from hardcoded local paths and a missing helper module. Six private auto-mode test files were excluded because that code is not on this branch.mkdocs build --strictpass.Real GPU
RTX 3080 10 GB, WSL2, CUDA 13.0, torch 2.13.0, vLLM 0.30.0, GuideLLM 0.7.4, local OPT-125M FP16. Manual space
max-num-seqsin [2, 4] with baseline 8, TPE with 2 trials, 16 requests of 32 input / 16 output tokens, 2 repeats, temperature 0.Every run recorded a
minimizeobjective, ranked lowest-first, and selected the lowest score, including the case where the baseline won. Report regeneration, reclassification, and retry planning kept the objective, and the server port was released after each case.This is a behaviour check on a small model with 16 requests. The latency differences are not evidence of a tuning gain. Evidence is private at
vTune-private-tests/objective-percentiles-evidence/88c9e6e/.Limitations
maximizeruns asMaximize X (mean).🤖 Generated with Claude Code