Repository navigation
feat(metrics): read the three-second tg_3s rate for engine speed - #277
Merged
Merged
Conversation
The whole-task tg average lags a rate that has just changed by minutes; llama.cpp now also prints tg_3s, a three-second rolling figure that answers how fast the engine is generating now. Read tg_3s where the timing line carries a usable one and fall back to tg otherwise, so older engines and other rate sources keep reporting unchanged.
outofcoffee
marked this pull request as ready for review
November 2, 2026 18:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Speed row now answers "how fast is it generating now" instead of showing the whole-task average.
Summary
ParseGenerationSpeedreads llama.cpp's three-secondtg_3srate when the timing line carries a usable one, falling back to the whole-tasktgaverage otherwise — so a rate that has just changed reads now rather than after the task average catches up.tg(and any engine printingtg-shaped timing lines) keep reporting exactly as before.tg_3s— the0.00/-nanllama.cpp prints before a slot's first token — falls through totgrather than quenching the line.prefer-tg-3s-generation-speedproposes the delta toengine-metrics(the "An average is not the current rate" scenario's rejected average becomes the whole-tasktg); docs, internals and the OpenAPI stat description follow the parser.Implementation details
Two narrow patterns tried in precedence order, rather than one alternation: each label's pattern stays separately testable and the existing boundary pins (
\btg\s*=cannot matchtg_3s =,\bstopsprint_tg_3s, and a longertg_30sbreaks the literal) are untouched. The validity filter is shared, so "usable" means the same plausible-positive rate it always did.