Report per-stage distribution statistics for GPU task metrics - #2146
Merged
Merged
Conversation
fixes NVIDIA#2131 Adds sampleCount, min, stddev, cv and maxOverMean to the GPU task metric reports at stage, SQL and application level. Dispersion comes from a second moment kept in the store rather than from retained per-task values: StatisticsMetrics carries welfordSumSqDev, updated in constant time per sample by Welford's method and merged across stages by Chan's parallel form, so cost is per (accumulable, stage) rather than per task. The statistics describe the task updates alone. StatisticsMetrics gains sampleTotal, so a value a stage reports at completion moves only the published total while count, sampleTotal and welfordSumSqDev keep describing one population whatever order the events arrive in. min and max are seeded from the first sample instead of folded against the placeholder zeros a stage completion event leaves behind, and the reports read them through sampleMin and sampleMax, so a stage no task reported publishes blank extrema rather than 0. The three GPU row classes share AggregatedMetricStats, which derives sum, avg, stddev, cv and maxOverMean from the stored fields and divides out the metric's fixed-point scale once. ProfileUtils.optionLongToString is replaced by a generic StringUtils.optionToString, and ToolUtils.formatDoublePrecision is repaired: it used scientific notation and floored instead of rounding. The avg was integer divided and floored while the report contract declared it Double. Compute and render it as a rounded Double. The report contract for the three GPU tables is updated, including the sampled population, which is task attempts rather than logical tasks. StatisticsMetricsSuite covers the store lifecycle and the cross-stage merge against a two-pass reference. GpuMetricReportSuite drives the analyzer over a hand-built event log covering gpuMaxTaskFootprint and a stage whose metric arrives only at completion, neither of which any stored fixture provides. Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #2131
Adds sampleCount, min, stddev, cv and maxOverMean to the GPU task metric reports at stage, SQL and application level. Dispersion comes from a second moment kept in the store rather than from retained per-task values: StatisticsMetrics carries welfordSumSqDev, updated in constant time per sample by Welford's method and merged across stages by Chan's parallel form, so cost is per (accumulable, stage) rather than per task.
The statistics describe the task updates alone. StatisticsMetrics gains sampleTotal, so a value a stage reports at completion moves only the published total while count, sampleTotal and welfordSumSqDev keep describing one population whatever order the events arrive in. min and max are seeded from the first sample instead of folded against the placeholder zeros a stage completion event leaves behind, and the reports read them through sampleMin and sampleMax, so a stage no task reported publishes blank extrema rather than 0.
The three GPU row classes share AggregatedMetricStats, which derives sum, avg, stddev, cv and maxOverMean from the stored fields and divides out the metric's fixed-point scale once. ProfileUtils.optionLongToString is replaced by a generic StringUtils.optionToString, and ToolUtils.formatDoublePrecision is repaired: it used scientific notation and floored instead of rounding.
The avg was integer divided and floored while the report contract declared it Double. Compute and render it as a rounded Double.
The report contract for the three GPU tables is updated, including the sampled population, which is task attempts rather than logical tasks.
StatisticsMetricsSuite covers the store lifecycle and the cross-stage merge against a two-pass reference. GpuMetricReportSuite drives the analyzer over a hand-built event log covering gpuMaxTaskFootprint and a stage whose metric arrives only at completion, neither of which any stored fixture provides.