Put the benchmark speedup table on the run summary page

[sbryngelson]

The speedup table only exists in the step log today, so reading it means opening the right step of the right matrix leg. This also writes it to $GITHUB_STEP_SUMMARY, which GitHub renders inline on the job page.

Rendered:

Benchmark: phoenix gpu acc

Speedups from master/bench-gpu-acc.yaml to pr/bench-gpu-acc.yaml; greater than 1 is faster.

Case	Pre Process	Simulation	Post Process
5eq_rk3_weno3_hllc	Exec: 1.02	Exec: 0.98 & Grind: 0.94	Exec: 1.00
ibm	Exec: 1.01	Exec: 1.05 & Grind: 1.07	N/A

Below threshold

• grind speedup 0.94 < 0.95 for simulation in 5eq_rk3_weno3_hllc

Notes on the shape of it:

• The rows are captured as table.add_row is called, so the summary cannot drift from the table printed to stdout - there is one source for both.
• The sub-threshold warnings (exec < 0.90, grind < 0.95) are collected and listed underneath. Those are the actionable part and are currently interleaved with everything else in the log.
• MFC_BENCH_SUMMARY_LABEL carries the matrix leg (cluster, device, interface) into the heading. Each leg writes its own summary, and the leg is not otherwise identifiable on the page.
• Everything still goes to stdout exactly as before; this only adds.
• Outside Actions GITHUB_STEP_SUMMARY is unset and the function returns immediately, so local ./mfc.sh bench_diff is unchanged.

Testing

./mfc.sh precheck passes 7/7. The writer was exercised directly with a stub row set and a warning, against a temp file standing in for $GITHUB_STEP_SUMMARY - output above is that run, pasted verbatim - and again with the variable unset to confirm it is a no-op locally.

What I have not done is run it through a real benchmark job; that needs a cluster allocation and a master-vs-PR pair. The failure mode if something is off is a missing or malformed summary, not a failed benchmark, since the write happens after the table is printed and the exit status is unchanged.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Adds a GitHub Actions step summary output for benchmark speedup results so they’re visible on the job summary page (instead of only in step logs).

Changes:

• Introduces a _write_step_summary(...) helper to render the benchmark speedup table and sub-threshold warnings into $GITHUB_STEP_SUMMARY.
• Collects per-row table data and warning strings during bench_diff execution and writes them after printing the Rich table.
• Adds MFC_BENCH_SUMMARY_LABEL to the bench workflow step to label the matrix leg in the summary heading.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
toolchain/mfc/bench.py	Collects table rows/warnings and writes a Markdown summary to $GITHUB_STEP_SUMMARY.
.github/workflows/bench.yml	Sets an env var to label each matrix leg in the job summary output.

Suppressed comments (1)

toolchain/mfc/bench.py:1

• The exec/grind thresholds are now duplicated across comparison logic and multiple human-readable strings (and use slightly different formatting: 0.9 vs 0.90). To avoid drift if thresholds change, define constants (e.g., EXEC_SPEEDUP_MIN, GRIND_SPEEDUP_MIN) and use them both in the comparisons and in the warning/summary messages (formatting them consistently).

import dataclasses

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.