Skip to content

metrics: report what is free unconditionally - #4766

Open
oharboe wants to merge 1 commit into
The-OpenROAD-Project:masterfrom
oharboe:metrics-free-always
Open

oharboe wants to merge 1 commit into
The-OpenROAD-Project:masterfrom
oharboe:metrics-free-always

Conversation

@oharboe

@oharboe oharboe commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

Report what is free unconditionally. With SKIP_REPORT_METRICS=1, report_metrics wrote no metrics at all, so a flow skipping it for speed also lost design area, instance counts and utilization. Those are a walk over the instances and need no timer. They are now written before the skip; SKIP_REPORT_METRICS skips only what costs time.

What SKIP_REPORT_METRICS=1 skips, measured on asap7/gcd (ms, summed over the stage's report_metrics calls, DETAILED_METRICS=1):

call floorplan detailed place cts grt final under log_cmd
report_power 49 48 65 203 60 yes
first report_tns (the timing update) 0.1 23 49 95 8 yes
input→register find_timing_paths ×2 – 17 42 143 94 yes (eval)
register→register report_checks ×2 – 14 33 115 67 yes (eval)
report_erc_metrics – 17 27 62 20 yes
report_check_types – 5 7 20 6 yes
report_clock_min_period 1.6 1.6 6 18 – yes
report_clock_skew, report_wns, report_worst_slack, unfiltered report_checks, other *_metric ≤ 4 each no

report_design_area_metrics, now unconditional: 0.07 ms. The tools' own counters (gpl, dpl, grt, drt, repair buffer counts, warnings) never went through report_metrics and are written either way.

The calls that cost time even on gcd run under log_cmd, so on a large design the log names any that takes five seconds or more. The register path searches go through log_cmd eval { ... } so the log prints the command, not the expanded register list.

Tests, asap7/gcd:

  • metrics on vs SKIP_REPORT_METRICS=1: byte-identical .odb at every stage and identical 6_final.v;
  • with SKIP_REPORT_METRICS=1, every stage's metrics .json has design__instance__area and no timing__setup__ws;
  • with metrics on, the metric keys are the same as before the change.

tclfmt/tclint 0.7.0, generate-variables-docs.py, yaml_to_json.py and yamlfix leave the tree unchanged.

Integration test: The-OpenROAD-Project/bazel-orfs#1433.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the behavior of SKIP_REPORT_METRICS so that it only skips metrics requiring heavy analysis (such as timing, power, and clock skew) while still reporting basic metrics like design area, instance counts, and utilization. To support this, report_design_area_metrics is now called before checking the SKIP_REPORT_METRICS flag, and various reporting commands in report_metrics.tcl have been wrapped with log_cmd. Documentation and variable definitions have been updated accordingly. The reviewer pointed out that the path reports/<stage>_*.rpt in the description contains <stage>, which resembles an HTML tag and could cause rendering issues in Markdown; they suggested wrapping it in backticks.

Comment thread flow/scripts/variables.yaml Resolved
With SKIP_REPORT_METRICS=1, report_metrics reported nothing at all, so a
flow that skips it for speed also lost design area, instance counts and
utilization, though those are a walk over the instances that needs no
timer.

Report them unconditionally, before the skip. SKIP_REPORT_METRICS now
skips only what costs time: the timing update behind setup and hold WS
and TNS, fmax, the max slew/cap/fanout checks and clock skew; the power
analysis; and the reports/<stage>_*.rpt text reports.

The calls that take measurable time even on asap7/gcd, 5 to 200 ms each
on a few hundred cells, run under log_cmd, so on a large design the log
names any that takes five seconds or more. The four register path
searches go through `log_cmd eval { ... }`, so the log prints the
command rather than the expanded register list.

On asap7/gcd, metrics on (DETAILED_METRICS=1) and off give
byte-identical .odb files at every stage; with SKIP_REPORT_METRICS=1
every stage's metrics .json has the design area and no timing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@oharboe
oharboe force-pushed the metrics-free-always branch from bc6e396 to f1a5274 Compare October 29, 2026 14:33
@openroad-ci

openroad-ci commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

🔍 QoR check

Metrics reflect the PR merge build — i.e. what will land on the target branch.

Commit 9a91cc9 · Jenkins build #3 · Baseline: build · View build on dashboard

62 design(s) checked — 1 with regression(s), 1 without a comparable baseline.

❌ asap7/aes-block base — 2 failing metric(s)
Metric target base delta limit band
cts__timing__setup__tns -2405.450 -409.639 -487.212% -499.639 PeriodPadding 20.000%
cts__timing__setup__ws -57.057 -27.467 -107.733% -49.967 PeriodPadding 5.000%

@oharboe
oharboe marked this pull request as ready for review November 2, 2026 17:16
@oharboe
oharboe requested a review from maliberty November 2, 2026 17:16
@oharboe

oharboe commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator Author

@maliberty I want as much free(fast) information as possible with the fewest ORFS variables

Hence I changed SKIP_REPORT_METRICS to report what it can for free and only disable reporting that takes time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants