From d23e21277b50e4f7285fe8540d816199b124e572 Mon Sep 17 00:00:00 2001 From: sunilaimlrymec-code Date: Sat, 24 Oct 2026 13:34:53 +0530 Subject: [PATCH] docs: add dashboard trend charts page --- docs-site/docs/reporting/dashboard-trends.md | 50 ++++++++++++++++++++ docs-site/sidebars.js | 19 ++++---- 2 files changed, 61 insertions(+), 8 deletions(-) create mode 100644 docs-site/docs/reporting/dashboard-trends.md diff --git a/docs-site/docs/reporting/dashboard-trends.md b/docs-site/docs/reporting/dashboard-trends.md new file mode 100644 index 0000000..74ae971 --- /dev/null +++ b/docs-site/docs/reporting/dashboard-trends.md @@ -0,0 +1,50 @@ +# Dashboard Trend Charts + +Selenium Boot's HTML report includes pass-rate and test-duration trend charts to help you understand how your test suite changes across runs. + +## Data Source + +The trend charts use historical test metrics stored as JSON files in the metrics-history directory. `HtmlReportGenerator.buildTrendData()` reads these files, sorts them by filename, and prepares the data for the HTML report. + +- **Pass-rate trend:** Shows how the test pass rate changes across test runs. +- **Duration trend:** Shows how test execution time changes across test runs. + +## Configuring Metrics History + +The trend charts depend on persisted metrics history. You can configure the history directory using `flakiness.historyDir` or the JVM system property `-Dseleniumboot.history.dir`. + +The history directory is resolved in the following order: + +1. **JVM system property:** `-Dseleniumboot.history.dir=/path/to/history` takes precedence when the property is nonblank. +2. **Flakiness configuration:** If the JVM property is absent or blank, Selenium Boot uses `flakiness.historyDir` when configured. +3. **Default directory:** If neither setting provides a directory, Selenium Boot uses `metrics-history` under its base directory. + +Keep the history directory available across test runs so that reports can include metrics from previous executions. + +## The 20-Run Window + +The trend charts display up to the 20 most recent runs. The report generator calls `buildTrendData(20)` to limit the history included in the charts. + +History files are sorted by filename, and only the latest 20 files are selected when more than 20 are available. Older runs fall outside the selected window as new history files are added. + +## Why Two Runs Are Required + +A trend requires data from multiple test runs. With only one run, the report has no previous run to compare against. + +The trend charts remain hidden until metrics from at least two runs are available. Once history contains two or more runs, the charts can show changes in pass rate and duration. + +## Preserving History in CI + +CI jobs often use fresh workspaces or disposable containers. If the metrics-history directory is lost after each run, later reports cannot use the earlier metrics to display trends. + +To preserve history across CI runs: + +1. Configure a consistent metrics-history directory. +2. Restore the previous history directory before running the tests. +3. Execute the tests and generate the HTML report. +4. Save the updated history directory as a CI artifact or in persistent storage. +5. Restore the saved history before the next test run. + +A CI cache can also be used, but caches may be evicted. Use retained artifacts or persistent storage when reliable history is required. + +Ensure that each run's metrics-history JSON files remain available to subsequent runs so the report can build a meaningful trend. diff --git a/docs-site/sidebars.js b/docs-site/sidebars.js index a8450a7..1104c67 100644 --- a/docs-site/sidebars.js +++ b/docs-site/sidebars.js @@ -87,14 +87,17 @@ const sidebars = { 'extensibility/report-adapters', ], }, - { - type: 'category', - label: 'Reporting', - items: [ - 'reporting/html-report', - 'reporting/junit-xml', - ], - }, + + { + type: 'category', + label: 'Reporting', + items: [ + 'reporting/html-report', + 'reporting/junit-xml', + 'reporting/dashboard-trends', + ], + }, + 'junit5', 'cucumber', 'email-verification',