Skip to content

Replace markdown result files with CSV + JSON output#46

Merged
tbroden84 merged 3 commits intomainfrom
csv-raw-results
May 4, 2026
Merged

Replace markdown result files with CSV + JSON output#46
tbroden84 merged 3 commits intomainfrom
csv-raw-results

Conversation

@tbroden84
Copy link
Copy Markdown
Contributor

@tbroden84 tbroden84 commented Apr 29, 2026

WHY are these changes introduced?

Markdown was a poor storage format for benchmark results: parsing required fragile regex, aggregation logic (medians) was duplicated between bash and Python. Moving to CSV + JSON separates raw measurements from metadata, lets aggregate_results.py compute statistics directly, and makes the data easy to process with standard tools.

Reference: #43

WHAT is this pull request doing?

The benchmark scripts (run_multiple_latency_tests.sh, run_multiple_throughput_tests.sh) now write two files instead of one markdown file:

  • a CSV with one row per run (raw measurements, no pre-aggregated summaries)
  • a JSON file with test configuration metadata (instance type, version, duration, sizes, rate limits)

aggregate_results.py is updated to read these CSV+JSON pairs and compute medians using statistics.median.

HOW was this pull request tested?

Small test run: #50

@tbroden84 tbroden84 marked this pull request as ready for review April 30, 2026 08:27
@tbroden84 tbroden84 requested a review from a team as a code owner April 30, 2026 08:27
@tbroden84 tbroden84 merged commit f5e821d into main May 4, 2026
8 checks passed
@tbroden84 tbroden84 deleted the csv-raw-results branch May 4, 2026 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants