Skip to content

docs: the full run's 180 run files, and where its transcripts are - #60

Merged
OmarAlex24 merged 1 commit into
mainfrom
docs/bench-full-run-data
Nov 16, 2026
Merged

OmarAlex24 merged 1 commit into
mainfrom
docs/bench-full-run-data

Conversation

@OmarAlex24

Copy link
Copy Markdown
Owner

Until now the tables of the full comparison run were in the repo, but the files they are built from were only on the machine that ran it.

  • reports/2026-11-16-full-v0.3.0/runs/: the run.json of each of the 180 runs (4 MB), copied unchanged.
  • The note: the command that rebuilds report.md and report.json from them, and a link to the release that holds the transcripts, snapshots and logs (bench-full-v0.3.0-2026-11-16, 110 MB). That release is not marked latest, so install.sh still gets v0.3.0.
  • bench/compare/README.md: one line says a report folder now also keeps the run files.

Before publishing, every file of the run was scanned, archives opened: no API key, token or host path; every name and e-mail address belongs to the fictional fixture or is public. In the release copy the Anthropic request ids are replaced with req_removed.

Test plan

  • The documented command rebuilds report.md and report.json byte for byte from runs/
  • python3 -m unittest discover bench/tests passes
  • releases/latest still resolves to v0.3.0 after the data release

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@OmarAlex24
OmarAlex24 merged commit acc8410 into main Nov 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant