Add a version for the golden set

[ritsukai]

This adds a version for the golden set, derived from the golden files themselves and recorded with every scoring run. It is the version marker I offered in #50.

Why

The golden set changes, and nothing records which version a score was measured against.

On 6 August, PR #49 took the golden set from 137 comments to 173. That looks like a straightforward improvement, and I think it is one.

But it means a score published on 5 August and a score published on 7 August were measured against different data. Nothing in either result says so. Both are just "the benchmark".

So if you compare two numbers, or check a number quoted somewhere else against the current dashboard, you have no way to tell whether you are looking at the same measuring stick.

This gets worse over time rather than better, because the golden set should keep improving. As things stand, every improvement to it quietly invalidates comparisons with earlier numbers, and leaves no trace that it did.

How it works

The version is a SHA-256 over golden_comments/*.json, taken in sorted filename order, with each file's JSON re-serialized canonically before hashing. It is computed at scoring time and written into benchmark_dashboard.json under a golden_set key. Both analysis entry points print it when they run.

Because it is derived rather than declared, it cannot drift out of sync with the data:

• editing a comment, category, severity, PR title or URL changes it
• adding or removing a golden comment changes it
• reformatting or re-indenting a file does not

It also carries the counts, so 5 files, 50 PRs, 173 comments is readable without decoding the hash.

To print it on its own:

python -m analysis.golden_version

What this is not

There is no numbering scheme, no semver, no tag convention, and no policy about when to bump anything. The set identifies itself, so there is nothing to remember to do.

Deciding what a version ought to cover is a separate and larger question. Golden set only? Golden set plus judge model plus tool-run set? That is a design call, it is yours to make, and I did not want to make it inside a fix for the smaller problem.

I also left the HTML dashboard alone. Where a version belongs on screen is a design question rather than a correctness one.

What it does not fix

Scores published before this lands still do not name their golden set, and nothing can establish that now. This makes the identity visible going forward. It does not recover it backwards.

No published number changes

I ran the dashboard generator and score_profiles across all three judge directories and all three profiles, before and after.

The only difference in benchmark_dashboard.json is the added golden_set key. Every other value is identical. The only difference in CLI output is one line naming the version.

Tests cover the three properties above:

• the version is stable across runs
• reformatting a file does not change it
• editing content does change it