[4waiz]

Root cause

The browser engine matches Python job regenerates site/data and runs git diff --exit-code -- site/data. All four CI runs on main so far (e.g. run 36266471164) failed there, with the same diff in site/data/results.json: only the last digit of a few floats changes.

-     "transmit_pct": 36.42266666666667,
+     "transmit_pct": 36.422666666666665,

json.dumps writes the shortest round-trip repr, which can be up to 17 significant digits. That last binary digit comes from the toolchain, not from the study. Two separate upstream changes make it drift:

1. CPython 3.12 changed sum() over floats to compensated (Neumaier) summation. Long.profile_mean used the built-in sum(). The committed file was exported on Python 3.11 and the runner uses 3.12, so the Table IV (right) averages came out one ulp apart. These are exactly the 4 lines CI reported. I reproduced it locally: Python 3.13 gives CI's values and 3.11 gives the committed ones.
2. pandas 3.1 made read_csv's float parser correctly rounded (and deprecated float_precision). pandas 2.x and 3.0 are one ulp off on 49,544 of the 795,356 float values in results/ (about 6%). The committed file carries the pandas 2.x parse. requirements.txt is unpinned, so the next runner would resolve pandas 3.1 and fail on about 100 more lines even with (1) fixed.

The research numbers are fine. The drift is about 1e-16 relative.

Fix (experiments/export_web_results.py)

• Round at serialization. write_json now passes the payload through canonical_floats, which rounds every float to JSON_SIGNIFICANT_DIGITS = 10 significant digits. It uses Python's correctly rounded float formatting, which doesn't depend on the platform C library.
  • Ten digits is about 10⁶ times coarser than the noise and finer than anything the dashboard prints (at most 4 decimals).
  • Because the rounding is relative, even the smallest nonzero value (~3e-5) keeps 10 digits.
  • rankings and every comparison in build() still see full-precision values.
  • The precision and the reasons for it are documented next to the constant.
• Fix the known cause at its root. Long.profile_mean now uses math.fsum, which is correctly rounded on every Python version.
• Regenerated site/data/results.json.

The CI check itself is unchanged.

Verification

• The new results.json is exactly the old file with each float rounded to 10 significant digits. The worst relative change is 4.4e-10, and keys, ints, booleans and strings are untouched.
• Output is byte-identical (same sha256) under Python 3.11.9 and 3.13.14 with pandas 2.3.3, 3.0.6 and 3.1.0. The full CI step (export_web_config + export_web_traces + export_web_results + git diff --exit-code -- site/data) passes in each.
• Margin: the closest of the 1,086 exported floats is about 2,400 ulps from a 10-digit rounding boundary. The toolchain drift observed is 1–2 ulps.
• Dashboard unchanged: I ran the site's own site/js/research.js under Node on the old and new file. The headline cards, the honest block and all six tables render to identical HTML.
• Tests: python -m pytest tests_web -q passes (7, including the JS/Python parity test), and so does python -m pytest -q (69).

Deliberately not changed

• The JS engine. It only renders results.json with toFixed; it doesn't emit or compare it.
• config.json and the traces. They were already byte-identical across these toolchains. The traces also have to stay at full precision: test_shipped_traces_are_the_python_scenarios requires exact array equality, and the dashboard's parity readout compares against their reference_metrics.

🤖 Generated with Claude Code