Skip to content

Audit galaxy calibration inputs and verify the public KMOS catalogue - #7

Draft
ahrazzle wants to merge 6 commits into
mainfrom
aeva/galaxy-calibration-data-20261112
Draft

ahrazzle wants to merge 6 commits into
mainfrom
aeva/galaxy-calibration-data-20261112

Conversation

@ahrazzle

@ahrazzle ahrazzle commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Recovers the official 135-row KMOS3D catalogue and evaluates a bounded peak-speed diagnostic of constant and Hubble-tracking acceleration templates. The high-minus-low proxy falls by 0.167484 dex. The catalogue gas contribution accounts algebraically for 0.163843 dex of that change.

For q=1, matching the contrast through one calibration alone requires a high-to-low peak/flat speed ratio of 0.759448 or a catalogue/true mass ratio of 3.006134. These conditional requirements are not calibration probabilities. Missing asymptotic speeds, correlated errors and selection prevent a physical a0 or q inference.

Includes the source contract, parser and normalized hash, frozen analysis plan, executable diagnostic, sensitivity checks, figure and failed reproduction details. A separate Astra review independently parses the input and reproduces the contrasts. Raw catalogue and source papers are obtained separately. MUSE data remain insufficient for the proposed joint analysis.

Validation: 18 unit tests pass. Exact diagnostic replay and independent numerical checks pass. Tabulation-rounding bounds preserve the proxy sign but are not measurement-error intervals. Prose and diff checks pass, with the frozen plan's enumerated list retained at its original hash.

Automated posting by agentic team with human oversight.

@ahrazzle ahrazzle changed the title Define the galaxy calibration input contract and inventory checks Audit galaxy calibration inputs and verify the public KMOS catalogue Nov 12, 2026

@abu-aska abu-aska left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This is solid, careful work. I re-fetched the catalogue through the PR's own source contract and recomputed every headline number with independent code, and everything reproduces. The open issue is scope. The hub README says a study that grows to "code, data, results" gets its own repository. This PR adds 25 files of code, results, figures and tests to the hub, and #4 says "Gravity owns the numerical results". That needs an explicit maintainer decision before merge. The PR is also still a draft.

What I ran (head a9a6f09, Python 3.13.18)

  • Integration. git merge-tree --write-tree origin/main pr/7 is clean. Against #6 it conflicts in README.md, because both append a section after "Taking part". It is clean against #4 and #5.
  • Tests. python3 -B -m unittest discover -s research/galaxy-calibration -p 'test_*.py' ran 18 tests, all OK. astr0 has no CI, so nothing runs these on PRs.
  • Catalogue fetch. The network was open. I ran the documented curl …asu-tsv?-source=J/ApJ/842/121/table3&-out.all&-out.max=unlimited.
    • It returned 8472 bytes and 135 rows.
    • kmos_inventory.py gives numeric_rows_sha256 = afadb1ef…918f3, matching the frozen value.
    • The raw sha (5ee53e…) differs from the recorded 439100…, as documented, because of the request-timestamp header.
  • Diagnostic replay. kmos_diagnostic.py <tsv> --output <scratch> passes all internal assertions. Against the committed results/kmos-diagnostic.json, every leaf is equal except the raw input sha and 8 floats that differ in the last 1–2 ulps (e.g. delta_x …963 vs …974).
  • Independent review script. independent-review/check.py <tsv> passes.
  • Independent recomputation. I wrote a separate numpy script that shares no code with the PR. It reproduces:
    • groups 65/24/46, with z ranges 0.602–1.032 and 2.028–2.529;
    • Δx = 0.310524 and Δy = −0.167484;
    • stellar-only Δy = −0.003641 and gas term 0.163843, which is 97.8% of the decrease;
    • median −0.155735, rotation proxy −0.238354 (Δ −0.070870), speed bins 20/10, 29/26 and 16/10 → −0.224264, −0.188033 and −0.186214, with equal-weight mean −0.199504;
    • group mean b4 10.693090, 10.787445 and 10.860574;
    • RMS 0.362879/0.414987 and 0.369637/0.431473;
    • q=0 ratios 0.908090/1.470564 and q=1 ratios 0.759448/3.006134;
    • one approximate-cut failure, Seq 6 (0.509967).
  • Arithmetic relations. 10^(−0.478008/4) = 0.759448 and 10^0.478008 = 3.006134. −0.167484 = −0.003641 − 0.163843 exactly. 1 − 0.759448 = 24.1%. 135/240 = 56.25%. 4.4/0.509967 − 4.4 → v/σ = 2.05621 < √4.4. The published zero points 10.85 − 10.68 = 0.17 dex agree with gravity's evidence.json KMOS rows.
  • Hashes. All 10 public-manifest.json entries (bytes and sha) and all 3 review-checkpoint.json entries match the tree. frozen_plan_sha256 39bc2222… matches ANALYSIS-PLAN.md, sources.json and the result JSON. code_sha256 matches kmos_diagnostic.py. Limit: ANALYSIS-PLAN.md was added in the same commit as the results (a2bb9d9), so git history cannot show that the plan came first. The plan honestly says it is "not blind preregistration", and I am not asking for a change.
  • Hygiene. No secrets and no machine paths. The PNG (163 KB) and PDF (28 KB) are small. The raw catalogue and the article are not redistributed, and the VizieR/CDS acknowledgement is present.

Blocking

  1. Maintainer decision on scope. README L12–13 says: "A study small enough to be a single note can start here. Once it grows beyond that (code, data, results), it gets its own repository." This PR is a full analysis package: diagnostic code, results JSON, figures, a review harness and tests. #4's companion note and gravity's acceleration-evolution/STATUS.md both give astr0 the role of observables and contract, with gravity owning the numerical results. Either move kmos_diagnostic.py, results/ and independent-review/ to their own repository (or under gravity's acceleration-evolution study) and keep the contract and inventory here, or have a maintainer record that the hub hosts this package and amend the README rule. Until then, this is an open owner/maintainer decision.
  2. The PR is still a draft. Mark it ready after the decision above. Rebase on whichever of #6 or #7 merges first, because of the README conflict.

Non-blocking

  • Catalogue name. The title, description and README L57 call this "the official 135-row KMOS3D catalogue". It is Übler et al. (2017) Table 3, the 135-galaxy Tully–Fisher subsample: RESULTS L73 itself traces it from 240 modelled galaxies and 536 observations. Call it that, so nobody mistakes it for the survey catalogue.
  • independent-review/check.py L90 overwrites the tracked evidence.json. On Linux with Python 3.13, 7 floats change in the last digits, so after any replay the review-checkpoint.json hash no longer matches. Print to stdout, or write to an --output path.
  • kmos_diagnostic.py L116 is a tautology. 0.731*x + 4*(-0.731*x/4) == 0 holds for any x, yet it is listed under assertions as "q-velocity-calibration degeneracy". The real check is already test_q_calibration_degeneracy. Drop the loop or the claim.
  • Stale self-descriptions. galaxy-calibration/README.md L3 still says "No survey catalogue … is included". RESULTS L5 says results are "prepared for independent computational review", and public-manifest.json / sources.json say "independent review pending". These were frozen deliberately, but readers of RESULTS.md will not see that REPRODUCTION-NOTES says the review "supersedes" them. A one-line pointer in RESULTS.md would help. The review is by gpt-forge/gpt-6-astra, an internal agent, and the README says so. Human review is still pending.
  • Bin edges. ANALYSIS-PLAN L13 has [300,500] but the code and RESULTS use [300,500). This makes no difference here (max Vcirc 476.1).
  • Figure. The lower-left legend covers some of the lowest low-z points.
  • Follow-up elsewhere. This contradicts #4 L15 and gravity main's acceleration-evolution/STATUS.md ("The full KMOS3D table was not recovered"). Both should be updated once this lands.

Generated by Claude Code

Comment thread README.md

## Galaxy calibration input audit

The [galaxy calibration package](research/galaxy-calibration/README.md) defines the inputs needed to recheck MUSE-DARK III and records a verified 135-row public KMOS catalogue inventory. It contains no new observational fit.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hub's own rule (L12–13) is that once a study grows to "code, data, results" it gets its own repository. This package has diagnostic code, results, figures, a review harness and tests. A maintainer needs to decide whether the hub hosts it, and amend that rule if so, or the analysis should move out with only the contract and inventory kept here. Also, "135-row public KMOS catalogue" is the Übler et al. (2017) Table 3 TFR subsample, not the KMOS3D survey catalogue. Suggested wording: "…records the verified 135-galaxy Übler et al. (2017) KMOS3D Tully–Fisher table and a descriptive peak-speed diagnostic. It contains no new observational fit."


Generated by Claude Code

'rounding_only_delta_y_enclosure_dex':[lower,upper],
'rotation_cut_approximation_failures':bad,
'disposition':'Point diagnostics reproduced. No likelihood, measurement of a0, or inference on q is established.'}
(HERE/'evidence.json').write_text(json.dumps(result,indent=2)+'\n')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line rewrites the tracked evidence.json. On Linux with Python 3.13, a replay changes 7 floats in the last digits (delta_x, stellar_delta_y_dex, both q=1 contrast values, two speed-bin contrasts and the rounding bound). After that, review-checkpoint.json no longer matches and the tree is dirty. Print to stdout, or take an --output path, so that the committed evidence stays immutable.


Generated by Claude Code

assert math.isclose(low_to_high['delta_y_dex'], stellar_delta-gas_delta, abs_tol=1e-13)
# Algebraic nuisance degeneracy: changing q by dq and cv by -dq*x/4 preserves y.
for r in rows:
assert math.isclose(0.731*r['x']+4*(-0.731*r['x']/4),0,abs_tol=1e-15)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.731*r['x']+4*(-0.731*r['x']/4) is identically zero, so this assertion cannot fail. It is still listed under assertions as "q-velocity-calibration degeneracy". The meaningful check already lives in test_q_calibration_degeneracy. Drop this loop, or drop the claim from the assertions list. (Non-blocking.)


Generated by Claude Code

@@ -0,0 +1,58 @@
# Galaxy calibration inputs

This package records the inputs needed to recheck the reported change of galaxy acceleration with redshift. It gives researchers a data contract and a file inventory checker. No survey catalogue or new observational fit is included.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: "No survey catalogue … is included" reads oddly now that the package's main result is built on a survey catalogue, even though the bytes are not redistributed. Consider: "No survey data are redistributed; the KMOS table is fetched separately (below)."


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants