Single source of truth for model names and reported results

[Colinho22]

Motivation

Model designations and reported figures appear in more than one place in the repository, which allows them to drift out of sync. Anything transcribed by hand from run output will eventually disagree with the run output.

Proposal

• One canonical model registry — internal id, provider id, display name — referenced everywhere.
• Generate reported figures programmatically into a machine-readable results file instead of transcribing them.
• Add a consistency check that fails when the same quantity appears with two values.

Done when:

• Model names resolve through the registry throughout
• Reported numbers derive from generated output only
• Consistency check runs in CI

Scope

• In scope: model registry, generated results file, consistency check, CI wiring.
• Out of scope: changing any reported value; restructuring the results schema beyond what the check requires.

Open questions

• How are model versions and snapshot dates handled — is a dated snapshot a distinct registry entry, or an attribute of one entry? Affects whether results across snapshots are comparable by default.
• Should the consistency check block CI or warn only? Blocking is stricter but may be noisy while the registry is being adopted.

Related

• Externalise model pricing into versioned config #98 — pricing config keys off the same registry.