feat: dated pricing snapshots and pricing_version provenance (#98)

docs/extending.md

@@ -169,7 +169,9 @@ supply-chain incident, a required new symbol, a moved import path).
 
 ### 1.6 Register a model
 
-Add an entry to `MODELS` in `src/maestro/experiment_config.py`:
+Add an entry to the active pricing snapshot in
+`src/maestro/pricing/snapshot_YYYY_MM.py` (the one `DEFAULT_VERSION` in
+`src/maestro/pricing/__init__.py` points at):
 
 ```python
 ModelPricing(
@@ -180,8 +182,20 @@ ModelPricing(
 ```
 
 Pricing is USD per 1M tokens, sourced from the provider's public pricing
-page on the date of the run. `cost_usd` is computed at write time from
-this rate, so a later pricing change does not alter historical rows.
+page on the date the snapshot represents. `cost_usd` is computed at write
+time from this rate, so a later repricing does not alter historical rows.
+
+**Repricing an existing model.** Do not edit the existing snapshot. Copy
+it to a new dated file (`snapshot_2026_10.py`, `snapshot_2027_01.py`, ...),
+update its top-level `VERSION` to the new `YYYY-MM` string and its
+`PRICING` list to the new rates, register it in `_VERSIONS`, and bump
+`DEFAULT_VERSION` to point at the new file. Historical runs stay pinned
+to whichever snapshot was default when they ran; every new run records
+its snapshot id in `run_environments.pricing_version` when pricing
+capture succeeds, and a soft-failed capture leaves the column NULL
+rather than aborting the run. Cross-snapshot analysis is therefore an
+explicit research choice (either the column matches a known snapshot or
+is NULL), never a silent side effect.
 
 ### 1.7 Smoke test
 
@@ -365,9 +379,12 @@ pre-change and post-change runs are never mixed. In particular:
 - A new provider or strategy is additive and does not require a bump on
   its own; the new rows are recognisably from the new configuration.
 
-A pricing change on an existing model is not a bump: `cost_usd` is
-computed at write time and stored, so historical rows retain their
-original cost even after a repricing.
+A pricing change on an existing model is not a scoring bump: `cost_usd`
+is computed at write time and stored, so historical rows retain their
+original cost. It is, however, a pricing-snapshot bump: land the new
+rates in a fresh `snapshot_YYYY_MM.py`, bump `DEFAULT_VERSION`, and every
+subsequent run's `run_environments.pricing_version` records the new
+snapshot id so a cross-snapshot analysis is an explicit choice.
 
 See `CHANGELOG.md` and `.github/CONTRIBUTING.md` for the release-line
 conventions.

docs/schema.md

@@ -65,6 +65,7 @@ result traces back to the exact software stack that produced it.
 | `git_dirty` | INTEGER | Tri-state: 1 = dirty, 0 = clean, NULL = probe failed. Never conflate NULL with clean. |
 | `lib_versions` | TEXT | JSON blob: `{"anthropic": "0.34.2", "openai": "...", ...}`. Whitelisted runtime deps only (see `db/environment.py`). |
 | `docker_image_digest` | TEXT | Value of `MAESTRO_IMAGE_DIGEST` at build time. NULL if the build did not pass it. |
+| `pricing_version` | TEXT | Active pricing snapshot id (`YYYY-MM`, see `maestro.pricing.DEFAULT_VERSION`) at run time. NULL on pre-migration rows and if pricing capture failed. |
 | `captured_at` | TEXT NOT NULL | UTC ISO 8601. |
 
 ### 2.2 `run_configs`
@@ -96,7 +97,7 @@ guarantee.
 | `prompt_tokens` | INTEGER NOT NULL | Prompt/input token count. 0 for controls and for cells that failed before any call. |
 | `completion_tokens` | INTEGER NOT NULL | Completion/output token count. |
 | `duration_ms` | INTEGER NOT NULL | Wall-clock latency of the cell. |
-| `cost_usd` | REAL NOT NULL | Computed at write time from token counts and the `ModelPricing` rate captured in `experiment_config.py`. Never recomputed at read time, so a later pricing change does not alter historical rows. |
+| `cost_usd` | REAL NOT NULL | Computed at write time from token counts and the `ModelPricing` rate from the active snapshot in `maestro.pricing`. Never recomputed at read time, so a later repricing does not alter historical rows; join to `run_environments.pricing_version` via `run_configs.environment_id` to recover which snapshot produced a row; the join yields a snapshot id when both `environment_id` and `pricing_version` are present, and NULL if capture failed. |
 | `error` | TEXT | Human-readable error string. NULL means success (the sole flag for `is_success`). |
 | `retry_count` | INTEGER NOT NULL DEFAULT 0 | Number of retries the provider's retry policy consumed for this cell. |
 

src/maestro/db/client.py

@@ -24,6 +24,10 @@
     git_dirty           INTEGER,
     lib_versions        TEXT,
     docker_image_digest TEXT,
+    -- Pricing snapshot the cost columns were computed against (YYYY-MM
+    -- form; see maestro.pricing). Nullable: pre-migration rows keep NULL,
+    -- and the environment probe records NULL if pricing capture fails.
+    pricing_version     TEXT,
     captured_at         TEXT NOT NULL
 );
 
@@ -147,6 +151,7 @@ def init_db(db_path: Path) -> None:
         _migrate_add_environment_id_column(conn)
         _migrate_add_retry_count_column(conn)
         _migrate_add_container_attachment_columns(conn)
+        _migrate_add_pricing_version_column(conn)
         conn.commit()
 
 
@@ -180,6 +185,18 @@ def _migrate_add_retry_count_column(conn: sqlite3.Connection) -> None:
         )
 
 
+def _migrate_add_pricing_version_column(conn: sqlite3.Connection) -> None:
+    """
+    Add ``run_environments.pricing_version`` to databases that predate the
+    column. Nullable TEXT: pre-migration rows stay NULL (they were recorded
+    before pricing was versioned, so the value is genuinely unknown and
+    fabricating a snapshot label would misrepresent the archive).
+    """
+    cols = {row[1] for row in conn.execute("PRAGMA table_info(run_environments)")}
+    if "pricing_version" not in cols:
+        conn.execute("ALTER TABLE run_environments ADD COLUMN pricing_version TEXT")
+
+
 def _migrate_add_container_attachment_columns(conn: sqlite3.Connection) -> None:
     """
     Add the container + attachment metric columns to databases that predate

src/maestro/db/environment.py

@@ -129,6 +129,21 @@ def _lib_versions() -> dict[str, str | None]:
     return resolved
 
 
+def _pricing_version() -> str | None:
+    """Best-effort probe: environment capture must never crash the run it
+    describes, so a broken or missing pricing package returns ``None`` rather
+    than propagating."""
+    try:
+        from maestro.pricing import DEFAULT_VERSION
+
+        return DEFAULT_VERSION
+    except Exception:
+        # Env capture must never crash the run it describes; a broken pricing
+        # package would be caught at import time by the runner long before this.
+        # Recording None preserves the failure signal instead of guessing.
+        return None
+
+
 def capture_environment(
     image_digest_env: str = "MAESTRO_IMAGE_DIGEST",
 ) -> RunEnvironment:
@@ -150,5 +165,6 @@ def capture_environment(
         git_dirty=_git_dirty(),
         lib_versions=json.dumps(_lib_versions(), sort_keys=True),
         docker_image_digest=os.environ.get(image_digest_env) or None,
+        pricing_version=_pricing_version(),
         captured_at=datetime.now(timezone.utc),
     )

src/maestro/db/queries.py

@@ -23,9 +23,9 @@ def insert_run_environment(conn: sqlite3.Connection, env: RunEnvironment) -> Non
         INSERT INTO run_environments
             (environment_id, os, arch, python, hostname,
              git_commit, git_dirty, lib_versions, docker_image_digest,
-             captured_at)
+             pricing_version, captured_at)
         VALUES
-            (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+            (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
         """,
         (
             str(env.environment_id),
@@ -38,6 +38,7 @@ def insert_run_environment(conn: sqlite3.Connection, env: RunEnvironment) -> Non
             None if env.git_dirty is None else int(env.git_dirty),
             env.lib_versions,
             env.docker_image_digest,
+            env.pricing_version,
             env.captured_at.isoformat(),
         ),
     )

src/maestro/experiment_config.py

@@ -4,7 +4,8 @@
 Single source of truth for the experiment matrix.
 
 To add a new input:   append to INPUTS
-To add a new model:   append to MODELS
+To add a new model:   add a row to the active pricing snapshot in
+                      ``maestro.pricing`` (never hardcode a rate here)
 To enable a strategy: add to STRATEGIES (once implemented)
 """
 
@@ -13,7 +14,8 @@
 import os
 from pathlib import Path
 
-from maestro.schemas import InputFile, ModelPricing, Strategy, Tier
+from maestro.pricing import DEFAULT_VERSION, load_pricing
+from maestro.schemas import InputFile, Strategy, Tier
 
 # ---------------------------------------------------------------------------
 # Base path for all data files (relative to project root)
@@ -280,98 +282,22 @@
 # Model registry - pricing per model for cost calculation
 # ---------------------------------------------------------------------------
 
-# Synthetic "model" used only for control-strategy rows. Controls bypass the
-# LLM entirely so no real model is involved; this entry exists so the
-# ``RunConfig.model`` column has an honest value ("control") rather than
-# borrowing the name of a real model and lying about what produced the row.
-# Zero pricing means control rows never affect cost rollups.
-CONTROL_MODEL = ModelPricing(
-    model="control",
-    input_price_per_1m=0.0,
-    output_price_per_1m=0.0,
-)
-
-# Two models per provider: a frontier ("best") model and an efficiency model,
-# so the experiment can compare quality against cost within and across vendors.
-# Prices are USD per 1M tokens, verified against each provider's pricing page in
-# April 2026 for the frozen main run. IDs are pinned to dated snapshots where
-# the provider offers one, so the run stays reproducible.
-#
-# The model id on each row is the canonical internal_id from
-# maestro.models.MODEL_REGISTRY (the single source of truth for model
-# naming). A consistency test asserts every non-control entry here is
-# registered, so a typo cannot land pricing under a name no other layer
-# recognises.
+# MODELS is derived from the active pricing snapshot in ``maestro.pricing``
+# so rates live in one place: a dated ``snapshot_YYYY_MM.py`` file whose
+# version string is recorded on every run (run_environments.pricing_version).
+# Every model id here is a canonical internal_id from
+# maestro.models.MODEL_REGISTRY; the model-registry consistency test asserts
+# the pricing/registry one-to-one invariant, and a typo would fail loudly at
+# import via _validate() in the pricing package.
 #
 # Note: provider dispatch (run.py) is by substring (claude / gpt / mistral /
 # gemini / deepseek), so any new model id must contain its provider's needle.
 # tests/providers/test_provider_dispatch.py enforces this for every entry here.
-MODELS: list[ModelPricing] = [
-    # Anthropic
-    ModelPricing(
-        model="claude-opus-4-8",  # frontier
-        input_price_per_1m=5.00,
-        output_price_per_1m=25.00,
-        # Opus 4.7+ removed sampling params; sending temperature returns 400.
-        supports_temperature=False,
-    ),
-    ModelPricing(
-        model="claude-haiku-4-5-20251001",  # efficiency
-        input_price_per_1m=1.00,
-        output_price_per_1m=5.00,
-    ),
-    # OpenAI (GPT-5 family: max_completion_tokens, no custom temperature)
-    ModelPricing(
-        model="gpt-5.5-2026-04-23",  # frontier
-        input_price_per_1m=5.00,
-        output_price_per_1m=30.00,
-        supports_temperature=False,
-    ),
-    ModelPricing(
-        model="gpt-5.4-mini-2026-03-17",  # efficiency
-        input_price_per_1m=0.75,
-        output_price_per_1m=4.50,
-        supports_temperature=False,
-    ),
-    # Mistral
-    ModelPricing(
-        model="mistral-medium-3-5",  # frontier
-        input_price_per_1m=1.50,
-        output_price_per_1m=7.50,
-    ),
-    ModelPricing(
-        model="mistral-small-2603",  # efficiency
-        input_price_per_1m=0.15,
-        output_price_per_1m=0.60,
-    ),
-    # Gemini
-    ModelPricing(
-        model="gemini-3.5-flash",  # frontier
-        input_price_per_1m=1.50,
-        output_price_per_1m=9.00,
-    ),
-    ModelPricing(
-        model="gemini-3.1-flash-lite",  # efficiency
-        input_price_per_1m=0.25,
-        output_price_per_1m=1.50,
-    ),
-    # DeepSeek: the cross-provider replication dimension's emerging-Chinese
-    # entry (proposal section 3.2), consumed via the OpenAI-compatible endpoint
-    # (see providers/deepseek.py). Pricing is the cache-MISS (standard) rate;
-    # DeepSeek also offers a cheaper cache-hit input price, but ModelPricing has
-    # a single input rate, so cache-miss makes the tracked cost an upper bound
-    # on actual spend (never an under-count).
-    ModelPricing(
-        model="deepseek-v4-pro",  # frontier
-        input_price_per_1m=0.435,
-        output_price_per_1m=0.87,
-    ),
-    ModelPricing(
-        model="deepseek-v4-flash",  # efficiency
-        input_price_per_1m=0.14,
-        output_price_per_1m=0.28,
-    ),
-]
+PRICING_VERSION, MODELS = load_pricing()
+
+# Re-export the pricing version constant for consumers (run.py's environment
+# capture, tests) that want to name it without importing the pricing package.
+DEFAULT_PRICING_VERSION = DEFAULT_VERSION
 
 
 # ---------------------------------------------------------------------------
@@ -394,7 +320,8 @@
 
 
 # Set used by ``build_matrix`` and analysis code to special-case controls:
-# - matrix builder uses CONTROL_MODEL and run_number=1 for these strategies
+# - matrix builder uses ``maestro.pricing.CONTROL_MODEL`` and run_number=1
+#   for these strategies
 # - analysis can exclude them from ANOVA / cost rollups with
 #   ``WHERE strategy NOT IN (SELECT value FROM control_strategies)`` or the
 #   in-Python equivalent ``s not in CONTROL_STRATEGIES``.