Add automatic versioned repricing of persisted usage

[starkdmi]

Compatibility with current main

Merged current main into this branch (account-plan CLI, quota-cycle sync, Grok 4.6 pricing, repository-rename rollup buckets). Conflict resolutions:

• Keep PRICING_RULESET_VERSION = 1 and the current catalog string official:2026-08-19. Existing stores have no applied ruleset yet, so the first pass includes Grok 4.6 rates.
• Preserve the QuotaQuery import next to PRICING_RULESET_VERSION.

Code review follow-ups

• Unreadable event, summary, and span JSON is skipped instead of failing every operational command. Pages always advance past those ids; the applied ruleset still advances. Dirty rollup refresh also skips unreadable sibling events in the same day bucket. Dangling task_span_event_links clear stale span cost rather than leaving it.
• Repricing still runs in one BEGIN IMMEDIATE transaction (atomic rollback + concurrent no-op). It is no longer started from diagnostic commands. Scan, report, import, export, task, sync, daemon, and snapshot reprice; status, doctor, quota, conversation, account, source, subscription, and privacy open with Store::open.
• Import overlays the compiled catalog on each reported summary before upsert, so a store already at this ruleset cannot keep an estimated-only figure from an older catalog. Provider-reported amounts stay.
• Menubar continues to use Store::open and does not reprice. The selected statsai binary owns the ruleset so a store-linked menubar cannot apply a different compiled catalog.
• Added tests: corrupt usage rows, dangling span links, open_operational_store happy path, command-routing coverage, and import overlay after already_current.

Quota windows can still print estimated_cost_micro_usd. That figure stays stale until a price-derived command runs. That is the intended trade-off.

Problem

Pricing is calculated while provider records are parsed. After a pricing-rule change, persisted UsageEvent rows keep old or missing estimated cost until an expensive raw rescan (statsai scan --provider … --no-cache) plus rollup rebuild. That defeats the reusable cloned-database workflow: --full sync alone does not reprice anything.

Solution

The selected statsai binary now owns a monotonic PRICING_RULESET_VERSION and, before price-derived commands use that data, applies that ruleset to the store.

• New numeric PRICING_RULESET_VERSION is separate from the descriptive PRICING_CATALOG_VERSION. Catalog strings are not ordered lexicographically. Every semantic pricing-rule change must increment the numeric version.
• Applied state lives in existing local_metadata (pricing.applied_ruleset_version, pricing.applied_catalog_version). Missing metadata is treated as a legacy store that needs one initial pass. No schema migration.
• Repricing is not run from Store::open. statsai-dev links statsai-store and may compile a different catalog than the exact-SHA binary it launches. store clone-to and supported-version probes stay reprice-free.
• Events are streamed in pages and repriced from persisted provider/model/usage/timestamp via estimate_cost_at. Only pricing-derived fields are written. Identity, usage, timestamps, source metadata, and provider-reported cost/provenance are preserved. Unknown models stay unpriced while the store-level applied version still advances.
• Summaries that span a known pricing boundary (pricing_changes_between) remain unknown rather than applying one price across the period.
• daily_rollups is unused by report/sync/snapshot and is left unchanged (covered by test). Linked task spans are recomputed from current persisted events even when those events already match the catalog, then work items for those project buckets are rebuilt and marked dirty.
• Changed sync_rollups are refreshed and marked dirty. Repriced passthrough summaries change payload hash. Ordinary statsai sync --sink http uploads both without --rebuild-rollups or --full.

Transaction boundary

ensure_pricing_ruleset uses with_immediate_transaction (busy-retry, BEGIN IMMEDIATE):

1. Recheck the applied version after acquiring the lock.
2. Stream and update events, summaries, rollups, and linked task spans.
3. Write applied metadata only after every payload and derived-data update succeeds.
4. Any error rolls back payloads, rollups, and metadata together.
5. Concurrent callers serialize on the immediate lock: one worker applies the ruleset, the other observes the new version and no-ops.

Only rows whose serialized pricing state changed are written. Event updates reuse the in-memory row (no extra read). Task-span refresh pages only spans with task_span_event_links and batch-loads those events.

Downgrade behavior

If the store was processed by a newer ruleset than the running binary supports, normal operation is refused with an explicit forward-pricing-version error (analogous to the schema-17 forward guard). The old binary does not reprice backward or insert events priced with old rules into a newer-priced store. Invalid applied-version metadata also fails closed.

Production-data safety

statsai-dev --prod-data remains fail-closed:

• statsai store supported-pricing-ruleset-version prints the compiled ruleset without opening a store.
• build.json schema is 2 and includes pricing_ruleset_version.
• Production applied pricing is read with database_applied_pricing_ruleset_version (no migrate, no reprice).
• --prod-data is allowed only when schema and pricing ruleset exactly match. Missing, older, or newer production pricing is refused.
• Isolated statsai-dev stores are still auto-repriced by the selected exact-SHA binary.

Large-fixture measurement

On-disk fixture of 512 legacy codex-auto-review events, streamed in pages of 256:

representative fixture: examined=512 changed=512 elapsed=95.807024ms

The test asserts the pass stays well under 30s. The whole store is not loaded into memory at once.

Tests

Store, CLI, pricing, artifact, and --prod-data tests cover legacy auto-review reprice, the 2026-07-30 date boundary, metadata-after-success, no-op, provider-reported survival, unknown models, in-window and boundary summaries, dirty rollups, incremental HTTP sync of rollups and passthrough summaries without --full, stale spans when events already match, unlinked spans left unchanged, dangling links clearing cost, unreadable payloads skipped, dirty task buckets, injected rollback, concurrent callers, forward-version refusal (store and open_operational_store), open_operational_store happy path, diagnostic vs price-derived command routing, import overlay after already_current, invalid metadata fail-closed, --prod-data mismatches, and schema-2 manifest verification.

Operator notes

• Repricing is automatic and version-driven.
• No raw rescan is needed after pricing updates.
• Normal incremental sync publishes corrected dirty rollups and changed passthrough summaries.
• Pricing-ruleset downgrades are refused.
• --prod-data requires exact schema and pricing-version compatibility.
• Diagnostic commands do not reprice; run a price-derived command (report, scan, sync, …) to apply a newer ruleset.
• statsai import overlays the current catalog on estimated-only imported summaries even when the store is already at this ruleset.

  

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba094cd43f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".