Skip to content

Refuse to boot into a dashboard that cannot draw a price - #31

Merged
mspinola merged 1 commit into
mainfrom
claude/price-store-boot-guard
Aug 9, 2026
Merged

Refuse to boot into a dashboard that cannot draw a price#31
mspinola merged 1 commit into
mainfrom
claude/price-store-boot-guard

Conversation

@mspinola

@mspinola mspinola commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Depends on marketdata#11, which adds
coverage_gaps. Merge that one first.

The failure this closes is silence, not an error

ADR-0007 moved bars to marketdata and step 4 repointed cotmetrics at it while the bars
were still in cotdata's store. Nothing raised. The app booted, bound its port, rendered
every positioning page off the other store, and drew blank price charts, because a futures
read against a store with no bars/futures/ returns an empty frame. It looked like a UI
regression and went unnoticed for days.

Where the check runs, and why there

check_price_store() runs before the indexer is built, which is the point: the indexer
is the expensive thing whose caches we do not want rebuilt against a store that cannot
serve them. It reads the manifest only, so it costs one small JSON read and opens no
parquet. It runs in the parent only, so under --debug it refuses before a serving child
is spawned.

The policy, and why the two halves differ

situation what happens why
no gaps one INFO line
nothing at all log every gap, exit 1 deployment error. Store never filled or MARKETDATA_STORE wrong. No chart on the site can work, and booting anyway is what hid this last time
subset missing / short / stale log the gaps, warn and carry on data gap. Most of the site still works, so taking the positioning half down with it is the worse trade

COT_ANALYZER_ALLOW_MISSING_PRICES=1 downgrades the refusal for a deliberately COT-only
deployment.

"Every series" is counted against stored tiers rather than symbols, because futures
store two per symbol and a store holding only backadj is half-filled rather than empty.

The policy is split into a pure price_store_verdict() so it can be tested without a
store. Getting that line wrong costs in both directions, which is what the new tests pin:
refuse too eagerly and a holiday weekend takes down a working site, refuse too rarely and
you are back to blank charts nobody notices.

Two corrections to server-side/README.md, both about the same confusion

It claimed the second store "fails the same way the first one does, by name, at the point
of read". Measured, it does not. Unset raises; set-and-empty serves an empty frame
silently. That paragraph was telling the next person to expect an error that never comes,
on exactly the shape a server lands in when a sync carries only the equities half. The
troubleshooting section now has separate entries for the refusal and the warning.

Also fixed a boot log line still saying prices come from the cotdata store.

Details worth a reviewer's eye

  • Universe read from params.yaml directly rather than through CotIndexer, because this
    has to run before that singleton exists.
  • Intersected with marketdata's futures registry, so the ETF-proxy symbols (MME, MFS) are
    not reported as gaps they are not. 45 of the 47 configured instruments are checked.
  • Skips with a warning on a marketdata checkout predating coverage_gaps. The siblings are
    editable installs, so the version on disk is whatever HEAD that checkout sits at, and an
    unbootable app is worse than a missing check.

Verification

Against the live stores:

healthy   -> INFO  price store OK: 45 instruments, every stored tier present
                   and no older than 7 days.                        boots, exit 0
empty     -> WARN  price store gap: 6E unadj: absent, ... (+80 more)
             ERROR MARKETDATA_STORE holds no bars for any of the 45 ...   exit 1
hatch set -> WARN  price store has 90 gap(s) across 45 instruments.  continues

Full boot with the guard active reaches Dash is running after the OK line. ruff clean,
138 tests pass (131 plus 7 new).

🤖 Generated with Claude Code

The failure this closes is silence, not an error. ADR-0007 moved bars to
marketdata and step 4 repointed cotmetrics at it while the bars were still in
cotdata's store. Nothing raised. The app booted, bound its port, rendered every
positioning page off the other store, and drew blank price charts, because a
futures read against a store with no bars/futures/ returns an empty frame. It
looked like a UI regression and went unnoticed for days.

check_price_store() runs before the indexer is built, which is the point: the
indexer is the expensive thing whose caches we do not want rebuilt against a
store that cannot serve them. It reads the manifest only, so it costs one small
JSON read and opens no parquet. It runs in the parent only, so under --debug it
refuses before a serving child is spawned.

The policy has two halves and they are deliberately different. Nothing at all is
a deployment error: the store was never filled or MARKETDATA_STORE points
somewhere wrong, no chart on the site can work, and booting anyway is what hid
the problem last time, so it refuses. A subset missing, short or stale is a data
gap: most of the site still works and a human has to decide whether it matters,
so it warns and carries on rather than taking the positioning half down with it.
COT_ANALYZER_ALLOW_MISSING_PRICES=1 downgrades the refusal for a deliberately
COT-only deployment.

"Every series" is counted against stored tiers rather than symbols, because
futures store two per symbol and a store holding only backadj is half-filled
rather than empty.

The policy is split into a pure price_store_verdict() so it can be tested without
a store. Getting the line wrong costs in both directions: refuse too eagerly and
a data gap takes down a site that was working, refuse too rarely and you are back
to blank charts nobody notices.

Two corrections to server-side/README.md while here, both about the same
confusion. It claimed the second store "fails the same way the first one does, by
name, at the point of read". Measured, it does not: unset raises, but set-and-
empty serves an empty frame silently. That paragraph was telling the next person
to expect an error that never comes, on exactly the shape a server lands in when
a sync carries only the equities half. Its troubleshooting section now covers
the refusal and the warning as separate entries. Also fixed a boot log line still
saying prices come from the cotdata store.

Universe read from params.yaml directly rather than through CotIndexer, because
this has to run before that singleton exists, and intersected with marketdata's
futures registry so the ETF-proxy symbols (MME, MFS) are not reported as gaps
they are not. Skips with a warning on a marketdata checkout predating
coverage_gaps: the siblings are editable installs, so the version on disk is
whatever HEAD that checkout sits at, and an unbootable app is worse than a
missing check.

Verified against the live stores: healthy store logs "price store OK: 45
instruments" and boots; an empty store logs every gap and exits 1; the escape
hatch downgrades it and continues. ruff clean, 138 tests pass (131 plus 7 new).

Depends on marketdata#11, which adds coverage_gaps.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mspinola
mspinola merged commit 1e1948e into main Aug 9, 2026
3 checks passed
@mspinola
mspinola deleted the claude/price-store-boot-guard branch August 9, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant