Refuse to boot into a dashboard that cannot draw a price - #31
Merged
Conversation
The failure this closes is silence, not an error. ADR-0007 moved bars to marketdata and step 4 repointed cotmetrics at it while the bars were still in cotdata's store. Nothing raised. The app booted, bound its port, rendered every positioning page off the other store, and drew blank price charts, because a futures read against a store with no bars/futures/ returns an empty frame. It looked like a UI regression and went unnoticed for days. check_price_store() runs before the indexer is built, which is the point: the indexer is the expensive thing whose caches we do not want rebuilt against a store that cannot serve them. It reads the manifest only, so it costs one small JSON read and opens no parquet. It runs in the parent only, so under --debug it refuses before a serving child is spawned. The policy has two halves and they are deliberately different. Nothing at all is a deployment error: the store was never filled or MARKETDATA_STORE points somewhere wrong, no chart on the site can work, and booting anyway is what hid the problem last time, so it refuses. A subset missing, short or stale is a data gap: most of the site still works and a human has to decide whether it matters, so it warns and carries on rather than taking the positioning half down with it. COT_ANALYZER_ALLOW_MISSING_PRICES=1 downgrades the refusal for a deliberately COT-only deployment. "Every series" is counted against stored tiers rather than symbols, because futures store two per symbol and a store holding only backadj is half-filled rather than empty. The policy is split into a pure price_store_verdict() so it can be tested without a store. Getting the line wrong costs in both directions: refuse too eagerly and a data gap takes down a site that was working, refuse too rarely and you are back to blank charts nobody notices. Two corrections to server-side/README.md while here, both about the same confusion. It claimed the second store "fails the same way the first one does, by name, at the point of read". Measured, it does not: unset raises, but set-and- empty serves an empty frame silently. That paragraph was telling the next person to expect an error that never comes, on exactly the shape a server lands in when a sync carries only the equities half. Its troubleshooting section now covers the refusal and the warning as separate entries. Also fixed a boot log line still saying prices come from the cotdata store. Universe read from params.yaml directly rather than through CotIndexer, because this has to run before that singleton exists, and intersected with marketdata's futures registry so the ETF-proxy symbols (MME, MFS) are not reported as gaps they are not. Skips with a warning on a marketdata checkout predating coverage_gaps: the siblings are editable installs, so the version on disk is whatever HEAD that checkout sits at, and an unbootable app is worse than a missing check. Verified against the live stores: healthy store logs "price store OK: 45 instruments" and boots; an empty store logs every gap and exits 1; the escape hatch downgrades it and continues. ruff clean, 138 tests pass (131 plus 7 new). Depends on marketdata#11, which adds coverage_gaps. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on marketdata#11, which adds
coverage_gaps. Merge that one first.The failure this closes is silence, not an error
ADR-0007 moved bars to
marketdataand step 4 repointedcotmetricsat it while the barswere still in cotdata's store. Nothing raised. The app booted, bound its port, rendered
every positioning page off the other store, and drew blank price charts, because a futures
read against a store with no
bars/futures/returns an empty frame. It looked like a UIregression and went unnoticed for days.
Where the check runs, and why there
check_price_store()runs before the indexer is built, which is the point: the indexeris the expensive thing whose caches we do not want rebuilt against a store that cannot
serve them. It reads the manifest only, so it costs one small JSON read and opens no
parquet. It runs in the parent only, so under
--debugit refuses before a serving childis spawned.
The policy, and why the two halves differ
MARKETDATA_STOREwrong. No chart on the site can work, and booting anyway is what hid this last timeCOT_ANALYZER_ALLOW_MISSING_PRICES=1downgrades the refusal for a deliberately COT-onlydeployment.
"Every series" is counted against stored tiers rather than symbols, because futures
store two per symbol and a store holding only
backadjis half-filled rather than empty.The policy is split into a pure
price_store_verdict()so it can be tested without astore. Getting that line wrong costs in both directions, which is what the new tests pin:
refuse too eagerly and a holiday weekend takes down a working site, refuse too rarely and
you are back to blank charts nobody notices.
Two corrections to
server-side/README.md, both about the same confusionIt claimed the second store "fails the same way the first one does, by name, at the point
of read". Measured, it does not. Unset raises; set-and-empty serves an empty frame
silently. That paragraph was telling the next person to expect an error that never comes,
on exactly the shape a server lands in when a sync carries only the equities half. The
troubleshooting section now has separate entries for the refusal and the warning.
Also fixed a boot log line still saying prices come from the cotdata store.
Details worth a reviewer's eye
params.yamldirectly rather than throughCotIndexer, because thishas to run before that singleton exists.
not reported as gaps they are not. 45 of the 47 configured instruments are checked.
coverage_gaps. The siblings areeditable installs, so the version on disk is whatever HEAD that checkout sits at, and an
unbootable app is worse than a missing check.
Verification
Against the live stores:
Full boot with the guard active reaches
Dash is runningafter the OK line.ruffclean,138 tests pass (131 plus 7 new).
🤖 Generated with Claude Code