Name the second store, in the launcher and in the docs - #29
Merged
Conversation
…ice read ADR-0007 step 4 landed in cotmetrics, so bars now arrive through marketdata.get_bars and a price read needs a second store. Without it the failure is quiet in the worst way: the import succeeds, the app binds :5001, every COT page renders off the other store, and only the first chart wanting bars raises. That reads as a UI regression rather than a missing environment variable, which is exactly the confusion the COTDATA_STORE check upstairs exists to prevent. Same check, same shape, same reason. The second branch is the collision case. The two stores may share a synced parent but never one directory, because each producer rewrites its own manifest.json read-modify-write and a shared manifest loses entries. Pointing both variables at one path is the easy way to arrive there, so refuse it here rather than let a producer discover it later. Verified all three branches by hand: unset exits 1 naming the variable and the .env path, equal paths exit 1 naming the manifest reason, distinct paths fall through to the app. bash -n clean, ruff clean on src and tests, 131 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s are not The install instructions were a step behind the code and would have led the next clone into the error that prompted this. README named two siblings and two editable installs; there are three. Missing ../marketdata does not fail at install time, it fails at import, because cotmetrics/__init__.py re-exports signals and signals imports marketdata at module level, so the error arrives as a ModuleNotFoundError from a package nothing in this repo mentions. That traceback is now in README verbatim. The same lag ran through the rest. README said prices are read through cotdata.get_prices; cotmetrics/src no longer calls it anywhere. ARCHITECTURE said cotmetrics depends on cotdata alone, and that caches are busted by two counters. There are three, and marketdata is the one that carries the weight: the case the guard was written for, reconstructed volume promoted to default, is a PRICE schema bump, so watching cotdata alone would leave uncovered the only case it exists for. data-path.md needed more than a correction, because the two ADR-0007 steps are out of step with each other and that is worth recording rather than smoothing over. Step 4 (repoint consumers) is done. Step 2 (move the producer and the bars) is on ice. So the read points at $MARKETDATA_STORE/bars/futures/ while 99 price parquets sit in $COTDATA_STORE/prices/ where the producer keeps writing them, and a futures read against a store holding only equities returns an empty frame with no exception. The old diagram is kept and marked superseded, since its vendor-abstraction argument still holds; the measured state goes in a new section beside it with a one-line check a reader can run. Measured 2026-08-08 against the working tree and both live stores, not read off the docs, which is how the staleness surfaced. ruff clean on src and tests, 131 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-on to #26 / #27 / #28. Those made the repo depend on
marketdata; this makes afresh clone and a fresh run say so, and records where the bars actually are.
Prompted by a real failure:
./run-local.shdied withModuleNotFoundError: No module named 'marketdata'on a venv that predated thedependency. The venv was the immediate cause, but the instructions that venv was built
from are still a step behind the code, so the next clone hits the same wall.
The launcher
MARKETDATA_STOREnow gets the same upfront checkCOTDATA_STOREhas, plus a guardagainst pointing both at one directory.
Without it the failure is quiet in the worst way: the import succeeds, the app binds
:5001, every COT page renders off the other store, and only the first chart wanting bars
raises. That reads as a UI regression rather than a missing environment variable.
The collision branch is there because the two stores may share a synced parent but never
one directory: each producer rewrites its own
manifest.jsonread-modify-write, so ashared manifest loses entries.
The docs
README.mdcotdata.get_pricesdocs/ARCHITECTURE.mddocs/data-path.mdcotdata.get_pricesdrawn as the live read pathThere are three siblings, three counters, and
cotmetrics/srcno longer callsget_pricesanywhere. Missing../marketdatafails at import rather than install(
cotmetrics/__init__.pyre-exportssignals, which importsmarketdataat modulelevel), so README now carries that traceback verbatim.
On the counters: marketdata is the one that carries the weight. The case the guard was
written for, reconstructed volume promoted to default, is a price schema bump, so
watching cotdata alone would leave uncovered the only case it exists for.
The finding worth reading
The two ADR-0007 steps are out of step with each other, and
data-path.mdnow recordsthat rather than smoothing over it.
So the read points at
$MARKETDATA_STORE/bars/futures/while 99 price parquets sit in$COTDATA_STORE/prices/where the producer keeps writing them. Measured against bothlive stores:
A local checkout therefore renders positioning without prices, silently. The old diagram
is kept and marked superseded, since its vendor-abstraction argument still holds; the
measured state goes in a new section beside it with a check a reader can run.
Verification
the
.envpath, equal paths exit 1 naming the manifest reason, distinct paths fallthrough to the app.
bash -n run-local.shclean.ruff check src testsclean, 131 tests pass.get_barsbehaviour measured, not assumed.Not in this PR
.envis gitignored, so theMARKETDATA_STOREline was added to the local file only.Anyone else needs to add it themselves, which the check now tells them to do.
docs/data_architecture.mdis stale in two unrelated ways: it still calls Databento theprimary price provider with a Yahoo fallback (:143) and still names
pardo_quant_frameworkrather than npf (:153).CLAUDE.mdstill lists/damageas a live crowdmon consumer. That pageleft in Remove the /damage page now that crowdmon is deprecated #22 and The last crowdmon references leave the repo with the page #25.
🤖 Generated with Claude Code