Skip to content

Name the second store, in the launcher and in the docs - #29

Merged
mspinola merged 2 commits into
mainfrom
claude/cot-analyzer-marketdata-import-20aa48
Aug 9, 2026
Merged

Name the second store, in the launcher and in the docs#29
mspinola merged 2 commits into
mainfrom
claude/cot-analyzer-marketdata-import-20aa48

Conversation

@mspinola

@mspinola mspinola commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Follow-on to #26 / #27 / #28. Those made the repo depend on marketdata; this makes a
fresh clone and a fresh run say so, and records where the bars actually are.

Prompted by a real failure: ./run-local.sh died with
ModuleNotFoundError: No module named 'marketdata' on a venv that predated the
dependency. The venv was the immediate cause, but the instructions that venv was built
from are still a step behind the code, so the next clone hits the same wall.

The launcher

MARKETDATA_STORE now gets the same upfront check COTDATA_STORE has, plus a guard
against pointing both at one directory.

Without it the failure is quiet in the worst way: the import succeeds, the app binds
:5001, every COT page renders off the other store, and only the first chart wanting bars
raises. That reads as a UI regression rather than a missing environment variable.

The collision branch is there because the two stores may share a synced parent but never
one directory: each producer rewrites its own manifest.json read-modify-write, so a
shared manifest loses entries.

The docs

file was
README.md two siblings to clone, two editable installs, prices read via cotdata.get_prices
docs/ARCHITECTURE.md cotmetrics depends on cotdata alone; caches busted by two counters
docs/data-path.md cotdata.get_prices drawn as the live read path

There are three siblings, three counters, and cotmetrics/src no longer calls
get_prices anywhere. Missing ../marketdata fails at import rather than install
(cotmetrics/__init__.py re-exports signals, which imports marketdata at module
level), so README now carries that traceback verbatim.

On the counters: marketdata is the one that carries the weight. The case the guard was
written for, reconstructed volume promoted to default, is a price schema bump, so
watching cotdata alone would leave uncovered the only case it exists for.

The finding worth reading

The two ADR-0007 steps are out of step with each other, and data-path.md now records
that rather than smoothing over it.

  • step 4, repoint consumers: done
  • step 2, move the producer code and the bars: on ice

So the read points at $MARKETDATA_STORE/bars/futures/ while 99 price parquets sit in
$COTDATA_STORE/prices/ where the producer keeps writing them. Measured against both
live stores:

~/code/cotdata_store/prices/           99 parquet files
~/code/marketdata_store/bars/          equities/yfinance only, 13 tickers, no futures/
marketdata.get_bars("GC", "backadj")   ->  0 rows, no exception

A local checkout therefore renders positioning without prices, silently. The old diagram
is kept and marked superseded, since its vendor-abstraction argument still holds; the
measured state goes in a new section beside it with a check a reader can run.

Verification

  • All three launcher branches exercised by hand: unset exits 1 naming the variable and
    the .env path, equal paths exit 1 naming the manifest reason, distinct paths fall
    through to the app.
  • bash -n run-local.sh clean.
  • ruff check src tests clean, 131 tests pass.
  • Store contents and get_bars behaviour measured, not assumed.

Not in this PR

🤖 Generated with Claude Code

mspinola and others added 2 commits August 8, 2026 23:44
…ice read

ADR-0007 step 4 landed in cotmetrics, so bars now arrive through
marketdata.get_bars and a price read needs a second store. Without it the
failure is quiet in the worst way: the import succeeds, the app binds :5001,
every COT page renders off the other store, and only the first chart wanting
bars raises. That reads as a UI regression rather than a missing environment
variable, which is exactly the confusion the COTDATA_STORE check upstairs
exists to prevent. Same check, same shape, same reason.

The second branch is the collision case. The two stores may share a synced
parent but never one directory, because each producer rewrites its own
manifest.json read-modify-write and a shared manifest loses entries. Pointing
both variables at one path is the easy way to arrive there, so refuse it here
rather than let a producer discover it later.

Verified all three branches by hand: unset exits 1 naming the variable and the
.env path, equal paths exit 1 naming the manifest reason, distinct paths fall
through to the app. bash -n clean, ruff clean on src and tests, 131 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s are not

The install instructions were a step behind the code and would have led the
next clone into the error that prompted this. README named two siblings and two
editable installs; there are three. Missing ../marketdata does not fail at
install time, it fails at import, because cotmetrics/__init__.py re-exports
signals and signals imports marketdata at module level, so the error arrives as
a ModuleNotFoundError from a package nothing in this repo mentions. That
traceback is now in README verbatim.

The same lag ran through the rest. README said prices are read through
cotdata.get_prices; cotmetrics/src no longer calls it anywhere. ARCHITECTURE
said cotmetrics depends on cotdata alone, and that caches are busted by two
counters. There are three, and marketdata is the one that carries the weight:
the case the guard was written for, reconstructed volume promoted to default,
is a PRICE schema bump, so watching cotdata alone would leave uncovered the
only case it exists for.

data-path.md needed more than a correction, because the two ADR-0007 steps are
out of step with each other and that is worth recording rather than smoothing
over. Step 4 (repoint consumers) is done. Step 2 (move the producer and the
bars) is on ice. So the read points at $MARKETDATA_STORE/bars/futures/ while 99
price parquets sit in $COTDATA_STORE/prices/ where the producer keeps writing
them, and a futures read against a store holding only equities returns an empty
frame with no exception. The old diagram is kept and marked superseded, since
its vendor-abstraction argument still holds; the measured state goes in a new
section beside it with a one-line check a reader can run.

Measured 2026-08-08 against the working tree and both live stores, not read off
the docs, which is how the staleness surfaced. ruff clean on src and tests, 131
tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mspinola
mspinola merged commit 770be06 into main Aug 9, 2026
3 checks passed
@mspinola
mspinola deleted the claude/cot-analyzer-marketdata-import-20aa48 branch August 9, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant