Skip to content

Refuse an unregistered --symbols instead of reporting wrote=0 - #18

Merged
mspinola merged 2 commits into
mainfrom
claude/refuse-unregistered-symbols
Aug 21, 2026
Merged

Refuse an unregistered --symbols instead of reporting wrote=0#18
mspinola merged 2 commits into
mainfrom
claude/refuse-unregistered-symbols

Conversation

@mspinola

Copy link
Copy Markdown
Owner

The problem

The three providers disagreed about what to do with a symbol the registry does not carry.

  • databento refuses it — KeyError: not in the marketdata registry: [...]
  • yfinance and norgate filtered it out of their target list and returned ok=True, wrote=0

main() turns ok=True into exit 0. So a fetch scoped to a symbol that is not in registry.yaml printed one line and reported a successful run:

$ marketdata-update --bars --symbols VIX
yfinance: no registry symbols resolve to this provider among ['VIX']

bars_yahoo: wrote=0 failed=0
$ echo $?
0

That matters because of how the producer wrappers read it. run-equities.cmd and run-prices.cmd both branch on if not errorlevel 1, so exit 0 means "fetch succeeded, go on to sync" — and the replica syncs run against a store nothing was written to.

The sharp edge is what it does to a symbol removed from registry.yaml. The removal itself is quiet and local; the failure surfaces later, wherever something still asks for that symbol, as a wrote=0 that looks like a typo rather than a consequence.

The change

--symbols is validated against the registry in main(), before dispatch, and refused via p.error — exit 2, matching the other malformed-request guards, so a fetch failure (1) stays distinguishable from a request that could never have fetched anything.

$ marketdata-update --bars --symbols NOTASYMBOL
marketdata-update: error: not in the marketdata registry: NOTASYMBOL. Nothing
would be fetched for these, so this is refused rather than reported as a run
that wrote 0 rows. Check the spelling, or add them to registry.yaml.
$ echo $?
2

This is not a new policy. It is the rule the file already applies to --require-final and --windowed-n1-stats"a modifier that cannot modify anything is refused, not ignored" (update.py) — extended to the one argument still exempt from it.

Two decisions worth review

The check is in main(), not in each provider. An unscoped --bars runs every domain, so an equities symbol legitimately resolves to nothing in norgate and a futures symbol resolves to nothing in yfinance. A per-provider refusal would fail a run the other half handled perfectly well. test_an_equities_symbol_does_not_fail_the_futures_half pins that.

A partially satisfiable request is refused whole. --symbols SPY NOTASYMBOL fetches nothing rather than fetching SPY. Three of four symbols arriving is not the request that was made, and which half to keep is the caller's call, not the producer's.

Tests

tests/test_cli_symbols.py, 5 cases: unknown symbol refused; refused even when other requested symbols are valid; a registered symbol is not refused (the guard must not over-fire); cross-domain unscoped runs unaffected; no --symbols means no check.

Suite: 215 passed, 2 skipped (was 210 passed, 2 skipped).

🤖 Generated with Claude Code

mspinola and others added 2 commits August 20, 2026 23:38
The three providers disagreed about a symbol the registry does not carry.
databento refused it (KeyError, "not in the marketdata registry"). yfinance and
norgate filtered it out of their target list and returned ok=True with wrote=0,
which main() turns into exit 0.

So `--bars --symbols VIX` against a registry that no longer carries VIX printed
one line and reported a successful run. A wrapper testing `if not errorlevel 1`
reads that as a good fetch and goes on to sync a store nothing was written to.
That is what a symbol retired out of the registry hands to its consumers: not a
failure at the point of retirement, but a silent no-op months later, at whichever
box still asks for it, looking like a typo.

This is the rule the file already applies to --require-final and
--windowed-n1-stats -- "a modifier that cannot modify anything is refused, not
ignored" -- extended to the one argument that was still being ignored. Refused
via p.error, so it exits 2 like the other malformed-request guards rather than 1,
and a fetch failure stays distinguishable from a request that could never have
fetched anything.

The check is in main() rather than in each provider on purpose. An unscoped
--bars runs every domain, so an equities symbol resolves to nothing in norgate
and a futures symbol resolves to nothing in yfinance; a per-provider refusal
would fail a run the other half handled perfectly well. There is a test for
exactly that.

A partially satisfiable request is refused whole. `--symbols SPY NOTASYMBOL`
fetches nothing, because three of four symbols arriving is not the request that
was made and the caller should decide, not the producer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ruff I001. `python -m ruff check src tests` is the CI lint step and the import
block was written as one combined `from ... import a as x, b as y` line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mspinola
mspinola merged commit 80b60df into main Aug 21, 2026
5 checks passed
@mspinola
mspinola deleted the claude/refuse-unregistered-symbols branch August 24, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant