Refuse an unregistered --symbols instead of reporting wrote=0 - #18
Merged
Conversation
The three providers disagreed about a symbol the registry does not carry. databento refused it (KeyError, "not in the marketdata registry"). yfinance and norgate filtered it out of their target list and returned ok=True with wrote=0, which main() turns into exit 0. So `--bars --symbols VIX` against a registry that no longer carries VIX printed one line and reported a successful run. A wrapper testing `if not errorlevel 1` reads that as a good fetch and goes on to sync a store nothing was written to. That is what a symbol retired out of the registry hands to its consumers: not a failure at the point of retirement, but a silent no-op months later, at whichever box still asks for it, looking like a typo. This is the rule the file already applies to --require-final and --windowed-n1-stats -- "a modifier that cannot modify anything is refused, not ignored" -- extended to the one argument that was still being ignored. Refused via p.error, so it exits 2 like the other malformed-request guards rather than 1, and a fetch failure stays distinguishable from a request that could never have fetched anything. The check is in main() rather than in each provider on purpose. An unscoped --bars runs every domain, so an equities symbol resolves to nothing in norgate and a futures symbol resolves to nothing in yfinance; a per-provider refusal would fail a run the other half handled perfectly well. There is a test for exactly that. A partially satisfiable request is refused whole. `--symbols SPY NOTASYMBOL` fetches nothing, because three of four symbols arriving is not the request that was made and the caller should decide, not the producer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ruff I001. `python -m ruff check src tests` is the CI lint step and the import block was written as one combined `from ... import a as x, b as y` line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The three providers disagreed about what to do with a symbol the registry does not carry.
databentorefuses it —KeyError: not in the marketdata registry: [...]yfinanceandnorgatefiltered it out of their target list and returnedok=True, wrote=0main()turnsok=Trueinto exit 0. So a fetch scoped to a symbol that is not inregistry.yamlprinted one line and reported a successful run:That matters because of how the producer wrappers read it.
run-equities.cmdandrun-prices.cmdboth branch onif not errorlevel 1, so exit 0 means "fetch succeeded, go on to sync" — and the replica syncs run against a store nothing was written to.The sharp edge is what it does to a symbol removed from
registry.yaml. The removal itself is quiet and local; the failure surfaces later, wherever something still asks for that symbol, as awrote=0that looks like a typo rather than a consequence.The change
--symbolsis validated against the registry inmain(), before dispatch, and refused viap.error— exit 2, matching the other malformed-request guards, so a fetch failure (1) stays distinguishable from a request that could never have fetched anything.This is not a new policy. It is the rule the file already applies to
--require-finaland--windowed-n1-stats— "a modifier that cannot modify anything is refused, not ignored" (update.py) — extended to the one argument still exempt from it.Two decisions worth review
The check is in
main(), not in each provider. An unscoped--barsruns every domain, so an equities symbol legitimately resolves to nothing innorgateand a futures symbol resolves to nothing inyfinance. A per-provider refusal would fail a run the other half handled perfectly well.test_an_equities_symbol_does_not_fail_the_futures_halfpins that.A partially satisfiable request is refused whole.
--symbols SPY NOTASYMBOLfetches nothing rather than fetching SPY. Three of four symbols arriving is not the request that was made, and which half to keep is the caller's call, not the producer's.Tests
tests/test_cli_symbols.py, 5 cases: unknown symbol refused; refused even when other requested symbols are valid; a registered symbol is not refused (the guard must not over-fire); cross-domain unscoped runs unaffected; no--symbolsmeans no check.Suite: 215 passed, 2 skipped (was 210 passed, 2 skipped).
🤖 Generated with Claude Code