Port the databento futures producer (ADR-0007 step 2, the last part)

[mspinola]

The one part of ADR-0007 that step 2 did not implement. Merge this before cotdata#108, which removes the original.

The ADR is explicit that vendors are providers and never packages, and its own step-2 inventory names providers/databento.py as the largest single file that moves. It stayed in cotdata because porting it is not a file move, and deleting it would have thrown away an ADR-0006-validated alternative.

What moved

The live two-stage producer: paged resumable ingest (paid), the batch-API variant, the both-directions reconcile of the resume ledger, and build (free, offline). Plus both parity harnesses and their tests.

What did not

The dormant per-symbol EOD path — fetch_daily_ohlc, run_batch_backfill, update_all_daily_prices. cotdata's docstring kept it for "the intraday news-failure work" and the code does not support that claim: it fetches ohlcv-1d, so it is a daily path duplicating what the two-stage producer does properly, with its own parallel cache and a yfinance fallback. A fleet-wide sweep found no caller — npf's same-named fetch_daily_ohlc is npf's own shim over get_bars. It stays in cotdata's git history.

One rule changed rather than moved

cotdata defaults a symbol's databento root to its internal symbol unconditionally, which is safe with one domain. Here that would hand every equity a GLBX root and let resolve_source route SPY to a vendor that cannot serve it — so the default is futures-only, and equities are always None.

The eight markets GLBX does not carry (ICE softs, lumber, the dollar index) are explicit nulls: 41 capable of 49, matching cotdata exactly.

This is the third time this port has turned up a rule that was right in a single-domain package and wrong in a two-domain one. The first two were the store's tier axis (one frame per symbol cannot hold backadj and unadj) and the manifest's vendor key.

Two deliberate design calls

_targets() does not filter on resolve_source, unlike the Norgate provider. --build-databento is an explicit ask for this vendor, and every research box defaults to another one, so honouring the deployment default would make an explicit command silently do nothing. A per-symbol price_source override still wins — that is a statement about the symbol, not the machine.

databento keeps its own CLI actions rather than becoming a --bars source. It is the only two-stage producer here, which does not fit a flag meaning "go get today's bars", and folding it in would let an unscoped --bars run on a research box spend money by accident. --batch and --windowed-n1-stats are refused rather than ignored when there is no ingest to modify — same rule as --require-final.

build() now gathers both frames before writing either, so both-tiers-or-neither holds on this producer by construction rather than resting on two adjacent writes.

The parity harness gets simpler

With the vendor as a path component, both series sit in one store — bars/futures/norgate/ beside bars/futures/databento/ — which is the comparison that layout was designed to make possible. --norgate-store/--databento-store now default to $MARKETDATA_STORE and remain for stores produced on different machines.

A test pins that the two vendors cannot read each other's series. In the old flat layout they were the same path, so a comparison could read a series against itself and report perfect agreement.

Verification

ruff check src tests scripts clean. MARKETDATA_NO_NETWORK=1 pytest tests/ — 200 passed, 12 skipped (152 on main + 48 here).

---

Generated by Claude Code