ADR-0007 §7.5: delete the price surface, keep databento

[mspinola]

Closes ADR-0007 step 2 §7.5. cotdata is CFTC positioning now; bars are marketdata.get_bars against $MARKETDATA_STORE.

Merge marketdata#13 first (the producer tests this deletion would otherwise orphan — see "What this exposed"). The --require-final prerequisite is already satisfied: marketdata#12 landed it.

Gone

prices.py (with the derived propadj tier) · providers/norgate.py · providers/yfinance.py · get_prices / roll_dates · store.{write,upsert,read}_metadata · --prices / --metadata / --prices-yahoo / --require-final / --final-cutoff / --full · the norgate and yahoo extras · the packaging dependency that existed only for norgatedata. Migration table is in the CHANGELOG.

Deleted rather than deprecated. A shim left importable would read a store the nightly job no longer fills, and stale data is much harder to notice than an AttributeError — the failure is a number that looks right and is months old. A test asserts the names stay gone.

roll_dates is dropped outright rather than ported: a sweep of the four consumer repos found no caller. npf's roll_dates is its own function (a threshold on the back-adjustment offset, not a Delivery Month change) and is unaffected.

What stayed, and why it is worth saying out loud

databento stays, and with it store.write_prices / read_prices, config.prices_dir() and the prices manifest half it writes through. It has no marketdata equivalent, and deleting it would destroy a validated provider-different alternative (ADR-0006) plus the only intraday-capable source in the fleet. So the consumer bar API left and the store-level pair the retained producer needs did not — read its output with store.read_prices. cotdata-prices survives, scoped to --ingest-databento / --build-databento.

That means "cotdata is COT-only" is now true of the public API and not yet of the store. ADR-0007 needs to record databento as a live exception to its own boundary rather than a step still to do.

The metadata manifest domain stays declared with no writer, so stores written before this still migrate and reconcile their entries instead of stranding them in the legacy aggregate.

What the deletion exposed (not caused)

Three gaps §7.1 left, invisible while cotdata still held a copy:

1. finals_ready() had no caller. Ported in §7.1 but never wired to a CLI flag — cotdata-update --prices --require-final was the only way to reach it. Deleting that would have left the Windows nightly job ungated, and a fetch run before Norgate settles writes a provisional bar over a real one with nothing in the store to say so. Already fixed by marketdata#12. Worth recording that fix: make PEP 604 union annotations import-safe on Python 3.9 #12 and a duplicate written from this branch converged on the same design independently, hours apart: a second pass reaching for the same gate is evidence the deletion could not proceed without it.

2. Six behaviours had tests only here — volume reconstruction, the volume-rank pick (the GC/SI case), the incremental window, full=True, the NDU-down abort, the all-null spec-row skip. §7.1 ported the provider and not its tests. A file-count check would have missed this entirely: the test files existed on both sides, with zero name overlap across 30 deleted tests and a real coverage gap underneath. Ported in marketdata#13, which is why that should merge first.

3. Two harnesses read the Norgate store by path, so no call-site grep finds them — the same class of coupling as the work order's §8.3, one layer out. validate_databento_vs_norgate.py and investigate_databento_roll_rule.py now try bars/futures/norgate/ first and fall back to the old layout, so the ADR-0006 parity gate still runs across the split.

A test that could not do its job

test_every_action_flag_is_assigned_to_a_half compared _HALF_ACTIONS against a list copied beside the parser. It stayed green through this change while naming three flags that no longer exist — so it could not have caught a fourth being added either. It now derives the flag set from the parser, which needed main's argparse block extracted into _parser().

Docs

README, CHANGELOG (0.4.0, breaking), WINDOWS_SCHEDULING, WINDOWS_SETUP, LINUX_SCHEDULING, SYNCING, CONTRIBUTING, the Windows/Linux example wrapper scripts, and the databento parity record. The Windows nightly task is retargeted to marketdata-update --bars --domain futures --require-final — same box, same time, same restart-on-failure trick; every operational fact about NDU and deferred exit codes is unchanged.

Two design records get a banner rather than a rewrite, per the doc lifecycle: finals_ready_data_driven.md (mechanism moved) and reading-the-store.md §4–§5 (updated to the new API but kept here — they are naming-and-composition traps a reader arriving from a COT question walks straight into). crowdmon_step2_normalisation.md is left untouched as a point-in-time proposal.

Known breakage, accepted

crowdmon (frozen, archived) and npf/docs/crowdmon/reproduce_forced_flow_mechanism.py still call cotdata.get_prices. Both are point-in-time records under their repos' doc lifecycle; §8.2 of the work order already voided the crowdmon repoint on the same grounds.

Verification

• ruff check src tests scripts clean; 254 pass (from 289 — the delta is the deleted Norgate/yfinance/propadj files, whose behaviours are now covered in marketdata).
• CLI smoke: --prices is refused with a non-zero exit, and the nothing-to-do message names marketdata-update --bars.
• npf's full suite run against this branch and against main: byte-identical failure sets, 38/38, zero new. Those 38 are pre-existing sibling-checkout gaps in this sandbox, not regressions. npf's CI checks out cotdata's default branch with no ref:, so it becomes the real integration test on merge.
• npf's contract-specs loader now resolves unambiguously to marketdata (specs source -> marketdata.store.read_metadata, also available: ()), instead of warning that both packages could answer — the ADR-0007 fallback it was built for, working.