ADR-0007: record step 2 as implemented, and the one part that was not

[mspinola]

ADR-0007 step 2 shipped 2026-08-09. Steps 1–4 are done, every consumer reads marketdata.get_bars, and cotdata 0.4.0 ships with no bar API, no Norgate and no yfinance.

Doc only — 407 tests pass, and docs/adr/ is excluded from the prose guard.

The part worth being precise about

databento is still in cotdata, and that is a deviation from this ADR — not an exception it carved out. The shipping PRs framed it as the latter; that was backwards, and this corrects it against the ADR's own text:

Norgate futures and databento are not two things. They are two providers of the same thing … Separating them would break the abstraction they were built to share.

and step 2's inventory names providers/databento.py (1015 lines) as the largest single file that moves.

Why it stayed — for no architectural reason. It was neither ported nor deleted:

• Porting it is its own piece of work. Not a file move: a two-stage paid-API producer with an append-only raw bronze store, its own ingest manifest, a two-directional reconcile path, a windowed statistics fetch and a build-stage unit scale — ~1,100 lines with no counterpart in marketdata, whose futures provider is Norgate-shaped.
• Deleting it destroys working, validated code with no replacement. ADR-0006 is Accepted on the strength of its end-to-end validation against Norgate, and it is the fleet's only intraday-capable source.

So it was left as the lesser of two bad options, and the status now says it is outstanding work.

What it costs, written down: cotdata's public API is COT-only; its store is not. Keeping databento keeps store.write_prices/read_prices, config.prices_dir() and the prices manifest half with it. Two packages can still write futures bars — the coupling the contract-specs amendment argued against under "one vendor integration, one home". Nothing is broken today: the dash is on synced Norgate per ADR-0006's Outcome, so the split store is latent, not live.

No shim was built, and none was needed

Consequences planned "a re-export shim and a deprecation window rather than a clean cut", with step 4 to remove it. The cut was clean. Ordering made the shim unnecessary — every consumer was repointed and verified before the delete — and would have made it harmful: a shim left reading cotdata's own store returns a stale number instead of an AttributeError, and that store is no longer filled by the nightly job. Step 4's shim half is void; its store-root half stays open.

Two predictions corrected

• ~700 lines moved, not ~1,700, because databento stayed. cotdata is still roughly 18% price code — the mismatch is reduced, not eliminated.
• It was not "a file move." The clean seam (cot.py importing only store) made the split clean, but the Norgate provider had to be rewritten against the tier-aware store: one frame per symbol cannot hold backadj and unadj at once.

A process finding for the next extract in this style

Step 2 ported the Norgate provider without its tests, invisible for two weeks because the test files existed on both sides. Diffing the test names across the 30 cotdata was about to delete gave zero overlap — which reads like a renaming and hid seven behaviours that would have had no test anywhere once the deletion landed.

The same shape appeared twice more in the same step: finals_ready() was ported and wired to no CLI flag, so deleting cotdata's --require-final would have silently ungated the nightly capture; and two databento parity harnesses read the Norgate store by path, so no call-site grep found them.

A call-site census is not a coverage census. Count what the tests assert and what reaches each entry point, not what files exist.

Open questions, refreshed

propadj is resolved and hardened (marketdata raises when only one stored tier is present rather than returning empty). Two were stale and are corrected:

• livebook now has the specs guard step 3 asked for — livebook.specs.broker_specs calls contract_specs(required=True) and proves Exchange resolves before the broker connects.
• New: livebook/bin/daily.sh guards COTDATA_STORE but not MARKETDATA_STORE, which is where its bars and specs now come from. It fails closed rather than silently, so the consequence is a traceback mid-run instead of a clear refusal at the top — worth making symmetric, in livebook.

---

Generated by Claude Code