Record the Windows verification of the marketdata futures port - #104
Merged
Conversation
…utput The handoff said "not yet run against real Norgate" and gave the first commands for whoever picked it up. It has now run, and §7 carries the result rather than the instruction: 49,892 rows per tier across ES, CL, GC, ZS and DC, both tiers, plus contract specs, all identical, exit 0. Exact equality rather than a tolerance, both producers driving the same Norgate install through two code paths. That is what §7.5 needs before cotdata's price code is deleted, and it was obtainable only while both halves exist. It is on the record now, so the deletion no longer waits on it. Two defects surfaced on the first two contacts with the box, and the pattern matters more than either. The provider was ported without its dependency, so --domain futures stopped at the import guard with nothing installing norgatedata. And get_bars had no volume= parameter: the producer half of volume reconstruction was ported, the consumer half was not, so the columns were written and nothing served them — npf's ml/labels.py passes volume= through, so a repoint would have raised TypeError. Neither is visible to a suite that cannot install the vendor or call a parameter that does not exist. §7 states the consequence for the remaining steps: offline green says nothing about the producer box, and each repoint should be exercised there before it is called done. One expectation corrected. The reconstruction columns were expected to drift, each producer reconstructing incrementally over its own store's history. They agree exactly, because Norgate's historical individual-contract volumes are immutable and the algorithm is identical, so incremental converges on full. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jo4iovRfc2fzE9MwcLp7r2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #103 (merged), which shipped the
marketdatafutures provider and leftdocs/handoffs/2026-08-08-adr7-step2-provider-shipped.md§7 reading "Not yet run against real Norgate" with the first commands for whoever picked it up.It has now run, on the Windows producer. §7 carries the result instead of the instruction.
Docs only — no
cotdatacode changes, same as #103.The verdict
marketdata/scripts/verify_against_cotdata.pyagainst acotdatastore built by the original producer:49,892 rows per tier, both tiers, plus contract specs for all five, exit 0. Exact equality, not a tolerance — both producers drive the same Norgate install through two code paths, so any difference would have been a port bug rather than vendor disagreement. Symbols span an index, an energy, a metal, and the two markets whose
backadjhistory goes non-positive.That is what §7.5 needs before
cotdata's price code is deleted, and it was obtainable only while both halves exist. It is on the record now, so the deletion no longer waits on it.What the real box found that the offline suite could not
Two defects, on the first two contacts, and the pattern matters more than either:
--domain futuresstopped at the import guard. The provider was ported without its dependency —cotdatadeclaresnorgate = ["norgatedata"]andmarketdatahad no such extra, so nothing installed it.get_barshad novolume=parameter. The producer half of volume reconstruction was ported and the consumer half was not, so the columns were written and nothing served them.npf'sml/labels.py:50passesvolume=through, so a repointed call would have raisedTypeError.Neither is visible to a suite that cannot install the vendor or call a parameter that does not exist. §7 states the consequence for the remaining steps: offline green says nothing about the producer box, and each repoint in §7.4/§7.5 should be exercised there before it is called done.
The second was caught only because the harness was changed to report the columns it had not compared. It had been printing reconstruction columns solely when they differed, so "compared and identical" and "never compared" rendered as the same silence.
One expectation this corrected
The reconstruction columns were expected to drift, since each producer reconstructs incrementally over its own store's history and
marketdata's was fresh wherecotdata's had months of accumulation. They agree exactly: Norgate's historical individual-contract volumes are immutable and the algorithm is identical, so the incremental path converges on what a full recompute produces.Fixes # (issue) — n/a, this executes crucible-stack ADR-0007 via
docs/handoffs/2026-08-04-adr7-step2-price-producer-split.md.Type of change
Checklist:
pytest) and they pass locally — 119 pass inmarketdata; nocotdatacode changedGenerated by Claude Code