docs(adr): ADR-0008 — COT vintage provenance stays in cotdata, persisted in Parquet - #13
Merged
Merged
Conversation
…quet Answers a question ADR-0007 leaves open: with cotdata being narrowed to COT only, is as-published provenance for that COT data inside the boundary or outside it. Decision: inside, and persisted in the existing Parquet + manifest contract with no database. The ADR-0007 narrowing runs along the instrument-domain axis (COT versus bars), not the derived-versus-raw axis, so vintage capture never crosses that seam. It also has to sit beside the fetch, since capture records what the source served at a moment in time. The original external spec called for DuckDB/SQLite; rejected, because change-only writes keep the whole history in single-digit MB and DuckDB can query the Parquet directly anyway without becoming the storage format. Recorded here rather than in cotdata because the scope half interprets ADR-0007, and a cotdata-local ADR pointing at 0007 would be a one-way link: someone reading 0007 later to decide what cotdata may contain would never discover the boundary had been reasoned about. ADR-0007 therefore gains a back-link at its Decision section, which is the edit that makes the relocation worth anything. cotdata keeps a stub pointing here. Status is Accepted as to the decision, implementation complete but unmerged (cotdata PR #78) — the mirror of 0007, which was accepted as a direction with the work outstanding. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…y monitor) Committed verbatim as authored, before any amendment. v0.1 system description for the equity-side monitor; its §12 names the futures/COT adapter as the current priority, which is what the cotdata vintage work implements. Filed under docs/adr/ as placed; relocated to docs/design/ in the following commit, since docs/adr/ otherwise holds only ADR-NNNN records. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Moved docs/adr/crowdmon_system_description.md -> docs/design/. It is a v0.1 system description, not an architecture decision record, and docs/adr/ otherwise holds only ADR-NNNN files. Left in place for one commit so the as-authored state is in history. - §12 records that step 1 of the futures/COT adapter is built (cotdata PR #78, decision in ADR-0008), plus the two outcomes that bear on this document's own plan: the shared store is Parquet rather than DuckDB, and vintage history is forward-only, so any validation resting on as-of correctness over pre-capture history is permanently unavailable rather than pending. The ADR-0008 link dangles on main until crucible-stack PR #13 merges; called out inline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ADR-0008 travels in the same PR as this document, so that link resolves on the branch and after merge; the dangling-link warning was written while they were on separate branches and no longer describes reality. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Moving that doc out of docs/adr/ brought it under the prose guard for the first time (EXCLUDED_DIRS exempts docs/adr/), and it failed on $100m — the statutory 13F filing threshold. That is a public SEC reporting rule bounding what ownership data can observe, not an account size or a result, so it is a false positive and takes a documented ALLOWED entry rather than a reworded fact. Trade-off worth knowing: ALLOWED exempts the whole file, so if this document later accumulates genuine findings the guard will not catch them. Rewording to drop the numeral is the alternative if that matters more than stating the threshold. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e ALLOWED entry Reverses the previous commit's approach. An ALLOWED entry exempts the WHOLE file from the prose guard, which is far broader than the one false positive warranted: this is a 433-line public-repo document likely to grow, and exempting it wholesale means a genuine finding added later would pass unnoticed. Rewording costs nothing here. The two sites said 'managers >$100m' and 'Managers under $100m'; they now name the 13F reporting threshold directly, which is what the number was standing in for. Arguably more durable too, since the doc no longer goes stale if the SEC ever revises the figure. The guard is now live on the file: it moves from exempted to scanned, and the suite gains a case (357 -> 358). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The status said 'implementation complete but unmerged, pending review'. #78 has now merged, so that line was about to become exactly the kind of stale status section this repo has had to correct before (see #12). Also states what did and did not ship, and that no production vintage has been captured yet -- the subsystem is merged but its scheduled task is not running, so the series still begins at first capture. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Records a decision that sits directly on ADR-0007's boundary, and answers a question 0007 leaves open: with
cotdatabeing narrowed to COT only, is as-published provenance for that COT data inside the boundary or outside it?Decision: inside, and persisted in the existing Parquet + manifest contract with no database.
Why here rather than in cotdata
The scope half of this decision interprets ADR-0007. A cotdata-local ADR pointing at 0007 would be a one-way link: someone reading 0007 later to decide what
cotdatamay contain would never discover the boundary had already been reasoned about.So the load-bearing change in this PR is the back-link added to ADR-0007's Decision section — without it, relocating the file accomplishes nothing.
cotdatakeeps a stub pointing here.Status
Accepted as to the decision, implementation complete but unmerged — it lives on cotdata branch
claude/cot-revision-snapshots-9b196f(cotdata PR #78). This is the ADR-0007 distinction in the opposite direction: 0007 was accepted as a direction with the work outstanding; here the work is done and only the merge is.If cotdata #78 merges first, the status line needs a one-word edit.
Open questions recorded
Carried into the ADR so they live in the architecture log rather than only in a PR description: futures-only capture (
combinedis constant-False, half the reportable universe absent), Legacy-only ingest wiring, whether a detected revision should invalidatecotmetricscaches (it keys onschema_version, which a revision does not bump), and retention policy.Verification
Docs-only, but the full suite was run rather than assumed inert — 357 passed — since this repo has a prose-sensitive
test_no_findings_in_prose.py.🤖 Generated with Claude Code