From ffe8ac9c97dfc700ac82feb54939fedc7d53e96d Mon Sep 17 00:00:00 2001 From: Matt Spinola Date: Thu, 6 Aug 2026 19:48:49 -0400 Subject: [PATCH 1/2] File the forced-flow respecification: the test measured a variable that cannot answer it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A specification correction, explicitly NOT a work order. Section 7 says what would have to change before any of it could be executed, and section 8 states the count executing it would carry. Nothing here changes a published number, a weight, a threshold in src/, or the verdict. The mechanism test's section 8 records four places its specification was wrong. This is a fifth, larger than those four together and invisible until the flow composition was looked at. The outcome variable is Δ pool_net. On the sell side an agreeing pool is long and sells when price falls to F*, while a contradicted pool is short and ADDS shorts at the same level. Both are sell flow, both give Δnet < 0, and only the first is forced. The variable cannot separate a forced exit from a fresh entry in the same direction. This is not a new insight about futures. It is the opening paragraph of futures/flow.py: "Net position change cannot tell them apart, because Δnet = ΔLong − ΔShort is the same either way, and that is the whole reason this module exists." flow.decompose exists for the distinction and flow_state is a published column; the frozen section 5 references neither. Advisory and untested, from the artifact the verdict already commits: on the published flow_state of the week following a crossing, the named pool does what the trigger says and the contradicted pool does the opposite, in all four (report type, side) cells, a 3x to 5x separation the net erases. Disaggregated sell shows long_liquidation at 20.1% agreeing against 3.9% contradicted, and new_shorts at 9.4% against 25.8%. Swapping Δnet for flow_state is NOT the fix and the file says so: the labels are partly mechanical, because a net-long book has a bigger gross long leg and is likelier to be labelled long_liquidation, so the same confound reappears one level down. Section 4 proposes the fractional change in the exposed leg instead (Δlong_pool/long_pool on the sell side), which is a level ratio rather than a category and needs no sign flip. It also moves the primary contrast to crossed vs not crossed WITHIN the agreeing group, gives the falsification a pass criterion testing the difference between contrasts directly, and adds a side-consistency condition: TFF's surviving residual is sell -0.0072 against buy +0.0037. Section 5 lists what carries over unchanged, so a re-specification does not rewrite the parts that worked. Section 6 is a separate presentation consequence that needs no re-run: pool_agrees == False currently SUPPRESSES the reading in report.format_offside and cot-analyzer's damage page, where the measurement says the level is live and only the flow's composition differs. Section 7 also records that waiting does not help. The sample spans 57 blocks of 13 weeks and a further year adds about 4, roughly 7% more data and 3% tighter standard errors. The original's section 1 chose this question partly because its evidence regenerates, which is true in the sense that it is never spent and false in the sense of getting usefully stronger. Co-Authored-By: Claude Opus 5 --- .../2026-08-06-forced-flow-respecification.md | 228 ++++++++++++++++++ docs/handoffs/README.md | 1 + 2 files changed, 229 insertions(+) create mode 100644 docs/handoffs/2026-08-06-forced-flow-respecification.md diff --git a/docs/handoffs/2026-08-06-forced-flow-respecification.md b/docs/handoffs/2026-08-06-forced-flow-respecification.md new file mode 100644 index 0000000..b53b387 --- /dev/null +++ b/docs/handoffs/2026-08-06-forced-flow-respecification.md @@ -0,0 +1,228 @@ +# Specification correction: the forced-flow mechanism test measured the wrong variable + +**Status:** **SPECIFICATION CORRECTION. NOT A WORK ORDER, and nothing here is to be executed +as it stands.** §7 lists what must be settled before any of it could be, and §8 states the +count that executing it would carry. A session that runs this as written is running an +un-pre-registered test on data that has already produced one verdict. + +Written 2026-08-06, after +[`2026-08-05-forced-flow-mechanism-test.md`](2026-08-05-forced-flow-mechanism-test.md) was +executed and closed. It amends nothing in that file: that body is the record of what was +actually asked, and its §8 is the record of what came back. + +--- + +## 1. What this is, and what it is not + +The mechanism test returned `supported` on §5.6's criteria and a **marginal lean** once its +own placebo was accounted for. Its §8 records four places the specification was wrong or +unexecutable. **This file is about a fifth, which is larger than the other four together and +was not visible until the flow composition was looked at.** + +It is a corrected design, filed so that the register carries the correction beside the result +rather than in a session's memory. It is **not** a request to re-run. §7 is explicit about why +not yet. + +**Nothing here changes any published number, any weight, any threshold in `src/`, or the +verdict in `npf/docs/crowdmon/2026-08-06-forced-flow-mechanism-verdict.md`.** + +--- + +## 2. The defect, in one paragraph + +The test's outcome variable is `u = Δ pool_net / open_interest` +([`2026-08-05-forced-flow-mechanism-test.md`](2026-08-05-forced-flow-mechanism-test.md) §5.3). +On the sell side, a pool that **agrees** is long and sells when price falls to `F*`; a pool +that is **contradicted** is short and *adds shorts* when price falls to the same level. Both +are sell flow, both give `Δnet < 0`, and only the first is forced. The variable cannot +separate a forced exit from a fresh entry in the same direction, so a differential built on it +is comparing two different mechanisms that share a sign. + +This is not a new insight about futures. It is the first paragraph of +[`../../src/crowdmon/futures/flow.py`](../../src/crowdmon/futures/flow.py): + +> Net position change cannot tell them apart, because `Δnet = ΔLong − ΔShort` is the same +> either way, and that is the whole reason this module exists. + +**The pre-registration chose an outcome variable that this package already documents as +unable to answer the question it was asked.** `flow.decompose` exists for exactly this +distinction, `flow_state` is a published column, and neither is referenced anywhere in the +frozen §5. + +--- + +## 3. The evidence, and its status + +**Advisory and descriptive. Not a result, not tested, and not to be cited as one.** Produced +2026-08-06 from the artifact the verdict already commits +(`npf/docs/crowdmon/reproduce_forced_flow_mechanism.py`, its `crossings.parquet`) joined to +the published `flow_state` of the week following each crossing. If any of it is to become +evidence it needs its own pre-registration and its own variant count. + +Share of the week after a crossing carrying each `flow_state`, by group: + +| report / side | group | `long_liquidation` | `new_shorts` | `short_covering` | `new_longs` | +|---|---|---|---|---|---| +| disaggregated / sell | agrees | **20.1%** | 9.4% | 2.6% | 5.6% | +| disaggregated / sell | contradicted | 3.9% | **25.8%** | 11.6% | 2.8% | +| disaggregated / buy | agrees | 2.4% | 6.7% | **31.0%** | 4.3% | +| disaggregated / buy | contradicted | 7.3% | 3.4% | 8.0% | **18.1%** | +| tff / sell | agrees | **15.7%** | 4.3% | 6.6% | 8.9% | +| tff / sell | contradicted | 4.8% | 9.0% | 12.6% | 6.3% | +| tff / buy | agrees | 5.4% | 10.5% | **11.2%** | 5.5% | +| tff / buy | contradicted | 12.6% | 6.8% | 4.7% | **11.9%** | + +In all four cells the named pool does what the trigger says it does and the contradicted pool +does the opposite thing, a 3x to 5x separation, while the net change the test measured is the +same sign for both. + +### The caveat that stops this being the fix as it stands + +**The flow labels are themselves partly mechanical, and the same confound reappears one level +down.** A net-long book has a larger gross long leg, so its weekly moves are more likely to +dominate the classification and be labelled `long_liquidation`. Classifying by which leg +dominated therefore inherits a version of the defect §8 already found in the sign of +`pool_net`. A design that swapped `Δnet` for `flow_state` and changed nothing else would be a +better test with the same structural flaw. + +--- + +## 4. The amended §5 + +Four changes. The first is the one that matters; the rest close the routes by which the +original could return a positive without a mechanism. + +### 4.1 Outcome variable: the exposed leg, fractionally + +Replace `u = Δ pool_net / open_interest` with the fractional change in the leg the trigger +claims is forced: + +``` +sell side: u = Δ long_pool / long_pool (the long book the flip forces out) +buy side: u = Δ short_pool / short_pool (the short book the flip forces to cover) +``` + +Scale-free, bounded below by -1, and directly the quantity the claim is about. It is a +**level ratio, not a category**, so it does not inherit §3's classification confound, and it +is far less exposed to net mean reversion than `Δnet` because it never mixes the two legs. + +No sign flip is needed: on both sides more negative already means more of the exposed book +leaving, so §5.3's `-1` on the buy side goes away and with it one place a convention error +could hide. + +**Data note.** The gross legs per category are **not** in the published damage panel; they +are in the canonical long panel that `futures/cot_adapter.py` loads. So this outcome variable +cannot be built from `CROWDMON_STORE` alone, which is a real change to §6's "the evaluator +needs no crowdmon computation" and must be stated in any re-specification. + +### 4.2 Primary contrast: crossed vs not crossed, within the agreeing group + +| | original | amended | +|---|---|---| +| primary | agrees vs contradicted, among crossings | crossed vs not crossed, **within agrees** | +| held fixed | the crossing | the book's side, and now its leg | +| contradicted group | the baseline | a **falsification**, §4.3 | + +Both arms of the amended primary hold the same sign of `pool_net`, so the mean-reversion +baseline is identical on both sides of the comparison and cannot generate the result. This is +what §8's fourth bullet was pointing at. + +### 4.3 The falsification needs a criterion, which is the whole gap + +§5.3 stated the expectation that the non-crossing reading "should be near zero" and §5.6 +attached no threshold to it. That is how the criteria came to be satisfied by an effect that +was mostly the group label. + +Whatever the amended primary is, the contradicted group's version of the same contrast is a +**pre-registered falsification with a pass condition**, and the condition tests the +*difference between the two contrasts directly*, with its own bootstrap draw. Comparing two +p-values is not a test of their difference, and "significant here, not there" is exactly the +reading that produced the original verdict. + +### 4.4 Side consistency, and the level of the position + +- **Side consistency, by analogy with §5.6's asset-class condition.** The original's stronger + surviving residual, TFF at p 0.0099, is a net of a sell side at -0.0072 and a buy side at + **+0.0037** (verdict §4 and its `ffm-per-side-2026-08-06.csv`). A condition requiring both + sides to carry the sign would have caught that; §5.6 has none. +- **Control the level of `pool_net`, not only its sign.** Mean reversion scales with position + size and crossing weeks are not size-neutral, so signed `pool_net / OI` belongs in the + design as a stratifier or a covariate rather than being collapsed to a sign. + +--- + +## 5. What carries over unchanged + +Not everything in §5 was wrong, and a re-specification that rewrites the parts that worked +would cost its own errors: + +- **§3.1's vectorisation.** Reproduced exactly at `2.220446049250313e-16` over 96 comparisons. + Keep it, keep the check, keep it as a gate. +- **§5.1's universe rule** (frozen by rule rather than by hand-picked list), the `{20, 60, 250}` + lookbacks, and the five-session crossing window. +- **§5.5's block bootstrap** over 13-week calendar blocks taking every market in those weeks, + and its ban on the IID version. Only the p-value definition needs the correction §8 records. +- **§5.9's store pinning**, in full. +- **§3.2's scope refusal.** Nothing here is real-time actionable, and a re-specification + inherits that or voids itself. + +--- + +## 6. The presentation consequence, which is separate and actionable now + +Not part of any re-run, and filed here only so the pointer exists. + +The measured non-specificity has a consequence for what is *rendered*, independent of whether +the mechanism test is ever re-run. `report.format_offside` +([`../../src/crowdmon/futures/report.py`](../../src/crowdmon/futures/report.py)) and +cot-analyzer's damage page both treat `pool_agrees == False` as a **suppression**: the +quadrant label is withheld and replaced with "the pool is on the other side, so this level +would force a book that is not there". + +That is half right. The forced book is not there. But price still reaches the level, and flow +of comparable net size still follows; what differs is its composition. Suppressing the row +hides a real event rather than describing it. + +The shape of the change, for whoever picks it up: lead with the level and its two existing +docstring caveats (time-series momentum rather than a breakout; a moving reference that closes +distance without the market moving); replace suppression with **re-labelling**, so a +contradicted row reads as "level live, flow expected, arriving as fresh exposure rather than +as an exit"; and demote `pool_agrees` from a gate to a composition flag, stating what it is, +the sign of `pool_net` and nothing more, with no lookback dependence. + +--- + +## 7. Why this is not a work order, and what would have to change + +**Do not open this as a task.** Three reasons, in order of weight: + +1. **It is a materially different test, so it needs its own pre-registration**, written by + someone who has not seen §3's table above, and executed by a session that did not write + it. Filing a corrected design and executing it in the same lineage is the failure the + generator/evaluator split exists to prevent. +2. **It would be a second look at data that has already produced a verdict.** The honest way + to run it is as declared re-analysis carrying its own count, not under the original + pre-registration's protection, and §8 must say so. +3. **Waiting does not fix that, and does not buy much either.** The sample already spans + 2012-05-15 to 2026-07-21, **57 blocks** of 13 weeks. A further year adds about 4, roughly + 7% more data and about 3% tighter standard errors. §1 of the original chose this question + partly because its evidence regenerates weekly, which is true in the sense that it is never + *spent* and false in the sense of getting usefully stronger on any horizon worth waiting + for. Recorded here because the original's reasoning implies otherwise. + +What would make it worth opening: a decision that the mapping's strength matters to how the +trigger column is **published**, which is §6 and does not need a re-run, or an independent +reason to revisit the composite. Neither is a strategy question, and neither licenses trading +`D` or the trigger (§5.7, and §A.10). + +--- + +## 8. Count, if it is ever executed + +The amended design as sketched is **2 report types x (primary + falsification)**, so **4**, +plus whatever robustness readings its own pre-registration declares. §3's table above is +advisory and carries no count because it is not evidence; it becomes countable the moment +anything is claimed from it. + +`run_gauntlet` is not the instrument, unchanged from §5.8: this produces no trades and no +`TradeLog`. diff --git a/docs/handoffs/README.md b/docs/handoffs/README.md index b40e218..b619318 100644 --- a/docs/handoffs/README.md +++ b/docs/handoffs/README.md @@ -49,6 +49,7 @@ A moved handoff leaves a **pointer row and no copy**. One document, one lineage. | [2026-08-03-index-share.md](2026-08-03-index-share.md) | **COMPLETE as far as it can go, 2026-08-03.** The only handoff here authored in another repo (cotdata#97, moved here as PR #36; a second session filed a duplicate copy before checking the remote, which is what the claim-first convention below exists to stop). §1: **the premise is retired, exactly as §4 anticipated.** Index positioning is not meaningfully stickier than swap. Swap is the MORE persistent at 12 weeks (median 0.826 vs 0.777, index ahead in only 4 of 13), index is 14% steadier week to week, and **under the worst 5% of weeks the swap book moves less than the index book** (-0.00167 vs -0.00336), adding to net long in 3 of 13 where index never does. §2 was recorded as blocked on **§B33-B36**, and re-derived blind as **`2026-08-03 §C1-C4`**. **Both readings are now corrected** (`2026-08-03-b-series-recovery.md` §2, and this file's appended §7): §B33-B37 **existed all along**, on an unpushed branch, so 08-02 closed at **B37** rather than B32 and §2 was in fact executable in full a day before it was called blocked. The cited figures were **not invented**: "22 of 39" and "17 in both halves" reproduce exactly. Of §2's three headline figures one **cannot respond to `w_SD`** (the shape rule reads two nets and their signs), one (`A_agnostic`) is **defined by `§B34` at a median of 3.0237**, not undefined as §C4 first read it, and `Q_sell`/`Q_buy` answers: median `A` swings **42.0% on the 13 Supplemental markets** against 0.6% pooled, so **`w_SD` is load-bearing exactly where the handoff cares**. Unsought finding: swap sits at 0.305 of Managed Money on routine turnover and 0.067 under stress, so `swap: 0.4` is incoherent **between regimes, not between markets**. **Metals are permanently outside coverage, so most of §0 is still open.** Weight table unchanged | | [2026-08-03-swap-dealer-weight-decision.md](2026-08-03-swap-dealer-weight-decision.md) | **COMPLETE 2026-08-03 (PR #44), decided by the human as option (a): `swap: 0.4` stands.** Its §4 released exactly one piece of work and that work **corrected the handoff's own §1**: `single_weight_sweep` shipped, and re-swept over the order-preserving band it showed §1's 42.0% was the wrong BAND rather than wrong arithmetic. Honest figures **17.9% on the 13 Supplemental markets, 3.5% pooled** over `w_SD ∈ [0.2, 0.4]` (`§C10`). **Decided in parallel with `§C6-C8`, which landed as PR #43 while this was in flight and reached the same place by a different road**: §C6 closed option **(c)** by settling static weights independently, §C7 measured the cost of (a) as a **+19.6% median `Phi` inflation, worst on gold at +27.8% (2.30x cocoa) and precisely during stress weeks**, and §C8 showed most of that never reaches `D`, because the composite consumes a percentile and the median market-week moves only 5.9 of them. **§C8's split is the operating rule: on classic outrights the band is a footnote (Spearman 0.954, 18 of 20 top markets in common); on the ERCOT and PJM book it can invert a market's own history (Transco Zone 6 at -0.416), so publish the band there.** Two independent measurements, one decision, and no re-litigation. Option (c) is unscheduled rather than closed; metals remain permanently outside coverage | | [2026-08-05-forced-flow-mechanism-test.md](2026-08-05-forced-flow-mechanism-test.md) | **COMPLETE, executed 2026-08-06 in `npf` (PR #79). Verdict a MARGINAL LEAN, and §5.6's `supported` must not be quoted on its own.** When the trigger says a pool is forced out at `F*` and price reaches `F*`, does that pool move? The criteria pass on both report types, but **§5.3's own placebo carries 73.8% (Disaggregated) and 51.9% (TFF) of the effect** where §5.3 says it should be near zero, because the group label **is** the sign of `pool_net` and `pool_net` mean-reverts. Removing the label leaves **-0.001388 (p 0.047)** and **-0.002657 (p 0.0099)**, about 1.2% and 1.9% of the pool against headline figures of 4.5% and 4.0%; Disaggregated would not survive a correction across even the two report types. **The gap is the finding: §5.6 attaches no criterion to the placebo**, so the design as written cannot separate forced flow from positioning mean reversion, and a re-run should fix that first. Three more corrections in §8: §4's control is **not** readable from the panel (39 rows, one week, as §3 itself measures) and was recomputed over all 1,051 weeks, `pool_agrees` reducing to the SIGN of `pool_net` with **no lookback dependence**; §4's "the two groups share the price mechanics exactly" is **false as measured**, the contradicted group crossing more often in both types (25.60% vs 20.11%, 24.20% vs 21.80%); and §5.5's p-value **cannot be computed as written**, a self-centred bootstrap giving `p_literal` 0.465-0.526 across all six variants including those with `p_null` 0.0000. §3.1's equivalence reproduced exactly at 2.220446049250313e-16 over 96 comparisons. Count **8 variants, not 6**. Verdict: `npf/docs/crowdmon/2026-08-06-forced-flow-mechanism-verdict.md` | +| [2026-08-06-forced-flow-respecification.md](2026-08-06-forced-flow-respecification.md) | **SPECIFICATION CORRECTION, NOT A WORK ORDER. Do not open this as a task; §7 says what would have to change first.** A fifth defect in the mechanism test above, larger than the four its §8 records: the outcome variable `Δ pool_net` **cannot answer the question it was asked**, and this package already says so. On the sell side an agreeing pool is long and sells at `F*` while a contradicted pool is short and **adds shorts** at the same level; both are sell flow, both give `Δnet < 0`, and only the first is forced. `futures/flow.py`'s opening paragraph states exactly this ("Net position change cannot tell them apart... that is the whole reason this module exists"), `flow.decompose` exists for the distinction, and the frozen §5 references neither. Advisory and untested: on the published `flow_state` of the following week the named pool does what the trigger says and the contradicted pool does the opposite, in **all four (report type, side) cells**, a **3x to 5x** separation the net erases (disaggregated sell `long_liquidation` **20.1% agreeing against 3.9% contradicted**, `new_shorts` 9.4% against 25.8%). **The obvious swap to `flow_state` is NOT the fix**: the labels are partly mechanical, since a net-long book has a bigger gross long leg and is likelier to be labelled `long_liquidation`, so the confound reappears one level down. §4 proposes the **fractional change in the exposed leg** instead, plus a within-group primary contrast, a falsification **with a criterion** (the gap that produced the original verdict), and a side-consistency condition (TFF's surviving residual is sell -0.0072 against buy **+0.0037**). §6 is a separate, actionable-now presentation consequence: `pool_agrees == False` currently **suppresses** the reading in `report.format_offside` and cot-analyzer's damage page, where the measurement says the level is live and only the flow's composition differs | | **2026-08-05-fragility-orthogonality.md** — **MOVED to `npf/docs/handoffs/`, 2026-08-05, before execution.** Authored here and relocated the same day, which its own §7 named as conditional on `npf` growing a status register. It is **npf-owned work** (a question about npf's book) and `crowdmon` must not be read as commissioning strategy research. **No copy is kept here**, deliberately: `2026-08-03-index-share.md` records what two lineages of one file cost, and a pointer from the first day is the remedy this repo's `CLAUDE.md` states. Its cross-repo citations were qualified on the move and the change is recorded in its own §0; no threshold, statistic or criterion moved. Status is tracked **there**, not here | **MOVED** | | [2026-08-03-b-series-recovery.md](2026-08-03-b-series-recovery.md) | **COMPLETE, outcome as §7 and §8.** §1 recovered `claude/template-followups-doc-corrections-45de1d` (`11b7c81`), local-only with 2,070 insertions, by pushing it unaltered. §2 landed its content on `main` by **porting rather than merging** (the branch predates six merges and would have reverted `flow.py`, `check_skips.py`, `conftest.py` and the whole 08-03 lineage), so **that branch should be CLOSED unmerged**. `amendments-2026-08-02.md` is now B1-**B37**: it never closed at B32, it closed on a branch nobody could see. **§C4 is superseded**, having read `A_agnostic` as WEIGHT-agnostic where `§B34` defines it as DIRECTION-agnostic at a median of **3.0237**; corrected in place with both readings pinned. The cocoa split point is **resolved**: same date, 2025-10-21, differing only on which half that week falls in, and every other figure identical under both rules. §3 changed the citation convention to **path plus reproducer** and made the bare form fail loudly (`tests/test_references.py` over 368 references in 42 files; one unresolved at the time, `§C5`, recorded rather than deleted and then forced out by its own guard when #42 merged). §4 ran the band `w_SD ∈ {0.067, 0.2, 0.4, 0.7}` under a **settled static-weights decision** and filed `2026-08-03 §C6-C8`: `w_SD = 0.4` overstates fragile capital on 99.31% of market-weeks by a median +19.6%, **gold at +27.8% is 2.30x cocoa at +12.1% so the prediction holds**, and 0.067 moves the ceiling 10.0 to 14.925 so the band is not on one scale. Most of it never reaches the composite, which consumes `pct(Phi)`, except on **98 of 264 markets** where the two tables reorder a market's own weeks. **Weight table unchanged** | From 4bcd1221c9af4cd7addc6f643721101dc73fbb16 Mon Sep 17 00:00:00 2001 From: Matt Spinola Date: Thu, 6 Aug 2026 19:54:52 -0400 Subject: [PATCH 2/2] Correct section 6, and file the copy change it releases as a work order Two corrections to section 6 of the respecification, both narrowing it, made because an overstated work order is how a copy change becomes a claim the data does not carry: - "Suppressing the row hides a real event" was wrong. The row is not hidden. It is plotted with a hollow marker, named in the legend, and given a grid cell. What is wrong is only the copy: "would force a book that is not there" implies nothing follows, and something does. - The quadrant suppression itself is correct and stays. The quadrant's severe axis is D, a conditional severity for the NAMED pool, so placing a contradicted row in a cell would attach the wrong pool's severity to a real level. Reading instruction 1 of cot-analyzer's damage page has this right. Section 6 also now states that the composition split may not enter user-facing copy. "It arrives as fresh shorts rather than as long liquidation" is the untested part of that file, and a rendered surface asserting it would convert an advisory crosstab into a published claim. New handoff 2026-08-06-trigger-contradicted-copy.md, status open, the only part of this lineage actionable without a re-run. It rests only on what the verdict tested: flow follows a crossing either way (disaggregated sell, within-group, -0.0190 agreeing against -0.0184 contradicted) and the crossing-specific difference is small (-0.0014 and -0.0027). Its section 5 is the reason it is a handoff rather than a three-line patch. The phrase "pool on the other side" is authored locally in cot-analyzer's damage.py in two places and is NOT in test_damage_vocabulary.py's FRAGMENTS, even though it carries crowdmon's reading exactly as the QUADRANT phrases do. A session that changes only crowdmon's copy ships a producer and a page saying different things with every test green. So the work is: crowdmon first (named constant, COLUMN_DEFINITIONS, manifest), cot-analyzer second (read it, delete both local copies, extend the guard). Section 4 names five things that must not change while doing it, including that SCHEMA_VERSION must not be bumped, since publish.py records that a bump ahead of a consumer release loses the whole /damage page rather than the new key. Co-Authored-By: Claude Opus 5 --- .../2026-08-06-forced-flow-respecification.md | 40 +++-- .../2026-08-06-trigger-contradicted-copy.md | 141 ++++++++++++++++++ docs/handoffs/README.md | 1 + 3 files changed, 172 insertions(+), 10 deletions(-) create mode 100644 docs/handoffs/2026-08-06-trigger-contradicted-copy.md diff --git a/docs/handoffs/2026-08-06-forced-flow-respecification.md b/docs/handoffs/2026-08-06-forced-flow-respecification.md index b53b387..4365ebe 100644 --- a/docs/handoffs/2026-08-06-forced-flow-respecification.md +++ b/docs/handoffs/2026-08-06-forced-flow-respecification.md @@ -179,16 +179,36 @@ cot-analyzer's damage page both treat `pool_agrees == False` as a **suppression* quadrant label is withheld and replaced with "the pool is on the other side, so this level would force a book that is not there". -That is half right. The forced book is not there. But price still reaches the level, and flow -of comparable net size still follows; what differs is its composition. Suppressing the row -hides a real event rather than describing it. - -The shape of the change, for whoever picks it up: lead with the level and its two existing -docstring caveats (time-series momentum rather than a breakout; a moving reference that closes -distance without the market moving); replace suppression with **re-labelling**, so a -contradicted row reads as "level live, flow expected, arriving as fresh exposure rather than -as an exit"; and demote `pool_agrees` from a gate to a composition flag, stating what it is, -the sign of `pool_net` and nothing more, with no lookback dependence. +That is half right. The forced book is not there. But price still reaches the level and flow +still follows, of a size the verdict measures as only slightly smaller. + +**Two corrections to an earlier draft of this section, both narrowing it.** They are made +here rather than left standing because an overstated work order is how a copy change becomes +a claim the data does not carry. + +- **"Suppressing the row hides a real event" was wrong.** The row is not hidden. It is + plotted with a hollow marker and named, and the grid cell says `(pool on the other side)`. + What is wrong is narrower and only about the copy: "would force a book that is not there" + implies nothing follows, and something does. +- **The quadrant suppression itself is correct and must stay.** Reading instruction 1 of + cot-analyzer's damage page has this right for a reason this section does not overturn: the + quadrant's severe axis is `D`, which is a conditional severity for the *named* pool, so + placing a contradicted row in it would attach the wrong pool's severity to a real level. + The change is that the row should carry its own positive description instead of an + absence, not that it should be given a cell. + +The shape of the change, for whoever picks it up, and see the work order in +[`2026-08-06-trigger-contradicted-copy.md`](2026-08-06-trigger-contradicted-copy.md): lead +with the level and its two existing docstring caveats (time-series momentum rather than a +breakout; a moving reference that closes distance without the market moving); replace the +absence with a statement resting **only on what the verdict tested**, which is that flow +follows either way and the difference between the groups is small; and demote `pool_agrees` +from a gate to a composition flag, stating what it is, the sign of `pool_net` and nothing +more, with no lookback dependence. + +**§3's composition split may not enter user-facing copy.** "It arrives as fresh shorts rather +than as long liquidation" is the untested part of this file, and a rendered surface asserting +it would convert an advisory crosstab into a published claim. --- diff --git a/docs/handoffs/2026-08-06-trigger-contradicted-copy.md b/docs/handoffs/2026-08-06-trigger-contradicted-copy.md new file mode 100644 index 0000000..b48448c --- /dev/null +++ b/docs/handoffs/2026-08-06-trigger-contradicted-copy.md @@ -0,0 +1,141 @@ +# Handoff: a contradicted trigger row is described as an absence, and it is not one + +**Status:** open + +Written 2026-08-06. Released by §6 of +[`2026-08-06-forced-flow-respecification.md`](2026-08-06-forced-flow-respecification.md), +which is a specification correction and deliberately not executable. **This one is a work +order**, and it is the only part of that lineage that can be acted on without a re-run. + +--- + +## 1. What this asks + +> When `trigger_{side}_pool_agrees` is `False`, both renderers say the level "would force a +> book that is not there". Make the copy say what was measured instead. + +A copy and vocabulary change across two repos. **No data change, no threshold change, no +change to `D`, no new column, and no change to what is suppressed.** If a diff touches a +number, it is out of scope. + +--- + +## 2. Why, and the line between what may and may not be said + +`npf/docs/crowdmon/2026-08-06-forced-flow-mechanism-verdict.md` measured what follows a +crossing in each group. The relevant part, and **only** this part, is what the new copy may +rest on: + +- Flow follows a crossing whether or not the named pool is present. Within-group, crossing + weeks against non-crossing weeks, the Disaggregated sell side moves **-0.0190** where the + pool agrees and **-0.0184** where it is contradicted (verdict §4 and + `ffm-per-side-2026-08-06.csv`). +- The crossing-specific difference between the groups is **small**: a + difference-in-differences of **-0.0014 (p 0.047)** and **-0.0027 (p 0.0099)**, against + headline group differences of 4.5% and 4.0% of the pool. + +**What may NOT be said, and this is the sharp edge of this handoff.** The composition split in +§3 of the respecification, that the contradicted pool adds fresh exposure while the agreeing +pool liquidates, is a **descriptive crosstab of published labels with no inference behind it +and a known mechanical confound**. It is motivation for a future test. A rendered surface +asserting it would convert an advisory number into a published claim, which is the failure +this whole lineage has been correcting. Copy that says "flow still follows, and the difference +is small" is supported. Copy that says "it arrives as fresh shorts" is not. + +--- + +## 3. What is actually wrong, precisely + +**The row is not hidden**, and an earlier draft of the respecification's §6 said it was. It is +plotted with a hollow marker, it is named in the legend, and it gets a grid cell. Three pieces +of copy are wrong, all in the same way: they describe an absence where there is an event. + +| where | today | why it is wrong | +|---|---|---| +| `report.format_offside`, [`../../src/crowdmon/futures/report.py`](../../src/crowdmon/futures/report.py) | "this level would force a book that is not there. Signal-implied, not held." | true of the forced book, implies nothing follows | +| `damage.py` legend, cot-analyzer | "pool on the other side, **no cell**" | names the missing label, not the state | +| `damage.py` grid cell, cot-analyzer | "(pool on the other side)" | reads as a null beside three populated columns | + +--- + +## 4. What must not change + +Five things, each of which someone could plausibly touch while doing this and must not. + +1. **The quadrant suppression stays.** Reading instruction 1 of the damage page is correct: + the quadrant's severe axis is `D`, a conditional severity for the **named** pool, so + placing a contradicted row in a cell attaches the wrong pool's severity to a real level. + This handoff gives the row a description, not a cell. +2. **`SCHEMA_VERSION` must NOT be bumped.** `publish.py` is explicit that a bump shipped ahead + of a consumer release loses the whole `/damage` page rather than the new key, and no + existing key changes meaning here. Copy is additive. +3. **No countdown, no forecast.** The reference bar moves faster than spot (on 6C, 1.68x), so + distance closes without the market doing anything. Both docstrings already say this and + neither surface does. +4. **`D` stays non-directional** and stays unmultiplied by the distance. +5. **Nothing real-time actionable.** The verdict is a mechanism claim on revised first + differences. Copy implying a reader could have acted voids it. + +--- + +## 5. The structural fix, which is the actual work + +The obvious version of this task is editing three strings. That would leave the same defect +that made it possible. + +`cot-analyzer/tests/test_damage_vocabulary.py` guards crowdmon's vocabulary against being +typed into the consumer: `SCORE_STATES`, `UNWIND_STATES`, `STRATA`, the `QUADRANT` phrases and +`DAMAGE_BANDS`. **"pool on the other side" is not on that list**, and it is authored locally in +`damage.py` in two places, even though it carries crowdmon's reading exactly as the `QUADRANT` +phrases do. So a session that changes only crowdmon's copy will ship a producer saying one +thing and a page saying the old thing, with every test green. + +The work is therefore, in order: + +1. **crowdmon**: give the contradicted state a named constant beside `QUADRANT`, put its + phrasing in `COLUMN_DEFINITIONS`, and publish it in the manifest with the rest of the + vocabulary. `format_offside` renders the constant. +2. **cot-analyzer**: read that string from the manifest in both places, delete both local + copies, and **add its fragments to `FRAGMENTS`** so the next copy cannot come back. + +Step 2's guard extension is the part that makes this stick, and it is the reason this is a +handoff rather than a three-line patch. + +--- + +## 6. Proposed copy, as a proposal + +Wording is the executing session's call; these carry the constraints of §2 and §4. + +- **The state**: "level live, pool on the other side". It names what is true rather than what + is missing. +- **The explanation**, `format_offside`: "The level is real and the pool the trigger names is + not on that side. Flow of a similar size has historically followed a crossing either way, + with the difference between the two groups small; what this flag says is which book is + positioned there, not whether anything happens." +- **`pool_agrees` itself**, in `COLUMN_DEFINITIONS`: state that it is the sign of `pool_net` + compared with the price signal, that it has **no lookback dependence**, and that it has + three states, of which "nobody checked" is not "no". + +--- + +## 7. Acceptance + +- `cot-analyzer`: `pytest tests/test_damage_vocabulary.py` passes with the new fragments + added, and fails if either local copy is reintroduced. Verify by reintroducing one and + watching it fail. +- `crowdmon`: full suite, plus whatever asserts the manifest's shape. +- The published `SCHEMA_VERSION` is unchanged, checked rather than assumed. +- The `/damage` page renders against the pinned 2026-07-28 panel with no blank markers and no + market dropped, per reading instruction 3. +- A diff review confirming no number moved anywhere. + +**Order matters**: crowdmon ships first, because the consumer reads the string from the +manifest and cannot render what has not been published. A cot-analyzer PR merged first would +render an empty state. + +--- + +## 8. Outcome + +*To be appended by the executing session. Leave `Status: open` until then.* diff --git a/docs/handoffs/README.md b/docs/handoffs/README.md index b619318..4842b9a 100644 --- a/docs/handoffs/README.md +++ b/docs/handoffs/README.md @@ -50,6 +50,7 @@ A moved handoff leaves a **pointer row and no copy**. One document, one lineage. | [2026-08-03-swap-dealer-weight-decision.md](2026-08-03-swap-dealer-weight-decision.md) | **COMPLETE 2026-08-03 (PR #44), decided by the human as option (a): `swap: 0.4` stands.** Its §4 released exactly one piece of work and that work **corrected the handoff's own §1**: `single_weight_sweep` shipped, and re-swept over the order-preserving band it showed §1's 42.0% was the wrong BAND rather than wrong arithmetic. Honest figures **17.9% on the 13 Supplemental markets, 3.5% pooled** over `w_SD ∈ [0.2, 0.4]` (`§C10`). **Decided in parallel with `§C6-C8`, which landed as PR #43 while this was in flight and reached the same place by a different road**: §C6 closed option **(c)** by settling static weights independently, §C7 measured the cost of (a) as a **+19.6% median `Phi` inflation, worst on gold at +27.8% (2.30x cocoa) and precisely during stress weeks**, and §C8 showed most of that never reaches `D`, because the composite consumes a percentile and the median market-week moves only 5.9 of them. **§C8's split is the operating rule: on classic outrights the band is a footnote (Spearman 0.954, 18 of 20 top markets in common); on the ERCOT and PJM book it can invert a market's own history (Transco Zone 6 at -0.416), so publish the band there.** Two independent measurements, one decision, and no re-litigation. Option (c) is unscheduled rather than closed; metals remain permanently outside coverage | | [2026-08-05-forced-flow-mechanism-test.md](2026-08-05-forced-flow-mechanism-test.md) | **COMPLETE, executed 2026-08-06 in `npf` (PR #79). Verdict a MARGINAL LEAN, and §5.6's `supported` must not be quoted on its own.** When the trigger says a pool is forced out at `F*` and price reaches `F*`, does that pool move? The criteria pass on both report types, but **§5.3's own placebo carries 73.8% (Disaggregated) and 51.9% (TFF) of the effect** where §5.3 says it should be near zero, because the group label **is** the sign of `pool_net` and `pool_net` mean-reverts. Removing the label leaves **-0.001388 (p 0.047)** and **-0.002657 (p 0.0099)**, about 1.2% and 1.9% of the pool against headline figures of 4.5% and 4.0%; Disaggregated would not survive a correction across even the two report types. **The gap is the finding: §5.6 attaches no criterion to the placebo**, so the design as written cannot separate forced flow from positioning mean reversion, and a re-run should fix that first. Three more corrections in §8: §4's control is **not** readable from the panel (39 rows, one week, as §3 itself measures) and was recomputed over all 1,051 weeks, `pool_agrees` reducing to the SIGN of `pool_net` with **no lookback dependence**; §4's "the two groups share the price mechanics exactly" is **false as measured**, the contradicted group crossing more often in both types (25.60% vs 20.11%, 24.20% vs 21.80%); and §5.5's p-value **cannot be computed as written**, a self-centred bootstrap giving `p_literal` 0.465-0.526 across all six variants including those with `p_null` 0.0000. §3.1's equivalence reproduced exactly at 2.220446049250313e-16 over 96 comparisons. Count **8 variants, not 6**. Verdict: `npf/docs/crowdmon/2026-08-06-forced-flow-mechanism-verdict.md` | | [2026-08-06-forced-flow-respecification.md](2026-08-06-forced-flow-respecification.md) | **SPECIFICATION CORRECTION, NOT A WORK ORDER. Do not open this as a task; §7 says what would have to change first.** A fifth defect in the mechanism test above, larger than the four its §8 records: the outcome variable `Δ pool_net` **cannot answer the question it was asked**, and this package already says so. On the sell side an agreeing pool is long and sells at `F*` while a contradicted pool is short and **adds shorts** at the same level; both are sell flow, both give `Δnet < 0`, and only the first is forced. `futures/flow.py`'s opening paragraph states exactly this ("Net position change cannot tell them apart... that is the whole reason this module exists"), `flow.decompose` exists for the distinction, and the frozen §5 references neither. Advisory and untested: on the published `flow_state` of the following week the named pool does what the trigger says and the contradicted pool does the opposite, in **all four (report type, side) cells**, a **3x to 5x** separation the net erases (disaggregated sell `long_liquidation` **20.1% agreeing against 3.9% contradicted**, `new_shorts` 9.4% against 25.8%). **The obvious swap to `flow_state` is NOT the fix**: the labels are partly mechanical, since a net-long book has a bigger gross long leg and is likelier to be labelled `long_liquidation`, so the confound reappears one level down. §4 proposes the **fractional change in the exposed leg** instead, plus a within-group primary contrast, a falsification **with a criterion** (the gap that produced the original verdict), and a side-consistency condition (TFF's surviving residual is sell -0.0072 against buy **+0.0037**). §6 is a separate, actionable-now presentation consequence: `pool_agrees == False` currently **suppresses** the reading in `report.format_offside` and cot-analyzer's damage page, where the measurement says the level is live and only the flow's composition differs | +| [2026-08-06-trigger-contradicted-copy.md](2026-08-06-trigger-contradicted-copy.md) | **OPEN, unclaimed.** Released by §6 of the respecification above and the only part of that lineage actionable without a re-run. When `trigger_{side}_pool_agrees` is `False`, both renderers say the level "would force a book that is not there"; the verdict measures flow following a crossing **either way** (Disaggregated sell, within-group: **-0.0190** agreeing against **-0.0184** contradicted) with a crossing-specific difference of only **-0.0014 / -0.0027**. Copy and vocabulary only: **no data change, no threshold, no change to `D`, and the quadrant suppression STAYS** (its severe axis is `D`, a conditional severity for the *named* pool, so a contradicted row must not be given a cell). **The sharp edge is what may not be said**: the composition split that motivates it is a descriptive crosstab with a known confound, so "flow still follows and the difference is small" is supported and "it arrives as fresh shorts" is not. **The real work is structural, not three strings**: `pool on the other side` is authored locally in cot-analyzer's `damage.py` in two places and is **not** in `test_damage_vocabulary.py`'s `FRAGMENTS`, so changing crowdmon alone ships a producer and a page saying different things with every test green. crowdmon first (constant + `COLUMN_DEFINITIONS` + manifest), cot-analyzer second (read it, delete both copies, **extend the guard**). Do **not** bump `SCHEMA_VERSION`: `publish.py` says a bump ahead of a consumer release loses the whole `/damage` page | | **2026-08-05-fragility-orthogonality.md** — **MOVED to `npf/docs/handoffs/`, 2026-08-05, before execution.** Authored here and relocated the same day, which its own §7 named as conditional on `npf` growing a status register. It is **npf-owned work** (a question about npf's book) and `crowdmon` must not be read as commissioning strategy research. **No copy is kept here**, deliberately: `2026-08-03-index-share.md` records what two lineages of one file cost, and a pointer from the first day is the remedy this repo's `CLAUDE.md` states. Its cross-repo citations were qualified on the move and the change is recorded in its own §0; no threshold, statistic or criterion moved. Status is tracked **there**, not here | **MOVED** | | [2026-08-03-b-series-recovery.md](2026-08-03-b-series-recovery.md) | **COMPLETE, outcome as §7 and §8.** §1 recovered `claude/template-followups-doc-corrections-45de1d` (`11b7c81`), local-only with 2,070 insertions, by pushing it unaltered. §2 landed its content on `main` by **porting rather than merging** (the branch predates six merges and would have reverted `flow.py`, `check_skips.py`, `conftest.py` and the whole 08-03 lineage), so **that branch should be CLOSED unmerged**. `amendments-2026-08-02.md` is now B1-**B37**: it never closed at B32, it closed on a branch nobody could see. **§C4 is superseded**, having read `A_agnostic` as WEIGHT-agnostic where `§B34` defines it as DIRECTION-agnostic at a median of **3.0237**; corrected in place with both readings pinned. The cocoa split point is **resolved**: same date, 2025-10-21, differing only on which half that week falls in, and every other figure identical under both rules. §3 changed the citation convention to **path plus reproducer** and made the bare form fail loudly (`tests/test_references.py` over 368 references in 42 files; one unresolved at the time, `§C5`, recorded rather than deleted and then forced out by its own guard when #42 merged). §4 ran the band `w_SD ∈ {0.067, 0.2, 0.4, 0.7}` under a **settled static-weights decision** and filed `2026-08-03 §C6-C8`: `w_SD = 0.4` overstates fragile capital on 99.31% of market-weeks by a median +19.6%, **gold at +27.8% is 2.30x cocoa at +12.1% so the prediction holds**, and 0.067 moves the ceiling 10.0 to 14.925 so the band is not on one scale. Most of it never reaches the composite, which consumes `pct(Phi)`, except on **98 of 264 markets** where the two tables reorder a market's own weeks. **Weight table unchanged** |