Strip: the same positioning, measured in dollars at risk

docs/analysis/2026-08-24-contracts-against-dollars.md

@@ -0,0 +1,156 @@
+# Contracts against dollars, market by market
+
+**Point-in-time analysis, 2026-08-24. Not amended after this date** (see the doc lifecycle note
+in the workspace CLAUDE.md: `analysis/` is a record of when a thing was learned).
+
+Answers the question raised by a third-party positioning report that draws the same speculative
+position twice, once in net contracts and once in US dollar notional, each against its own
+history. The question for this app is narrower than the report's: **the Crowding Strip already
+draws a 0-100 positioning index per market, so is a dollar version of that index a different
+series, or the same one wearing bigger numbers?**
+
+It is a different series, but only on one of the two dollar rungs. **Dollar RISK parts from the
+contract reading by a median 30 index points at the 95th percentile and changes the model's own
+answer on 11.7% of weeks. Dollar NOTIONAL does not: over a rolling window it is very nearly the
+contract count again.** That split is what decided which reading the strip draws and which one it
+keeps to the hover.
+
+## Reproducer
+
+```bash
+.venv/bin/python scripts/measure_dollar_wedge.py --out docs/analysis/2026-08-24-contracts-against-dollars.json
+```
+
+Run from the `cot-analyzer` repo root with the environment `run-local.sh` sets. Nothing here is
+sampled, so there is no seed and a rerun against the same store reproduces every figure. Full
+output is in the JSON beside this file.
+
+**Data reference.** `COTDATA_STORE` at `~/code/cotdata_store`, `schema_version` 2, all four
+report families at `newest_data` 2026-08-18, legacy 53 entries / 81,870 rows.
+`MARKETDATA_STORE` at `~/code/marketdata_store`: 98 Norgate futures entries all at `last_date`
+2026-08-21, 15 equities, and a 49-row `contract_specs` table written 2026-08-22.
+`COTMETRICS_PARAMS` pointed at the private `cotmetrics-config/params.yaml`, so the universe is
+47 markets including `heldout`, of which **45 can be priced**. The two that cannot are MSCI EAFE
+and MSCI Emerging Mkts, which have no contract multiplier in the specs table: they are ICE MSCI
+futures that Norgate carries no continuous series for.
+
+## What was measured
+
+One statistic, four series, everything else held constant. For each market the reading is
+`indicators.calculate_range_index` over **that market's own tuned lookback**, which is the window
+the page draws under its default Custom setting, applied to:
+
+| series | what it is | who draws it |
+|---|---|---|
+| `raw` | Commercial net contracts | the Raw PF index |
+| `norm` | Commercial net over open interest | the NPF index |
+| `notional` | contracts x point value x price | nothing; hover only |
+| `risk` | notional x daily volatility | the strip's dollar mark |
+
+`cotmetrics.exposure` supplies the last two, on the tiers it insists on: `unadj` for price levels
+and `propadj` for the volatility factor. The leg is Commercials throughout, matching the mark the
+strip already draws. The Legacy legs sum to zero, so the speculator mirror is exact and the
+choice of leg is presentation rather than measurement.
+
+## Result 1: dollar risk is a different series, dollar notional is not
+
+Per market, over every week both readings exist (44 markets clear the 104-week minimum):
+
+| pair | correlation | p95 \|gap\|, index points | disagree on the band |
+|---|---|---|---|
+| contracts vs **risk** (Raw PF, 5/95) | **0.917** | **30.4** | **11.7%** |
+| contracts vs notional (Raw PF, 5/95) | 0.979 | 14.5 | 5.4% |
+| share of OI vs **risk** (NPF, 20/80) | 0.896 | 33.9 | **20.7%** |
+
+All three figures are medians across markets. "Disagree on the band" means the two readings do
+not land in the same one of the model's three bands (below its low gate, between, above its high
+gate), which is the only difference between two lenses that changes an answer on this page.
+
+The middle row is the useful negative. Notional is contracts times a slowly-moving price, so over
+a 26 to 52 week window it is close to a monotone transform of the contract count and a second
+mark for it would sit on top of the first. Risk multiplies by volatility, which moves on its own
+schedule and is the term no positioning index can carry.
+
+The third row is the largest of the three and is worth reading carefully: it compares two
+different NORMALIZERS as well as two units, because NPF's index is already a share of open
+interest. See Result 5 for why there is no dollar version of that share to compare against
+instead.
+
+## Result 2: it is not just "volatility is high"
+
+The obvious deflation of Result 1 is that `risk = contracts x point value x price x sigma`, so the
+gap could be nothing but a volatility chart wearing a positioning label. Measured against each
+market's own daily volatility, as a range index over the same window:
+
+- correlation of the gap with the volatility index: **0.25 at the median**,
+- and it **flips sign across markets**: 33 of 44 positive, 11 negative.
+
+The flip is the mechanism rather than noise. Volatility acts on a position that has a side, so the
+same volatility collapse widens the gap upward on a short and downward on a long. A single
+volatility overlay cannot reproduce this, because it does not know the sign of the position it is
+scaling.
+
+## Result 3: the gap is a state, not a jitter
+
+Lag-1 autocorrelation of the gap is **0.918** at the median (p10 0.75, p90 0.95). A market whose
+two lenses disagree this week disagreed last week. That is what makes it worth a mark on a weekly
+board rather than a warning that fires and clears.
+
+## Result 4: the live week, 2026-08-18
+
+Of the 45 priceable markets, **13 disagree about the band** on the Raw PF reading and 11 on the
+NPF reading. The three worth naming:
+
+| market | window | contracts | dollars at risk | what changed |
+|---|---|---|---|---|
+| **Silver** | 24w | **0** | **96** | daily volatility fell from 6.7% to 2.7% across the window, so a record short (-44,792 lots) carries -$385m against the window's -$1,235m |
+| **Natural Gas** | 70w | **99** | **9** | volatility fell from 7.9% to 2.5%; the largest contract position in the window is a fifth of the money the window has seen |
+| **Gold** | 26w | **0** | **63** | Commercials are at their most short in contracts and in notional, and mid-range in risk, because volatility sits near the bottom of its own window |
+
+Silver is the case the whole comparison was built from, and it is the one a boolean test misses:
+it sits at the bottom of the contract range and near the top of the dollar range, which is both
+ends of the axis at once. Counting "is it through a gate" scores that as agreement, which is why
+`strip_traces.band_of` returns three bands rather than two.
+
+## Result 5: a share of open interest is unit-free, exactly
+
+Across all 45 priceable markets and every week in the store, the position's share of open interest
+computed in contracts and computed in dollars at risk differ by at most **2.2e-16**.
+
+This is algebra rather than a finding (the point value, the price and the volatility all cancel
+between numerator and denominator, so the residual above is floating-point division and not a
+measurement of anything), and the same identity was frozen independently as P5 of the pre-registration
+behind `cotmetrics` 0.8.0 and is asserted in that package's own tests, from the numeraire side:
+a share does not move when the Gold switch does. That is the side worth asserting in code, because
+it is the side that can break. It did break once, in a first cut that deflated the position columns
+and not the open-interest ones, so the share carried 1/gold. The version measured here cannot break
+that way, because nothing in this script applies a numeraire.
+
+It decides something concrete: **there is no dollar
+version of the NPF basis to draw.** The dollar lens is inherently a LEVEL lens, so under NPF the
+mark compares two different normalizations, and the caption has to say the window and the unit
+rather than pretend only one thing changed.
+
+## What this is not
+
+Description, not a signal. Whether the gap predicts anything is a question for the ladder in
+`npf`, judged by someone other than whoever proposed it, and the prior is not neutral: `crowdmon`
+tested a close cousin of this (damage = crowding x illiquidity x fragility) across four
+pre-registered tests and got no positive result. Nothing here licenses trading the wedge.
+
+It is also not a claim about the printed report that prompted it. That report ranks a position
+against its ENTIRE history rather than against a rolling window, which is a different statistic;
+this app's expanding-percentile version of that question already lives on `/exposure`.
+
+## Bottom line
+
+In plain language: measuring the same positions in money instead of in contracts genuinely changes
+the picture, and it changes it most where a reader would care, at the extremes. About one week in
+nine, the two readings put a market in different bands, and this week thirteen of forty-five
+markets are in that state, with silver the extreme case: the crowd's largest short of the year in
+contracts is an ordinary-sized bet in dollars because volatility has halved. The effect is a
+persistent state rather than weekly noise, and it is not simply a volatility chart, since the same
+volatility move pushes the gap in opposite directions depending on which way the position leans.
+The dollar-notional version of the same idea, which is what the source report plots, adds almost
+nothing over a rolling window, so it is kept in the hover rather than drawn. None of this is
+evidence that the gap predicts returns; it is a better description of what the position is.

docs/design/positioning-report-view-ideas.md

@@ -880,3 +880,113 @@ the week it matters.
 question for the ladder in npf, judged by someone other than whoever proposed it. The
 prior is not neutral: `crowdmon` tested a close cousin, damage = crowding x illiquidity x
 fragility, across four pre-registered tests and got no positive result.
+
+
+## After the eleventh review: contracts against dollars, on the whole board
+
+The tenth review put the contracts-against-dollars wedge on `/exposure`, for one market at a
+time. This puts the same comparison on the strip, where it is forty-odd markets at once, and the
+measurements behind it are in
+[`docs/analysis/2026-08-24-contracts-against-dollars.md`](../analysis/2026-08-24-contracts-against-dollars.md)
+with a reproducer script beside it.
+
+The prompt was another printed report, this time gold and silver drawn twice each: net position
+in contracts, then the same position in US dollar notional, each with a percentile against its
+entire history. Its own numbers are the argument for looking. Silver's speculators were at the
+52nd percentile in contracts and the 97th in dollars in the same week.
+
+**What the strip draws is the risk rung, not the notional one, and that is measured rather than
+preferred.** Holding the market, the leg, the window and the statistic constant and changing only
+the unit: dollar risk correlates 0.917 with the contract reading at the median, parts from it by
+30 index points at the 95th percentile, and lands in a different one of the model's three bands
+on 11.7% of weeks. Dollar notional scores 0.979, 14.5 points and 5.4% on the same test. Over a
+rolling window notional is contracts times a slowly-moving price, so it is close to a monotone
+transform of the count, and a second mark for it would sit on top of the first on most rows. It
+is carried in the hover instead, one line, because it is the reading the printed reports plot and
+a reader comparing the two boards should be able to see both numbers.
+
+Worth being explicit that this is not a rebuttal of the source. The report ranks against the
+ENTIRE history, where notional carries the price level and therefore says a great deal; the strip
+ranks against a rolling window, which renormalizes exactly that away. The all-history version of
+the question already has a home on `/exposure`, and its expanding percentile is what answers it.
+
+**A range index, not a percentile, and the difference is the whole reason the two marks
+can share an axis.** The dollar mark is `calculate_range_index` on the dollar series: position
+between the window's min and max, which is exactly what the lollipop beside it measures on
+contracts. A percentile rank over the same window is a different statistic, set by the whole
+distribution rather than by two observations, and drawing one against the other would vary the
+STATISTIC and the UNIT at once, leaving a reader unable to say which difference they were looking
+at. This document already warns about that confusion in the other direction ("Whichever we draw,
+label it as what it is") and the page had to learn it again the hard way: the first reader of the
+new mark asked whether the diamond was a percentile, so the word "index" now appears on the hover,
+in the legend key and in the caption rather than only here. The percentile version of this question
+is the `/exposure` page's, where the rank is expanding or windowed and the units are dollars
+throughout.
+
+**One reference mark per row, chosen by a control, rather than a fifth mark.** A row already
+carries the head, its stem and a tick per gated leg. The two comparisons worth putting beside them
+are the same shape of question, "this leg measured differently", and they compete for the same few
+pixels: where the index stood six weeks ago (differently in TIME) and where it sits in dollars at
+risk (differently in UNIT). Drawing both was tried first. At ROW_PX a hollow ring and a hollow
+diamond a few points apart are one smudge, and on a quiet row they are the same colour as well, so
+the row stops saying which is which. The COMPARE control picks one, and a third setting turns both
+off, which the page could not do before.
+
+**The dollar mark takes the row's colour and a different shape, because the palette has no free
+slot.** Every slot is spoken for twice over: `grid_colors` builds the bull verdict from slot 3,
+which is also Price, and the bear verdict from slot 0, which is also Commercials. So a
+"this is the dollar reading" colour would collide with a verdict on the one figure whose whole
+colour vocabulary is verdicts. Shape carries which comparison it is; colour goes on saying what it
+says everywhere else on the row.
+
+**The connector stays, which is the opposite call from the prior mark.** That mark deliberately has
+no line back to the head, on the argument that 42 connectors are a lot of ink for a move of a few
+points. Here the GAP is the subject rather than a by-product, and the ink is self-limiting in a way
+the momentum version's would not be: a market where money and contracts agree draws its diamond
+around its own head and a line of zero length, so the rows that spend ink are exactly the rows
+worth reading. Where the dollar reading falls between neutral and the head the connector lies along
+the stem and is invisible, which is the same information (the diamond is ON the stem, and the
+distance to the head is the gap) at no cost.
+
+**Disagreement is counted on the bands, and on three of them.** The caption says how many drawn
+markets the two lenses put in different bands, not how many differ by some number of points:
+contracts at 98 against dollars at 90 is a wide gap the model answers the same way twice, while 96
+against 94 under NPF straddles the line. Three bands rather than a boolean "through a gate",
+because Silver on 2026-08-18 sat at 0 on contracts and 96 on dollars, which is both ends of the
+axis at once, and a boolean scores the sharpest disagreement on the board as agreement.
+
+**Two counts, for the reason the caption already counts hidden and index-less markets.** A row with
+no diamond and a row whose two marks coincide look identical, and only one of them is a
+measurement. MSCI EAFE and MSCI Emerging Mkts have no contract multiplier, so they can never carry
+one; they are `heldout` today and therefore off the live board, which is why the sentence usually
+does not appear.
+
+**Off by default.** It is the only thing this page draws that needs the price store, so it is the
+only thing on it that can fail for reasons that have nothing to do with COT. The join costs about
+four seconds cold for the whole universe and about a third of a second warm, since `exposure`
+caches the daily price and volatility series per symbol and the strip caches the per-market
+readings keyed by the store's newest date, exactly as the Heatmap's two joined columns do.
+
+**Where the arithmetic lives.** Both halves are cotmetrics functions (`exposure.market_exposure`
+for the dollar series, `indicators.calculate_range_index` for the index) and this repo only
+composes them, the same shape as the Heatmap's `_spec_risk`, which pairs `market_exposure` with
+`expanding_pct_rank`. The composition is a range index of a dollar series and nothing else in the
+app wants one yet. The moment a second surface does, it moves to `cotmetrics.exposure` beside
+`windowed_pct_rank` rather than being copied.
+
+**One thing that fell out of the arithmetic and constrains what can ever be drawn here.** A
+position's share of open interest is the same number in contracts and in dollars, to 2.2e-16
+across the universe, because the point value, the price and the volatility cancel between the
+position and the market it sits in. The same identity was frozen independently as P5 of the
+pre-registration behind `cotmetrics` 0.8.0, where it is asserted from the numeraire side (a share
+must not move when the Gold switch does), which is the side that can actually break and did once.
+So there is no dollar version of the NPF basis: the dollar lens is inherently a LEVEL lens, and
+under NPF the two marks differ in normalizer as well as in unit. That is the one place this comparison is not a controlled one, it is stated in the caption
+rather than hidden, and it is also why the NPF disagreement rate (20.7% of weeks) is the largest
+of the three measured.
+
+Still not built, from the list this document opened with: the window min/max/current tooltip, time
+in state, and the horizon ladder. The percentile tick from the whiskers section is now cheap,
+since `cotmetrics.exposure` grew `windowed_pct_rank` and `expanding_pct_rank` for the exposure
+page, but it is a second statistic on one axis and would need the same care about labelling that
+this comparison needed about units.