Skip to content

feat(validation): deflated_expectancy, the number the monitor should anchor to - #116

Merged
mspinola merged 1 commit into
mainfrom
claude/deflated-expectancy
Aug 3, 2026
Merged

feat(validation): deflated_expectancy, the number the monitor should anchor to#116
mspinola merged 1 commit into
mainfrom
claude/deflated-expectancy

Conversation

@mspinola

@mspinola mspinola commented Aug 3, 2026

Copy link
Copy Markdown
Owner

docs/edge_monitor.md carried this at the top of its open list since the monitor shipped: EdgeBaseline took a deflated_expectancy float and nothing in the package produced one. deflated_sharpe corrects a Sharpe and returns a probability, which is the right output for a gate and useless to a monitor. A monitor needs a number in R, and without one the only anchor available was the sample mean, which is the number the parameters were optimized on.

The conversion

It rides on the bar deflated_sharpe already uses. SR0 is the expected maximum per-trade Sharpe of N noise trials; carry it back into R by the winner's own sigma and subtract:

deflated = mu - sigma * SR0

Both functions now call one _expected_max_sharpe, so two corrections for one search cannot disagree about how big the search was.

Three choices worth recording

  • It takes trial LOGS, not trial Sharpes, unlike deflated_sharpe. The Sharpes are computed inside so their clock cannot be got wrong: multiplying a per-month Sharpe by a per-trade sigma gives a haircut in no units at all, silently, which is the v0.4.0 units bug in a new costume.
  • It is a bias correction, NOT a significance test, and the docstring, the __str__ and the property name all say so. It removes the selection bias a search of this size is expected to produce, so a pure-noise winner still clears zero roughly half the time: measured 56% / 47% / 44% for N = 5 / 20 / 100, against deflated_sharpe correctly calling 0% of the same draws significant. I named the property survives first, which reads as a verdict it does not deliver; it is is_positive, and the measurement that caught this is pinned as its reproducer.
  • EdgeBaseline.from_log accepts the result object as well as a float, because handing over the wrong field of a result you already computed anchors the monitor to the pre-correction number while reporting deflated=True.

Two findings from wiring the example to the real function

  • The worked example faked this step with raw_mean * 0.8 under a comment beginning "Pretend". With a real 64-config search the haircut is 26% of the raw edge rather than 20%, and the naive baseline flatters by 36% rather than 25%. Every figure in tutorial §14 moved and is re-pinned.
  • tests/test_edge_monitor_example.py rebuilt the baseline it exists to pin instead of importing it, so all five assertions passed unchanged while the example they guard printed entirely different numbers. It now imports promoted_book(). A guard that reconstructs what it guards is not a guard.

The payoff, as a test rather than a claim

Run one untouched, fully healthy live book against both baselines: the naive one returns DEGRADED where the deflated one returns HOLDING. An inflated reference does not merely mis-scale the ratios, it manufactures alarms on books that never decayed.

373 tests pass, ruff clean, mkdocs build --strict clean.

🤖 Generated with Claude Code

…anchor to

docs/edge_monitor.md carried this at the top of its open list since the monitor
shipped: EdgeBaseline took a deflated_expectancy float and nothing in the package
produced one. deflated_sharpe corrects a Sharpe and returns a PROBABILITY, which is
the right output for a gate and useless to a monitor. A monitor needs a number in R,
and without one the only anchor available was the sample mean, which is the number
the parameters were optimized on.

The conversion rides on the bar deflated_sharpe already uses. SR0 is the expected
maximum per-trade Sharpe of N noise trials; carry it back into R by the winner's own
sigma and subtract:

    deflated = mu - sigma * SR0

Both functions now call one _expected_max_sharpe, so two corrections for one search
cannot disagree about how big the search was.

Three choices worth recording:

  * It takes trial LOGS, not trial Sharpes, unlike deflated_sharpe. The Sharpes are
    computed inside so their clock cannot be got wrong: multiplying a per-month
    Sharpe by a per-trade sigma gives a haircut in no units at all, silently, which
    is the v0.4.0 units bug in a new costume.
  * It is a bias correction, NOT a significance test, and the docstring, the __str__
    and the property name all say so. It removes the selection bias a search of this
    size is EXPECTED to produce, so a pure-noise winner still clears zero roughly
    half the time: measured 56% / 47% / 44% for N = 5 / 20 / 100, against
    deflated_sharpe correctly calling 0% of the same draws significant. I named the
    property `survives` first, which reads as a verdict it does not deliver; it is
    `is_positive`, and the measurement that caught this is pinned as its reproducer.
  * EdgeBaseline.from_log accepts the result OBJECT as well as a float, because
    handing over the wrong field of a result you already computed anchors the monitor
    to the pre-correction number while reporting deflated=True.

Two findings from wiring the example to the real function rather than a stand-in:

  * The worked example faked this step with `raw_mean * 0.8` under a comment starting
    "Pretend". With a real 64-config search the haircut is 26% of the raw edge rather
    than 20%, and the naive baseline flatters by 36% rather than 25%. Every figure in
    tutorial §14 moved and is re-pinned.
  * tests/test_edge_monitor_example.py REBUILT the baseline it exists to pin instead
    of importing it, so all five assertions passed unchanged while the example they
    guard printed entirely different numbers. It now imports promoted_book(). A guard
    that reconstructs what it guards is not a guard.

The payoff is now a test rather than a claim: run one untouched, fully healthy live
book against both baselines and the naive one returns DEGRADED where the deflated one
returns HOLDING. An inflated reference does not merely mis-scale the ratios, it
manufactures alarms on books that never decayed.

373 tests pass, ruff clean, mkdocs --strict clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mspinola

mspinola commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

Rebased onto #115, which landed while this was open. The two overlap more than the titles suggest and both changes are kept:

#115 moved the ARL budget to calendar years, so every figure in the worked example and tutorial §14 moved again on top of the ones this PR already moved. Recomputed with both applied and re-pinned: h 43.17 → 35.09 sigma, ARL0 3,752 trades (25.0 yr), ARL1 658 trades (4.4 yr), and the halved book now alarms at live trade 276 rather than 1,138. The measured in-control inflation is 1.09x and the shifted figure 0.96x, which I updated rather than keeping #115's 1.01x.

#115 also fixed the dead firing-rate channel from crucible's end, by falling back to exit_date in _trades_per_year. That is the same defect I fixed from the seam end earlier today in crucible-stack#17 and npf#184 (the seam built its TradeLog from bare floats, so no date column existed to read at all). The two fixes are complementary, not redundant: #115 makes the baseline side work for exit-dated logs, and the seam work makes the live side work at all. entry_date stays preferred, so npf continues to supply it.

The headline result is unchanged by the rebase: run one untouched, fully healthy live book against both baselines and the naive one still returns DEGRADED where the deflated one returns HOLDING.

378 tests pass, ruff clean, mkdocs build --strict clean.

@mspinola
mspinola force-pushed the claude/deflated-expectancy branch from 15413f3 to 19814cb Compare August 3, 2026 02:01
@mspinola
mspinola merged commit d80dbf0 into main Aug 3, 2026
3 checks passed
@mspinola
mspinola deleted the claude/deflated-expectancy branch August 3, 2026 02:04
mspinola added a commit that referenced this pull request Aug 3, 2026
Bumps pyproject and crucible.__version__ to 0.6.0 and moves [Unreleased] into a
dated [0.6.0] section.

The second release today, deliberate rather than sloppy. 0.5.0 shipped the edge
monitor; running it against a real 47-market book rather than the synthetic
fixtures it was built on immediately exposed three calibration problems and
confirmed the one gap already known. Leaving those in a published version for
weeks is worse than two tags in one day.

In order of how much it matters to anyone already on 0.5.0:

1. deflated_expectancy exists (#116). 0.5.0's EdgeBaseline accepted a deflated
   number and nothing in the package produced one, so every real baseline was the
   raw in-sample mean and the monitor's central argument was a docstring.
2. The false-alarm budget is denominated in years, not trades (#115). Any design
   built under 0.5.0 against a baseline that knows its firing rate recalibrates
   here, and for a slow book substantially: the old default gave a 23-trades/yr
   book a nominal detection latency of 24 years.
3. The opportunity-set channel works for logs without entry_date (#115), where
   0.5.0 silently switched it off for an ordinary class of books.

Minor rather than patch on the merits: 2 changes behaviour and 1 adds public API.

378 tests pass, ruff clean, mkdocs --strict clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant