Fix/dual noise units and compose cholesky - #16
Open
0xgny wants to merge 2 commits into
Open
Conversation
…sition
Two dimensional-analysis bugs in the pure-numpy core, each with regression
tests that fail on the parent commit.
1. state/noise.py — λ_η was a count, not a rate
calibrate() computed `lambda_eta = N_jumps * dt`, the reciprocal of an
intensity. Three jumps planted in one day of 5-min bars reported 0.038
instead of ~3, and the value did not change when the sample got longer.
σ_τ had the matching problem: `sqrt(BPV/n)` is per-BAR volatility, so
Theorem III.1 was adding a per-bar variance to a per-day jump term.
Both are now per unit time on the clock dt is expressed in:
T = n·dt, σ_τ² = BPV/T, λ_η = N_jumps/T
which makes cramer_rao_bound(h) invariant to sampling frequency, as a
statement about a process should be.
The only existing assertion on lambda_eta was `>= 0`, which a
dimensionally inverted quantity satisfies, so the suite stayed green.
demo/run_egamec.py passed dt=1/252 for DAILY bars (declaring each bar to
be 1/252 of a day); corrected to dt=1.0.
2. events/operators.py — compose() squared the Cholesky factor wrongly
Sigma_w is documented as a Cholesky factor and read as `S @ S.T`
everywhere else in the codebase, but compose() used `S @ S`. These agree
only for diagonal S.
systemic_crisis_operator is the one constructor with off-diagonal noise,
and nothing composed it. For that operator `S @ S` is not PSD, so the
np.linalg.cholesky in compose() raised LinAlgError: a systemic crisis
could not be composed with any other event at all.
Tests: 133 -> 144. New tests pin semantics (λ is a rate; covariance obeys
L1 L1ᵀ + A1 L2 L2ᵀ A1ᵀ and stays associative) rather than magic numbers, and
each suite keeps a case covering the previously-correct diagonal path to
guard against over-correction.
Not addressed here, filed separately: demo/data/memory2026/*.csv are
referenced by SOURCES.md and hindcast_memory_2026.py but never committed, so
that demo raises FileNotFoundError on a fresh clone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix for issue:#15
What problem this addresses
Two dimensional-analysis bugs in the pure-numpy core. Both were invisible to
the test suite: the only assertion on λ_η was
>= 0, which a dimensionallyinverted quantity satisfies.
1.
state/noise.py— λ_η was a count, not a rate.calibrate()computedlambda_eta = N_jumps * dt, the reciprocal of anintensity. σ_τ had the matching problem:
sqrt(BPV/n)is per-BAR volatility.2.
events/operators.py—compose()squared the Cholesky factor wrongly.Sigma_wis documented as a Cholesky factor and read asS @ S.Teverywhereelse, but
compose()usedS @ S. Forsystemic_crisis_operator— the oneconstructor with off-diagonal noise —
S @ Sis not PSD, sonp.linalg.choleskyraised
LinAlgError. A systemic crisis could not be composed with any otherevent at all.
Connection to the theoretical framework
Theorem III.1 ADDS σ_τ² and λ_η·m₂^η, so both must share a time unit. They
didn't, which made the bound depend on your bar size rather than on the market.
Now: T = n·dt, σ_τ² = BPV/T, λ_η = N_jumps/T.
Fix 2 restores the composition law in §IV: Cov = L₁L₁ᵀ + A₁(L₂L₂ᵀ)A₁ᵀ, which
is what makes the monoid/groupoid claim in the module docstring hold.
Results
Ground truth: 3 jumps/day, 2% daily vol, 10 days, two sampling rates.
compose(rate_change, systemic_crisis):LinAlgError→ works.Replication:
python -m pytest tests/test_noise.py tests/test_events.py -vTesting
Tests 133 → 144. Each new test was verified to FAIL on the parent commit:
5/6 new noise tests, 4/5 new event tests. The ones that pass on old code are
deliberate — each suite keeps a diagonal-noise case guarding against
over-correction.
Tests pin semantics, not magic numbers: λ is a rate (held jump count fixed
while varying elapsed time — the comparison a count-shaped estimator can't
fake), and covariance stays associative under composition.
Known limitation, not addressed here
λ_η units are now correct, but the Lee–Mykland detector is underpowered on
daily bars (needs ~3.5σ moves).
demo/run_egamec.pyreports 0.001 jumps/dayagainst a synthetic truth of 0.01–0.10. Separate problem; flagging it rather
than papering over it.
One downstream effect
notebooks/day03_dual_noise.ipynbprints σ_τ against a hardcoded "true"string. The estimate is now correct (0.190 vs true annual vol 0.20 — that
notebook's
dtis in years), but the hardcoded comparison is a per-bar numbermislabeled "/day", so the line now visibly disagrees. Two-line notebook fix;
happy to include it here or split it out — reviewer's call.
Questions for reviewers
calibrate()take an explicit unit (per="day") rather thaninferring from
dt? The current contract is "whatever unit dt is in,"which is what the notebook tripped on.
1e-9Cholesky jitter incompose()the right regularizer, orshould near-singular Σ raise instead of being silently nudged?
PS: More details in changes.md