Skip to content

Probability bands and the Replay page's files, computed locally - #18

Merged
wJubran merged 3 commits into
mainfrom
claude/sharp-wozniak-49jry7
Oct 17, 2026
Merged

wJubran merged 3 commits into
mainfrom
claude/sharp-wozniak-49jry7

Conversation

@wJubran

@wJubran wJubran commented Oct 17, 2026

Copy link
Copy Markdown
Owner

What and why

Several features read files that only the old data server could produce, so they stayed empty in the open-source edition:

  • the chart's BANDS overlay;
  • the bands family in the Pulse Room;
  • the whole REPLAY page.

backend/local_analysis/ now writes those files from the active feed's daily bars. This covers MetaTrader 5 and the simulator, and any future data source.

Bands (local_analysis/bands.py → {T}_bands.json)

  • Anchor: each trading day's GBM / MJD / Bates forecast starts at that day's open, so the envelope follows the price.
  • Width: taken from the last 60 trading days. The tails keep the cones' ~250-day shape: MJD's jumps, and Bates' long-run variance, vol of vol and correlation.
  • Points: one history point per day, half a trading day after the open, plus a 20-day forward fan from the newest open. mean is the shared centre: every model has a zero expected log return, so it equals the anchor price.
  • Format: the exact keys and formats the chart, the Pulse Room and the Replay read:
    • timestamps as "YYYY-MM-DD HH:MM" on the feed's clock;
    • bat_* for Bates;
    • _last_modified for the stale marker.

Replay page

  • {T}_bands_replay.json (bands_replay_v2) holds every daily anchor's forward fan and its select point.
  • {T}_historical_cones.json holds the past weekly and monthly cones.
  • Quarterly cones now cover eight quarters and carry meta.anchors; the chart still reads the first four.
  • Reuse: past anchors never change, so a daily rebuild reuses them. The first build of the replay files takes about 15 s per ticker in the background; after that it takes about 0.25 s.

Runner

  • Each pass first writes the live files for every ticker (cones, quarterly, bands), so the overlays appear within seconds. It then writes the replay files.
  • When the live files change, it rebuilds the Pulse Room snapshot.
  • JSON is written with allow_nan=False.

Models

  • Jump detection: jumps are now flagged relative to an EWMA of the volatility of their time. Before, one threshold covered the whole year. The new bands tests caught the problem: after a lasting rise in volatility, the new regime's ordinary days were all treated as jumps, and the recent volatility was underestimated. This also improves the MJD and Bates cones.
  • MJD speed: quantiles are solved for all horizons at once. That is 5.6× faster, and results match the old code to 1e-15.

Also on this branch: two building blocks for the next PRs (regimes and options), with their own tests. Nothing uses them yet.

  • a Gaussian HMM (local_analysis/hmm.py);
  • option-chain arithmetic for GEX, the gamma flip, walls and max pain (local_analysis/options_math.py).

Docs

  • The READMEs (EN/AR) and the Learning Center describe the bands and the Replay files.
  • The README claimed that Money Flow fetches live data. It doesn't: the page reads cache files like the other panels. That claim is removed.
  • The changelog is updated.

How it was tested

  • python -m pytest passes in backend/: 164 tests on Python 3.10 and 3.13. New tests (tests/test_local_bands.py) cover:
    • the shape and ordering of the lines;
    • √τ width scaling;
    • recent-volatility estimation, and jumps judged against the volatility of their time;
    • no look-ahead;
    • the replay file matching the live history;
    • reuse of past anchors;
    • eight quarterly anchors;
    • the runner's files and the rule against overwriting another producer's files;
    • the routes (/api/bands, /api/historical_cones, /api/bands_replay, /api/quarterly_cones);
    • a Pulse Room snapshot with the bands family.
  • UI changes: the Learning Center text only. Rebuilt with CI=true npm run build; frontend_build/ matches the source build.
  • Tried it with python launcher.py --sim:
    • Chart: BANDS on an AUDUSD M15 chart, with the envelope following the price and a widening forward fan.
    • Replay: a 6-month XAUUSD replay on H4 with weekly cones and bands, 52 anchors loaded.

Screenshots

Checked in a local browser run; the screenshots are not attached here.


Generated by Claude Code

Building blocks for the locally computed regimes and options levels. Nothing
uses them yet.

- hmm.py: a diagonal-Gaussian hidden Markov model in numpy. Baum-Welch runs in
  log space from deterministic restarts; the module adds filtered (past-only)
  and smoothed probabilities, Viterbi, state reordering, expected durations and
  the stationary distribution. Displayed regimes will use the filtered
  probabilities, so a day is never labelled with hindsight.
- options_math.py:
  - Black-Scholes gamma;
  - dealer GEX per contract and per strike ($ per 1% move: calls +, puts -);
  - the net-GEX curve and the gamma flip;
  - call/put walls and max pain.
  Contracts without open interest or a sane implied vol carry no gamma.

Tests: parameter recovery on a synthetic two-regime series, no look-ahead in
the filtered probabilities, determinism; gamma against the textbook value and
finite differences, GEX signs and units, the flip's position, walls, and max
pain against brute force.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01649i4SjUGvhXrPV3AiNhHN
The log-space recursion cost ~30 ms per EM iteration on 1,000 daily
observations; rescaled emissions with per-step normalisers do the same work
in ~7 ms, which makes walk-forward refits affordable. fit() can now
warm-start from a previous model, and n_params()/bic() support choosing the
number of states.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01649i4SjUGvhXrPV3AiNhHN
The chart's BANDS overlay, the Pulse Room's bands family and the REPLAY page
read files the open-source edition could not produce. backend/local_analysis/
now writes them from the active feed's daily bars.

Bands (local_analysis/bands.py, {T}_bands.json):
- Each trading day's GBM / MJD / Bates forecast starts at that day's open, so
  the envelope follows price. The width comes from the last 60 trading days;
  the tails keep the cones' ~250-day shape (jumps; Bates long-run variance,
  vol of vol and correlation).
- One history point per day, half a trading day after the open, plus a 20-day
  forward fan from the newest open. `mean` is the shared centre: the anchor
  price, since every model has a zero expected log return.
- Keys and formats are the ones the chart, Pulse Room and Replay read:
  timestamps "YYYY-MM-DD HH:MM" on the feed's clock, bat_* for Bates, and
  _last_modified for the stale marker.

Replay page:
- {T}_bands_replay.json (bands_replay_v2) holds every daily anchor's forward
  fan and its select point.
- {T}_historical_cones.json holds the past weekly and monthly cones.
- Quarterly cones now cover eight quarters and carry meta.anchors; the chart
  still reads four.
- Past anchors never change, so a daily rebuild reuses them. The Replay page's
  archives use 10,000 Bates paths and half-day steps.

Runner:
- Each pass first writes the live files (cones, quarterly, bands) for every
  ticker, then the replay files, and rebuilds the Pulse Room snapshot when the
  live files change.
- JSON is written with allow_nan=False.
- ENGINE_VERSION 2, DAILY_BARS 1200.

Models:
- Jumps are now flagged relative to an EWMA of the volatility of their time
  (capped updates), not one robust threshold for the whole window. Before,
  after a lasting rise in volatility, the new regime's ordinary days were all
  treated as jumps, and the recent volatility was underestimated.
- MJD quantiles are solved for all horizons at once: 5.6x faster, with
  results identical to 1e-15.

Docs: READMEs (EN/AR) and the Learning Center describe the bands and the
Replay files. The README's claim that Money Flow fetches live data is removed:
the page reads cache files. The changelog is updated.

Tests: shape and ordering, √τ width scaling, recent volatility, no
look-ahead, replay and live consistency, reuse of past anchors, eight
quarters, the runner's files and producer rule, the routes, and a Pulse Room
snapshot with the bands family. 164 pass on Python 3.10 and 3.13. Checked
visually: bands on an AUDUSD M15 chart, and a 6-month XAUUSD replay with
weekly cones and bands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01649i4SjUGvhXrPV3AiNhHN
@wJubran
wJubran merged commit 73c097e into main Oct 17, 2026
6 checks passed
wJubran added a commit that referenced this pull request Oct 18, 2026
The chart's BANDS overlay, the Pulse Room's bands family and the REPLAY page now work without a data server.

backend/local_analysis writes the following from the active feed's daily bars:
- rolling GBM/MJD/Bates bands, re-anchored at each trading day's open, with width from the last 60 trading days;
- every day's forward fan;
- past weekly and monthly cones;
- eight quarterly anchors.

Jumps are now detected relative to the volatility of their time. The branch also adds a Gaussian HMM and option-chain arithmetic for the next PRs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants