Skip to content

Market regimes computed by the terminal: MICRO, IMMINENT and MACRO - #20

Merged
wJubran merged 2 commits into
mainfrom
claude/sharp-wozniak-49jry7
Oct 17, 2026
Merged

wJubran merged 2 commits into
mainfrom
claude/sharp-wozniak-49jry7

Conversation

@wJubran

@wJubran wJubran commented Oct 17, 2026

Copy link
Copy Markdown
Owner

What and why

The REGIMES strip above the chart read "— no data" because nothing in the repository produced its files. The terminal now computes them itself, from the active data source and public FRED data, in the exact format the UI reads. I worked that format out from RegimeBar.jsx, MainChart.jsx and ReplayView.jsx, since no schema document exists.

MICRO: {TICKER}_regime.json (local_analysis/regimes.py, a new regime phase in the runner)

  • Model: a Gaussian HMM on each day's return and the log of the 5-day realized volatility, from the feed's daily bars. It has 2–4 states chosen by BIC, ordered from calm to stressed.
  • Walk-forward, no look-ahead: the model is refitted monthly on up to 1,000 days, warm-started from the previous fit. Each day is labelled with the filtered probabilities of a model fitted before it. A test checks that truncating the series leaves all earlier labels unchanged.
  • History rows: history.slow_d1 has one row per completed day, dated by the bar's own day. The forming bar is excluded, and there are no duplicates and no NaN.
  • Today's pill: slow.regime is the regime expected for today, from the last completed close.
  • Cost: the daily part is rebuilt once per trading day. Otherwise it is reused from the file, so a restart costs nothing.

⚠ IMMINENT: fast.imminent / fast.minutes_since_alarm

  • Inputs: 15-minute returns divided by their usual size at that time of day. The scale is robust (median absolute return), so neither the session clock nor a burst itself raises the "usual".
  • Model: a two-state normal/burst HMM forecasts the volatility of the next 2 hours. The alarm fires at ≥ 1.5× the usual for those hours.
  • Design note: a first version used "chance of reaching the active state within 2 h". On test data it was on 100% of the time, so I replaced it with the volatility forecast.
    • On synthetic data with four news bursts and a busy New York afternoon, the alarm fires on every burst and nowhere else.
    • With the session pattern alone, it never fires.
  • Refresh: every new 15-minute bar (about 1–2 s per ticker).

MACRO: GLOBAL_macro_regime.json (+ GLOBAL_regime_transitions.json)

  • Inputs (FRED, free CSV download, no key): growth from the S&P 500 (SP500); inflation from the 10-year breakeven (T10YIE) and WTI crude (DCOILWTICO).
  • Scaling: each input's 3-month change is divided by its trailing 3-year RMS. It is not demeaned, so a long trend keeps its sign. A first version used trailing z-scores and mislabelled long trends; a test caught it.
  • Model: a four-state HMM pinned to the corners of GOLDILOCKS / REFLATION / STAGFLATION / DEFLATION smooths the readings.
  • History: one row per calendar day, so weekend bars are shaded too.
  • FRED client (local_analysis/fred.py): it caches each series and refreshes it every 6 hours. When FRED is unreachable it keeps the last copy, and retries a failed series after 30 minutes.

Docs

  • Learning Center article Market regimes (HMM) in English and Arabic.
  • A How the regimes are computed section in README.md and README.ar.md, plus the Arabic part of the main README.
  • CHANGELOG entry.
  • The mean-reversion setup now refers to the strip's actual states.

How it was tested

  • python -m pytest passes in backend/: 199 tests on Python 3.13 and 3.10, with 15 new or updated regime tests. The new tests cover:

    • FRED parsing, caching and outage behaviour;
    • BIC selection and calm → stressed ordering;
    • walk-forward with no look-ahead;
    • the forecast ratio;
    • the IMMINENT alarm on news bursts versus the session clock;
    • macro quadrants from four synthetic phases (>80% correct after the 3-month window);
    • another producer's files left alone;
    • /api/regime/{T} served end to end on the simulator, dated by the bars' own days and NaN-free.

    No network calls.

  • UI changes: ran npm run build in frontend/ and committed the rebuilt frontend_build/. A second build was identical.

  • Tried it with python launcher.py --sim in Chromium:

    • The strip shows CURRENT MACRO GOLDILOCKS · 98% and EURUSD MICRO CALM · 81% with the QUIET chip. The macro file was built from synthetic FRED-format data, because FRED is blocked here.
    • MICRO shading on EURUSD D1 turns red through the simulated selloff.
    • MACRO shading renders.
    • No page errors.
  • Not done: MACRO was not run against live FRED (fred.stlouisfed.org is blocked in this environment). The CSV format it parses is FRED's documented fredgraph.csv (observation_date,<ID>, . for missing values).

Screenshots

Taken locally, not attached:

  • EURUSD D1 with MICRO shading (calm/normal in green and teal, stressed in red during the selloff), and the strip pills.
  • The same chart with MACRO shading.

Generated by Claude Code

- fred.py: daily FRED series through the public CSV download (no API key),
  cached per series, refreshed every 6 hours; the last copy is used while
  FRED can't be reached, and a failed series is retried after 30 minutes.
- regimes.py: per-asset HMM regimes (log 5-day realized volatility and the
  day's return, 2–4 states by BIC, ordered calm → stressed), labelled
  walk-forward with monthly warm-started refits so no day uses later data;
  15-minute features with the time-of-day pattern removed and a switching
  hazard; macro quadrants (GOLDILOCKS / REFLATION / STAGFLATION / DEFLATION)
  from a four-state HMM pinned to growth/inflation corners.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01649i4SjUGvhXrPV3AiNhHN
The REGIMES strip and its chart shading now work without a data server:

- MICRO ({T}_regime.json): each instrument's volatility regime, calm to
  stressed, from an HMM on its daily bars (2–4 states by BIC), refitted
  walk-forward every month so every day is labelled with a model fitted
  before it. Rows are dated by each completed bar's own day; the pill shows
  the regime expected for today.
- ⚠ IMMINENT: 15-minute returns over their usual size at that time of day
  (robust median scale, so neither the session clock nor a burst itself
  sets the "usual"), a two-state normal/burst HMM, and an alarm when the
  forecast volatility of the next two hours is ≥ 1.5× the usual.
  Refreshed with every new 15-minute bar; the daily part is reused.
- MACRO (GLOBAL_macro_regime.json, GLOBAL_regime_transitions.json): the
  GOLDILOCKS / REFLATION / STAGFLATION / DEFLATION quadrant from FRED
  (S&P 500, 10-year breakeven, WTI), each 3-month change over its trailing
  RMS so a long trend keeps its sign, smoothed by a four-state HMM pinned
  to the corners; one row per calendar day.
- runner v3: a "regime" phase between live and replay, and a macro pass.
- Learning Center article "Market regimes (HMM)" (EN/AR), README sections
  (EN/AR), CHANGELOG; the mean-reversion setup uses the strip's states.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01649i4SjUGvhXrPV3AiNhHN
@wJubran
wJubran merged commit ed60182 into main Oct 17, 2026
6 checks passed
wJubran added a commit that referenced this pull request Oct 18, 2026
Market regimes computed by the terminal: MICRO, IMMINENT and MACRO
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants