diff --git a/README.md b/README.md index 9033e03..2c4497b 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ [](https://www.instagram.com/mr.abstractor_ust/) [](https://www.linkedin.com/in/hongjinhe-hkust-edu) -**[English](README.md) | [中文文档](README_CN.md)** +**[English](README.md) | [中文文档](README_CN.md) | [日本語](docs/i18n/README_JA.md) | [한국어](docs/i18n/README_KO.md)** **Alpha Flow Research · HongJin HE · HKUST / Stanford IHP · July 2026** @@ -89,31 +89,37 @@ python demo/global_demo.py # reproduce this GIF end-to-end (~2 min, CPU, no *— Alpha Flow Research · a Stanford basement · July 2026* +*The longer version — how a world-model skeptic was converted by robots that dream, cars that predict, and a factor zoo that violates first principles — is **[docs/JOURNEY.md](docs/JOURNEY.md)**.* + --- ## Table of Contents 1. [The Problem Nobody Has Solved](#the-problem-nobody-has-solved) -2. [Why This Question Is Urgent — Right Now](#why-this-question-is-urgent--right-now) -3. [What Came Before — And Why It Falls Short](#what-came-before--and-why-it-falls-short) -4. [Two Kinds of World Model: Type 1 and Type 2](#two-kinds-of-world-model-type-1-and-type-2) -5. [The Market as a Four-Level Game](#the-market-as-a-four-level-game) -6. [The E-Game-C Architecture](#the-e-game-c-architecture) -7. [The Mathematical Framework](#the-mathematical-framework-two-threads-one-theory) -8. [The Unified Evolution Equation](#the-unified-evolution-equation-theorem-91) -9. [The Seven Theorems](#the-seven-theorems) -10. [Reflexivity — Soros, Formalized](#reflexivity--soros-formalized) -11. [The Agent Taxonomy](#the-agent-taxonomy) -12. [Connection to the 2026 Fields Medal](#connection-to-the-2026-fields-medal-deng-yu-邓煜) -13. [What This Makes Possible](#what-this-makes-possible) -14. [Data Requirements & Research Roadmap](#data-requirements--research-roadmap) -15. [The 17-Day Notebook Series](#the-17-day-notebook-series) -16. [Repository Structure](#repository-structure) -17. [Quick Start](#quick-start) -18. [Related Work & Positioning](#related-work--positioning) -19. [Project Roadmap](#project-roadmap) -20. [References](#references) -21. [Citation](#citation) +2. [The First-Principles Bet — Causality over Correlation](#the-first-principles-bet--causality-over-correlation) +3. [Why This Question Is Urgent — Right Now](#why-this-question-is-urgent--right-now) +4. [What Came Before — And Why It Falls Short](#what-came-before--and-why-it-falls-short) +5. [Two Kinds of World Model: Type 1 and Type 2](#two-kinds-of-world-model-type-1-and-type-2) +6. [The Market as a Four-Level Game](#the-market-as-a-four-level-game) +7. [The E-Game-C Architecture](#the-e-game-c-architecture) +8. [The Mathematical Framework](#the-mathematical-framework-two-threads-one-theory) +9. [The Unified Evolution Equation](#the-unified-evolution-equation-theorem-91) +10. [The Seven Theorems](#the-seven-theorems) +11. [Reflexivity — Soros, Formalized](#reflexivity--soros-formalized) +12. [The Agent Taxonomy](#the-agent-taxonomy) +13. [Connection to the 2026 Fields Medal](#connection-to-the-2026-fields-medal-deng-yu-邓煜) +14. [What This Makes Possible](#what-this-makes-possible) +15. [Two Products, One Engine](#two-products-one-engine) +16. [Data Requirements & Research Roadmap](#data-requirements--research-roadmap) +17. [The 17-Day Notebook Series](#the-17-day-notebook-series) +18. [Repository Structure](#repository-structure) +19. [Quick Start](#quick-start) +20. [Related Work & Positioning](#related-work--positioning) +21. [Project Roadmap — Phase 1 and Phase 2](#project-roadmap--phase-1-and-phase-2) +22. [Star History](#star-history) +23. [Partnerships & Contact](#partnerships--contact) +24. [References](#references) +25. [Citation](#citation) --- @@ -133,6 +139,28 @@ The answer is yes. This repository presents that theory, and its engineering imp --- +## The First-Principles Bet — Causality over Correlation + +Every mainstream quant methodology — factor mining, time-series modelling, feature engineering — rests on one unexamined premise: **that historical data will repeat.** It will not. Every backtested pattern is a snapshot of a game whose players have since changed their strategies, partly *because* the pattern was found. That is the Lucas critique with teeth, and it is why the factor zoo decays and deployed models die. + +But something *does* repeat. Not the data — the **world that generates the data**: institutions with mandates, regulators with rules, incentives that do not change when a signal is published. History does not repeat its prices; it repeats its *mechanisms*. So the only factors that can carry meaning are the ones derived by going back to the world itself — and this project refuses to model the residue when it can model the generator: + +| | Pattern paradigm (factors, TS-ML) | World-model paradigm (this repo) | +|---|---|---| +| **Object modeled** | The data the market left behind | The market itself: agents, constraints, equilibrium | +| **A "factor" is** | A correlation that worked in-sample | A term in an equilibrium condition, with a mechanism attached | +| **When regimes break** | Silent failure — nothing to explain with | The explanation *is* the model: which agents, which constraint, which coupling | +| **Deployment** | Erodes the signal (crowding) | Reinforces the signal (equilibria are self-consistent) | + +Two consequences follow, and they are this project's identity: + +1. **Explainability is the direction of inference, not a feature.** We do not fit prices and hope for meaning; we model the mechanism and *derive* what prices must do. Every output is attributable to named agents, named constraints, and an equilibrium condition you can inspect. +2. **The model survives contact with the market.** A prediction that is an equilibrium does not evaporate when acted on — being acted on is how equilibria assert themselves. + +> **We did not find structure in the data. We modeled the structure that makes the data.** That sentence is the entire difference between this repository and every pattern-mining stack in production today. + +--- + ## Why This Question Is Urgent — Right Now Three simultaneous forces are breaking the old paradigm faster than at any point in history: @@ -753,6 +781,38 @@ When retail investors coordinate (GME, AMC, any future short squeeze), the behav --- +## Two Products, One Engine + +The same E-Game-C core surfaces as two products — one per audience. + +### To-C · The denoised equilibrium price (mid/long-horizon) + +Theorem 1 says the behavioral noise ν_η cannot be out-traded — by anyone, retail least of all. What a long-horizon investor actually needs is the component *underneath* the noise: the **equilibrium track P^eq** — what the asset is worth once every agent has played its rational strategy — and the divergence **D_t = P_t/P^eq_t − 1**, which answers exactly one question: *are you buying value, or buying crowding?* + +[](demo/denoised_price_2026.py) + +The scenario above is **synthetic** — a concept demo shaped on the July 2026 memory-sector unwind, the month the Philadelphia Semiconductor Index lost 19% (its worst since 2008) while retail flows capitulated at lows that institutions were buying. In the simulation, the signal — divergence above threshold *while the institutional mean field rotates out* — fires **11 trading days (≈2 weeks) before the unwind accelerates**. The real-data version, under the same honesty rules as the 2008 hindcast (walk-forward, no look-ahead), is specified as experiment **[E7](RESOURCES.md)**. + +```bash +python demo/denoised_price_2026.py # reproduce the figure — synthetic, CPU, no keys +``` + +> *Research signal, not advice.* The denoised price is an instrument-level research layer for mid/long-horizon positioning. It is not personalized investment advice, and this repository is not an advisor. + +### To-B · Structural risk for institutions + +For funds, risk desks, and platforms, the same machinery runs in the other direction — selling not the equilibrium but the **distance from it**: + +- **Λₜ regime monitoring** — the 2008/2020-grade basin-exit signal, intraday ([`online/regime_detector.py`](online/regime_detector.py)) +- **Crowding decomposition** — how much of a book's P&L is equilibrium drift vs behavioral wedge, per position +- **Event-operator scenario analysis** — M&A, policy moves, delistings applied as first-class operators to *today's* state, not as historical analogies + +The production loop (Airflow DAG → noise → encoder → MFG → signal → execution) is already scaffolded in [`online/`](online/) with an Alpaca paper-trading stub. This is Horizon 3 of the roadmap. + +**One engine, two surfaces:** the retail product is the equilibrium; the institutional product is the deviation from it. Both are outputs of the same solve. + +--- + ## Data Requirements & Research Roadmap The mathematics is closed and the code runs end-to-end on synthetic data. The road from prototype to validated instrument is **data** — hundreds of fragmented streams, each feeding a specific term of a specific equation. The complete acquisition plan lives in **[DATA_REQUIREMENTS.md](DATA_REQUIREMENTS.md)**: every source named, every stub located, every cost tiered, and — for the data that exists nowhere — the experiments that create it. @@ -847,6 +907,8 @@ MicroWorld/ ├── demo/ │ ├── run_egamec.py # 30-second E-Game-C pipeline demo │ ├── global_demo.py # ★ The animated world-model demo (GIF above) +│ ├── hindcast_2008.py # The 2008 walk-forward hindcast (figure at top) +│ ├── denoised_price_2026.py # To-C concept demo: denoised equilibrium price │ └── synthetic_market.py # Dual-noise synthetic market generator │ ├── scripts/make_figures.py # Regenerates every README figure from library code @@ -854,6 +916,8 @@ MicroWorld/ ├── figures/ # All SVG diagrams + generated PNGs + demo GIF ├── tests/ # 50 tests, all passing (noise · events · features) ├── DATA_REQUIREMENTS.md # ★ The complete data & experiment roadmap +├── RESOURCES.md # ★ Data + compute, priced — the use-of-funds view +├── docs/ # JOURNEY · PHASE2_NEURAL_GAME · ONE_PAGER · i18n ├── CITATION.cff # Citable metadata └── .github/workflows/ci.yml # CI: pytest on 3.11 / 3.12 ``` @@ -866,6 +930,10 @@ MicroWorld/ |---|---|---| | **Engineering Implementation** (E-Game-C) | [us-equity-world-model](https://github.com/hongjin-he/us-equity-world-model) | Full build manual: data layer, encoder, MFG solver, controller, backtest, deployment | | **Mathematical Paper** | [mathmatical-framework-for-world-models-in-quant-finance](https://github.com/hongjin-he/mathmatical-framework-for-world-models-in-quant-finance) | Alpha Flow 02: all proofs, 9 theorems, 25 pages | +| **The Journey** | [docs/JOURNEY.md](docs/JOURNEY.md) | From world-model skeptic to this architecture — with the robots, cars, and dreams that did the converting | +| **Phase 2 Design** | [docs/PHASE2_NEURAL_GAME.md](docs/PHASE2_NEURAL_GAME.md) | The Neural Network Game Structure: every neuron an agent-network | +| **Resources & Use of Funds** | [RESOURCES.md](RESOURCES.md) | Data + compute, priced by scenario; experiments E7–E11 | +| **One-Pager** | [docs/ONE_PAGER.md](docs/ONE_PAGER.md) | The whole project on one page, for partners and investors | --- @@ -882,6 +950,9 @@ python demo/global_demo.py # 2 · The 30-second pipeline demo python demo/run_egamec.py +# 2b · The denoised-price concept demo (To-C product line, synthetic) +python demo/denoised_price_2026.py + # 3 · Regenerate every figure in this README from library code python scripts/make_figures.py @@ -934,7 +1005,7 @@ Where MicroWorld sits relative to each adjacent literature: --- -## Project Roadmap +## Project Roadmap — Phase 1 and Phase 2 The project holds itself to a **dual standard**, on purpose: @@ -951,6 +1022,12 @@ E3 (equilibrium consistency vs 13F/COT), E6 (L0 transmission), encoder trained o **Horizon 3 — the industrial product (P2).** Live daily pipeline (Airflow DAG already scaffolded), paper-trading via Alpaca stub, dashboard, and — compute permitting — the Type 2 sandbox with the Type 1 equilibrium as its outer loop. +--- + +Everything above is **Phase 1 — the game-theoretic core**: mathematical structure standing in for scarce data, and not as a compromise — Theorem 1 proves part of the residual can *only* be explained by mechanism, never estimated away, so structure-first is the theoretically correct regime at today's data access. When the cleaned panel and H200-class compute both exist, the architecture is scheduled to shed its skin: + +**Phase 2 — the [Neural Network Game Structure](docs/PHASE2_NEURAL_GAME.md).** A network in which **every neuron is itself a small neural network** — one per institution or retail cohort, with its own objective and information set; regulatory reality imposed as architecture (all agents of a type share their regulator's constraint module); the forward pass *is* the game being played. Whether the environment and the population live in one network or two is assigned to experiment, not taste (E8). Training difficulty roughly squares — which is exactly why the design document is public now and the training waits for the hardware. Triggers, pilots, and budget: [RESOURCES.md](RESOURCES.md). + Contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). --- @@ -1013,6 +1090,43 @@ Contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). [39] J. Sirignano, K. Spiliopoulos, "DGM: a deep learning algorithm for solving partial differential equations," *J. Computational Physics*, 2018. [40] R. Cont, J.-P. Bouchaud, "Herd behavior and aggregate fluctuations in financial markets," *Macroeconomic Dynamics*, 2000. +**World models across domains, and the factor zoo (the journey — [docs/JOURNEY.md](docs/JOURNEY.md))** +[41] P. Wu, A. Escontrela, D. Hafner, P. Abbeel, K. Goldberg, "DayDreamer: world models for physical robot learning," *CoRL*, 2022. +[42] A. Hu et al., "GAIA-1: a generative world model for autonomous driving," arXiv:2309.17080, 2023. +[43] J. Bruce et al., "Genie: generative interactive environments," *ICML*, 2024. +[44] NVIDIA, "Cosmos world foundation model platform for physical AI," arXiv:2501.03575, 2025. +[45] M. Assran et al., "V-JEPA 2: self-supervised video models enable understanding, prediction and planning," arXiv:2506.09985, 2025. +[46] J. H. Cochrane, "Presidential address: discount rates," *Journal of Finance*, 2011. + +--- + +## Star History + +Stars are this project's market validation — public, timestamped, and unfakeable in slope. The curve updates live: + +[](https://star-history.com/#hongjin-he/MicroWorld&Date) + +If the framework earned your star, the next-highest-leverage contribution is one issue: tell us which claim you'd attack first. + +--- + +## Partnerships & Contact + +MicroWorld is looking for exactly four kinds of counterparty: + +| Who | What we bring | What we need | +|---|---|---| +| **Academic research groups** | A NeurIPS-workshop-ready experiment suite (E1–E7) executable in one quarter | WRDS-grade data access; co-authorship welcome | +| **Quant funds & prop desks** | The Λₜ early-warning and crowding-decomposition line (To-B) — plus first access to E7 results | A pilot conversation and honest adversarial review | +| **Compute partners** | A publicly specified Phase 2 architecture ([design doc](docs/PHASE2_NEURAL_GAME.md)) that is H200-shaped by construction | H100/H200 hours for experiments E8–E11 | +| **Retail platforms & media** | The denoised-price research layer (To-C) — a story retail investors actually need after July 2026 | Distribution and product feedback | + +Start with the **[one-pager](docs/ONE_PAGER.md)**; budgets and scenarios are in **[RESOURCES.md](RESOURCES.md)**. + +**Channels:** [LinkedIn](https://www.linkedin.com/in/hongjinhe-hkust-edu) · [X](https://x.com/Mr_Abstractor) · [GitHub issues](https://github.com/hongjin-he/MicroWorld/issues) — technical objections get the fastest replies. + +> *Compliance note: this repository is research software. Nothing in it is investment advice, portfolio management, or an offer of securities.* + --- ## Citation diff --git a/README_CN.md b/README_CN.md index ce417c5..e583c12 100644 --- a/README_CN.md +++ b/README_CN.md @@ -27,7 +27,7 @@ [](https://www.instagram.com/mr.abstractor_ust/) [](https://www.linkedin.com/in/hongjinhe-hkust-edu) -**[English](README.md) | [中文文档](README_CN.md)** +**[English](README.md) | [中文文档](README_CN.md) | [日本語](docs/i18n/README_JA.md) | [한국어](docs/i18n/README_KO.md)** **Alpha Flow Research · 何泓锦 · 香港科技大学 / 斯坦福 IHP · 2026年7月** @@ -83,31 +83,37 @@ python demo/global_demo.py # 端到端复现此 GIF(约 2 分钟,仅 CPU *—— Alpha Flow Research · 斯坦福某间地下室 · 2026年7月* +*更长的版本——一个世界模型怀疑论者,如何被会做梦的机器人、会预判的自动驾驶和一个违背第一性原理的因子动物园说服——在 **[docs/JOURNEY.md](docs/JOURNEY.md)**(英文)。* + --- ## 目录 1. [没有人真正解决的问题](#没有人真正解决的问题) -2. [为什么这个问题现在最紧迫](#为什么这个问题现在最紧迫) -3. [前人的研究——以及它们为什么不够](#前人的研究以及它们为什么不够) -4. [两种世界模型:Type 1 与 Type 2](#两种世界模型type-1-与-type-2) -5. [市场作为四层博弈](#市场作为四层博弈) -6. [E-Game-C 架构](#e-game-c-架构) -7. [数学框架:双线并行](#数学框架双线并行) -8. [统一演化方程](#统一演化方程定理91) -9. [七大定理](#七大定理) -10. [反身性——形式化索罗斯](#反身性形式化索罗斯) -11. [主体分类法](#主体分类法) -12. [与2026年菲尔兹奖的联系](#与2026年菲尔兹奖的联系邓煜) -13. [这让什么成为可能](#这让什么成为可能) -14. [数据需求与研究路线图](#数据需求与研究路线图) -15. [17 天笔记本系列](#17-天笔记本系列) -16. [仓库结构](#仓库结构) -17. [快速开始](#快速开始) -18. [相关工作与定位](#相关工作与定位) -19. [项目路线图](#项目路线图) -20. [参考文献](#参考文献) -21. [引用](#引用) +2. [第一性原理之赌——因果而非相关](#第一性原理之赌因果而非相关) +3. [为什么这个问题现在最紧迫](#为什么这个问题现在最紧迫) +4. [前人的研究——以及它们为什么不够](#前人的研究以及它们为什么不够) +5. [两种世界模型:Type 1 与 Type 2](#两种世界模型type-1-与-type-2) +6. [市场作为四层博弈](#市场作为四层博弈) +7. [E-Game-C 架构](#e-game-c-架构) +8. [数学框架:双线并行](#数学框架双线并行) +9. [统一演化方程](#统一演化方程定理91) +10. [七大定理](#七大定理) +11. [反身性——形式化索罗斯](#反身性形式化索罗斯) +12. [主体分类法](#主体分类法) +13. [与2026年菲尔兹奖的联系](#与2026年菲尔兹奖的联系邓煜) +14. [这让什么成为可能](#这让什么成为可能) +15. [同一引擎,两条产品线](#同一引擎两条产品线) +16. [数据需求与研究路线图](#数据需求与研究路线图) +17. [17 天笔记本系列](#17-天笔记本系列) +18. [仓库结构](#仓库结构) +19. [快速开始](#快速开始) +20. [相关工作与定位](#相关工作与定位) +21. [项目路线图——第一阶段与第二阶段](#项目路线图第一阶段与第二阶段) +22. [Star 增长曲线](#star-增长曲线) +23. [合作与联系](#合作与联系) +24. [参考文献](#参考文献) +25. [引用](#引用) --- @@ -127,6 +133,28 @@ python demo/global_demo.py # 端到端复现此 GIF(约 2 分钟,仅 CPU --- +## 第一性原理之赌——因果而非相关 + +每一种主流量化方法——因子挖掘、时间序列建模、特征工程——都建立在一个未经审视的前提上:**历史数据会重演。** 它不会。每一个回测出来的模式,都是一场博弈的快照,而博弈的玩家早已改变了策略——部分原因恰恰是这个模式被发现了。这就是长出牙齿的卢卡斯批判,也是因子动物园持续衰减、已部署模型不断死亡的原因。 + +但确实有东西在重演。不是数据——而是**生成数据的那个世界**:有法定授权的机构、有规则的监管者、不因信号被发表而改变的激励结构。历史不会重复它的价格,历史重复的是它的*机制*。因此,唯一能承载意义的因子,是回到世界本身推导出来的因子——本项目拒绝为残渣建模,因为我们可以为生成器建模: + +| | 模式范式(因子、时序ML) | 世界模型范式(本仓库) | +|---|---|---| +| **建模对象** | 市场留下的数据 | 市场本身:主体、约束、均衡 | +| **一个"因子"是** | 一条样本内有效的相关性 | 均衡条件中的一项,附带机制 | +| **机制断裂时** | 静默失效——无从解释 | 解释*就是*模型:哪些主体、哪条约束、哪个耦合 | +| **部署之后** | 信号被侵蚀(拥挤) | 信号被强化(均衡自洽) | + +由此推出两个结论,它们就是本项目的身份: + +1. **可解释性不是附加功能,而是推断的方向。** 我们不是拟合价格再祈祷其中有意义;我们建模机制,然后*推导*价格必须如何运动。每一个输出都可归因于具名的主体、具名的约束和一个可检查的均衡条件。 +2. **模型经得起与市场的接触。** 一个本身就是均衡的预测,不会因为被执行而蒸发——被执行,正是均衡实现自身的方式。 + +> **我们没有在数据里找到结构。我们建模了生成数据的结构。** 这一句话,就是本仓库与今天所有在产模式挖掘系统的全部区别。 + +--- + ## 为什么这个问题现在最紧迫 三股力量正在以历史上最快的速度瓦解旧范式: @@ -734,6 +762,38 @@ $\pi^* = (\gamma\Sigma + \lambda I)^{-1}(\mu + \lambda\,\mu^{\text{MFG}})$ --- +## 同一引擎,两条产品线 + +同一个 E-Game-C 内核,面向两类受众呈现为两个产品。 + +### To-C · 去噪均衡价格(中长期持有) + +定理1说明:行为噪声 ν_η 是任何人都无法通过交易战胜的——散户尤其如此。一个长期投资者真正需要的,是噪声*之下*的那个分量:**均衡轨迹 P^eq**——当每一个主体都打出理性策略后,这个资产值多少钱——以及偏离度 **D_t = P_t/P^eq_t − 1**。它只回答一个问题:*你买入的是价值,还是拥挤?* + +[](demo/denoised_price_2026.py) + +上图是**合成数据**——一个以2026年7月存储板块崩盘为情境原型的概念演示:那个月费城半导体指数下跌19%(2008年以来最差单月),散户在低点恐慌抛售,而机构在同一低点买入。在模拟中,信号——偏离度突破阈值*且机构平均场正在撤出*——在崩盘加速前 **11个交易日(约两周)** 触发。真实数据版本遵循与2008年回测完全相同的诚实规则(滚动前向、无未来函数),已定义为实验 **[E7](RESOURCES.md)**。 + +```bash +python demo/denoised_price_2026.py # 复现上图——合成数据,CPU,无需密钥 +``` + +> *研究信号,而非投资建议。* 去噪价格是面向中长期持仓的标的级研究层。它不是个性化投资建议,本仓库也不是投资顾问。 + +### To-B · 机构级结构性风险 + +对基金、风控台和平台,同一套均衡机器反向运行——卖的不是均衡本身,而是**与均衡的距离**: + +- **Λₜ 机制监控**——2008/2020级别的稳定域逃逸信号,盘中运行([`online/regime_detector.py`](online/regime_detector.py)) +- **拥挤度分解**——一本账簿的盈亏中,多少来自均衡漂移、多少来自行为楔子,逐仓位归因 +- **事件算子情景分析**——并购、政策、退市作为第一等算子作用于*今天*的状态,而非历史类比 + +生产环闭环(Airflow DAG → 噪声 → 编码器 → MFG → 信号 → 执行)已在 [`online/`](online/) 搭好脚手架,含 Alpaca 模拟盘 stub。这是路线图的地平线3。 + +**同一引擎,两个表面:** 散户产品卖的是均衡,机构产品卖的是对均衡的偏离。二者是同一次求解的两个输出。 + +--- + ## 数据需求与研究路线图 数学已闭合,代码在合成数据上端到端运行。从原型到经过验证的研究工具,路上缺的是**数据**——数百条零散的数据流,每一条喂给特定方程的特定一项。完整获取计划在 **[DATA_REQUIREMENTS.md](DATA_REQUIREMENTS.md)**:每个数据源具名、每个 stub 定位到行、每项成本分档,以及——对于世界上还不存在的数据——创造它们的实验设计。 @@ -828,6 +888,8 @@ MicroWorld/ ├── demo/ │ ├── run_egamec.py # 30秒 E-Game-C 流水线演示 │ ├── global_demo.py # ★ 动画世界模型演示(顶部GIF) +│ ├── hindcast_2008.py # 2008滚动前向回测(顶部图) +│ ├── denoised_price_2026.py # To-C概念演示:去噪均衡价格 │ └── synthetic_market.py # 双噪声合成市场生成器 │ ├── scripts/make_figures.py # 从库代码重新生成README全部图表 @@ -835,6 +897,8 @@ MicroWorld/ ├── figures/ # 全部SVG图 + 生成的PNG + 演示GIF ├── tests/ # 50个测试,全部通过(噪声·事件·特征) ├── DATA_REQUIREMENTS.md # ★ 完整数据与实验路线图 +├── RESOURCES.md # ★ 数据+算力定价——资金用途视角 +├── docs/ # JOURNEY · PHASE2_NEURAL_GAME · ONE_PAGER · i18n ├── CITATION.cff # 可引用元数据 └── .github/workflows/ci.yml # CI:pytest on 3.11 / 3.12 ``` @@ -847,6 +911,10 @@ MicroWorld/ |---|---|---| | **工程实现**(E-Game-C) | [us-equity-world-model](https://github.com/hongjin-he/us-equity-world-model) | 完整构建手册:数据层、编码器、MFG求解器、控制器、回测、部署 | | **数学论文** | [mathmatical-framework-for-world-models-in-quant-finance](https://github.com/hongjin-he/mathmatical-framework-for-world-models-in-quant-finance) | Alpha Flow 02:全部证明,9个定理,25页 | +| **心路历程** | [docs/JOURNEY.md](docs/JOURNEY.md) | 从世界模型怀疑论者到这套架构——以及完成说服的那些机器人、汽车与梦境 | +| **第二阶段设计** | [docs/PHASE2_NEURAL_GAME.md](docs/PHASE2_NEURAL_GAME.md) | 神经网络博弈结构:每个神经元本身是一个主体网络 | +| **资源与资金用途** | [RESOURCES.md](RESOURCES.md) | 数据+算力按场景定价;实验E7–E11 | +| **一页纸** | [docs/ONE_PAGER.md](docs/ONE_PAGER.md) | 面向合作方与投资人的单页项目全景 | --- @@ -863,6 +931,9 @@ python demo/global_demo.py # 2 · 30秒流水线演示 python demo/run_egamec.py +# 2b · 去噪价格概念演示(To-C产品线,合成数据) +python demo/denoised_price_2026.py + # 3 · 从库代码重新生成本README全部图表 python scripts/make_figures.py @@ -915,7 +986,7 @@ MicroWorld 相对每条相邻研究线的位置: --- -## 项目路线图 +## 项目路线图——第一阶段与第二阶段 本项目刻意坚持**双重标准**: @@ -932,6 +1003,12 @@ E3(均衡一致性 vs 13F/COT)、E6(L0传导)、在真实面板上训练 **地平线3——工业产品(P2)。** 每日实盘流水线(Airflow DAG已搭好脚手架)、经Alpaca stub的模拟盘交易、监控面板,以及——算力允许时——以 Type 1 均衡为外层循环的 Type 2 沙盒。 +--- + +以上全部属于**第一阶段——博弈论内核**:用数学结构补偿稀缺的数据,而且这不是妥协——定理1证明了残差中有一部分*只能*被机制解释、永远无法靠堆数据估计掉,所以在今天的数据获取水平上,结构优先在理论上就是正确的选择。当清理好的数据面板与 H200 级算力同时就位,这套架构将按计划蜕皮: + +**第二阶段——[神经网络博弈结构](docs/PHASE2_NEURAL_GAME.md)(NNGS)。** 一个**每个神经元本身就是一个小神经网络**的网络——每个机构或散户群体一个,各有自己的目标函数与信息集;监管现实作为架构直接施加(同一类型的所有主体共享其监管者的约束模块);前向传播*就是*博弈的进行。环境与博弈人群放在一个网络还是两个网络,交给实验裁决而非审美(E8)。训练难度大约平方化——这正是设计文档现在就公开、而训练等待硬件的原因。触发条件、试点与预算:[RESOURCES.md](RESOURCES.md)。 + 欢迎贡献——见 [CONTRIBUTING.md](CONTRIBUTING.md)。 --- @@ -994,6 +1071,43 @@ E3(均衡一致性 vs 13F/COT)、E6(L0传导)、在真实面板上训练 [39] J. Sirignano, K. Spiliopoulos, "DGM: a deep learning algorithm for solving partial differential equations," *J. Computational Physics*, 2018. [40] R. Cont, J.-P. Bouchaud, "Herd behavior and aggregate fluctuations in financial markets," *Macroeconomic Dynamics*, 2000. +**跨领域世界模型与因子动物园(心路历程——[docs/JOURNEY.md](docs/JOURNEY.md))** +[41] P. Wu, A. Escontrela, D. Hafner, P. Abbeel, K. Goldberg, "DayDreamer: world models for physical robot learning," *CoRL*, 2022. +[42] A. Hu et al., "GAIA-1: a generative world model for autonomous driving," arXiv:2309.17080, 2023. +[43] J. Bruce et al., "Genie: generative interactive environments," *ICML*, 2024. +[44] NVIDIA, "Cosmos world foundation model platform for physical AI," arXiv:2501.03575, 2025. +[45] M. Assran et al., "V-JEPA 2: self-supervised video models enable understanding, prediction and planning," arXiv:2506.09985, 2025. +[46] J. H. Cochrane, "Presidential address: discount rates," *Journal of Finance*, 2011. + +--- + +## Star 增长曲线 + +Star 是这个项目的市场验证——公开、带时间戳、斜率无法造假。曲线实时更新: + +[](https://star-history.com/#hongjin-he/MicroWorld&Date) + +如果这套框架配得上你的 star,那么杠杆率第二高的贡献是提一个 issue:告诉我们你最想攻击哪一条主张。 + +--- + +## 合作与联系 + +MicroWorld 正在寻找恰好四类对手方: + +| 对象 | 我们带来什么 | 我们需要什么 | +|---|---|---| +| **学术研究组** | 一季度内可执行、达到 NeurIPS workshop 标准的实验套件(E1–E7) | WRDS 级数据权限;欢迎共同署名 | +| **量化基金与自营交易台** | Λₜ 预警与拥挤度分解产品线(To-B),外加 E7 结果的优先获取 | 一次试点对话与诚实的对抗性审查 | +| **算力伙伴** | 一套公开成文、天生为 H200 而设计的第二阶段架构([设计文档](docs/PHASE2_NEURAL_GAME.md)) | 用于实验 E8–E11 的 H100/H200 机时 | +| **零售平台与媒体** | 去噪价格研究层(To-C)——2026年7月之后散户真正需要的那个故事 | 分发渠道与产品反馈 | + +从**[一页纸](docs/ONE_PAGER.md)**开始;预算与场景在 **[RESOURCES.md](RESOURCES.md)**。 + +**渠道:** [LinkedIn](https://www.linkedin.com/in/hongjinhe-hkust-edu) · [X](https://x.com/Mr_Abstractor) · [GitHub issues](https://github.com/hongjin-he/MicroWorld/issues)——技术性质疑回复最快。 + +> *合规声明:本仓库是研究软件。其中任何内容均不构成投资建议、投资组合管理或证券要约。* + --- ## 引用 diff --git a/RESOURCES.md b/RESOURCES.md new file mode 100644 index 0000000..9ed3483 --- /dev/null +++ b/RESOURCES.md @@ -0,0 +1,92 @@ +# RESOURCES — What This Project Needs, Priced + +Two things stand between the current repository and a validated instrument: +**cleaned data** and **training compute**. This file prices both. It is +simultaneously the project's internal shopping list and — for a potential +partner or investor — the use-of-funds statement. Experiment-level detail +for the data lives in [DATA_REQUIREMENTS.md](DATA_REQUIREMENTS.md); the +Phase 2 architecture that consumes the compute lives in +[docs/PHASE2_NEURAL_GAME.md](docs/PHASE2_NEURAL_GAME.md). + +--- + +## 1 · Pillar one: cleaned, *typed* data + +The framework does not want "more data" — it wants **each agent type's +behavior observed at the level that type actually acts**. That is a +different shopping list from a factor shop's, organized by which term of +the model each stream feeds: + +| Agent type / model term | Data | Source | Cost tier | +|---|---|---|---| +| Physical vs behavioral noise (Thm 1) | Intraday trades & quotes, 500 equities × 10 yr | WRDS TAQ / Polygon | academic access / ~$2k·yr | +| Event operators (all 22) | Corporate actions, M&A, IPO, delistings, earnings | CRSP + SDC + I/B/E/S | academic access | +| Institutional mean field μ* (L1–L2) | Quarterly 13F holdings, futures COT positioning | EDGAR (free) + CFTC (free) | free | +| Retail cohort policies (Day 16) | **The E5 LLM query atlas** — stratified audit of consumer AI investment advice; the dataset exists nowhere and we create it | consumer LLM APIs | ~$200 | +| Funding / stress channel (Λₜ) | FRED spreads, FINRA margin, OFR indices, CBOE | public | free | +| Cross-market L0 | TIC flows, dollar indices | public | free | +| Alt-data layer (later) | News embeddings, positioning surveys | vendor | deferred | + +**The punchline stays the same as DATA_REQUIREMENTS.md:** experiments +E1 + E2 + E4 + E5 — enough for the first real-data paper — are executable +in one quarter by one person inside any research group with WRDS access +plus about **$200** of API budget. The scarce resource is access, not money. + +### E7 — the real-data denoised-price validation (To-C line) + +The synthetic concept demo ([`demo/denoised_price_2026.py`](demo/denoised_price_2026.py)) +graduates to a real-data experiment: + +> **E7.** Reconstruct the denoised equilibrium track P^eq for the memory +> sector through the July 2026 unwind (SOX −19%, worst month since 2008) +> using only point-in-time data: prices (CRSP/Polygon), institutional +> positioning (13F + COT), retail-flow proxies, and the E5 retail-AI +> kernel. Measure: did divergence D_t cross threshold, with the +> institutional field rotating out, materially before July 24? Deliverable: +> a walk-forward figure exactly like the 2008 hindcast — same honesty +> rules, no look-ahead. + +Data unlock: same as E1–E3 (nothing new to buy). Priority: **P0 for the +product line** — this is the first exhibit any retail-facing partner will +ask for. + +## 2 · Pillar two: training compute + +Phase 1 (everything in this repo) runs on a laptop; that was the point. +Phase 2 — the [Neural Network Game Structure](docs/PHASE2_NEURAL_GAME.md), +where every neuron is itself a small network — is where the compute bill +arrives. Costs below are honest ranges at mid-2026 cloud prices, not +precision estimates: + +| Stage | What runs | Hardware | Est. cost | +|---|---|---|---| +| Phase 1 (today) | HJB–FPK solvers, demos, 50 tests | laptop / free Colab | ~$0 | +| Real-data Phase 1 (E1–E7) | Encoder training on real panel, walk-forward backtests | 1× consumer GPU or A100 spot | $1–3k | +| Phase 2 pilots (E8–E10) | Architecture ablations on synthetic market, ~10² agent-networks | 1× H100/H200 | $5–15k | +| Phase 2 full train (E11) | L1+L2 NNGS: ~10³ neuron-networks × 10⁵–10⁶ params, adversarial co-training + reflexivity loop | **8× H200 node, weeks-scale runs** | $100–250k per campaign | +| Type 2 sandbox (Horizon 3+) | Agent-level "capitalism simulator" disciplined by the Type 1 equilibrium | multi-node cluster | deferred until E11 says it's earned | + +Why it squares: each unit's learning signal depends on every other unit's +current policy (non-stationary co-training), and the units themselves are +models — see the complexity-wall section of the +[Phase 2 design doc](docs/PHASE2_NEURAL_GAME.md#5--the-complexity-wall--and-the-four-tools-against-it) +for the four reductions (mean-field factorization, latent embeddings, +hierarchy-as-curriculum, sparse strategic attention) that keep the bill in +five figures for pilots rather than seven. + +## 3 · Use of funds, by scenario + +| Scenario | Budget | Buys | +|---|---|---| +| **Bootstrap** (status quo) | ~$500 | E5 atlas ($200) + Polygon starter + misc. Everything else free/academic. | +| **Seed research grant** | ~$25k | All of the above + real-data Phase 1 (E1–E7) + Phase 2 pilots (E8–E10) on rented H100/H200. Output: the NeurIPS-workshop paper *and* the E7 product exhibit. | +| **Partner / pre-seed** | ~$300k | One full NNGS training campaign (E11) + one year of data subscriptions + paper-trading infrastructure live (Airflow + Alpaca, already scaffolded in [`online/`](online/)). | + +No headcount is priced in: the maintainer cost of this project is one +student who refuses to stop. + +--- + +*Offering access, compute, or capital: see +[Partnerships & Contact](README.md#partnerships--contact) — or open an +issue.* diff --git a/demo/denoised_price_2026.py b/demo/denoised_price_2026.py new file mode 100644 index 0000000..fcf5e05 --- /dev/null +++ b/demo/denoised_price_2026.py @@ -0,0 +1,205 @@ +""" +Denoised Equilibrium Price · 2026 — the retail product line, as a concept demo +on SYNTHETIC data, qualitatively calibrated to the July 2026 memory-sector +unwind (SOX −19% in July 2026, its worst month since 2008). + +⚠️ THIS IS A SYNTHETIC SCENARIO, NOT A HINDCAST ON REAL DATA AND NOT + INVESTMENT ADVICE. Every path below is generated by the dual-noise + + mean-field machinery of this repo. The July 2026 episode supplies the + *narrative shape* (institutional crowding → LLM-homogenized retail + chase → fundamental re-rating → forced unwind with retail capitulating + at the bottom while institutions re-enter); the numbers are simulated. + The real-data version of this demo is experiment E7 in RESOURCES.md. + +The product thesis (To-C line): + A retail investor cannot out-trade the behavioral noise ν_η — Theorem 1 + (dual Cramér-Rao) says nobody can. But the *equilibrium* component of + the price — what the asset is worth once every agent has played its + rational strategy — is exactly what the MFG solves for. Publish that + denoised track and the divergence D_t = P_t/P_t^eq − 1 becomes a + mid/long-horizon positioning signal: when the market trades 25% above + its own equilibrium while the institutional mean field is already + rotating out, you do not need to call the day of the crash — you need + to not be the one buying the top. + +Mechanics (same objects as the rest of the repo): + P_t^eq — equilibrium price: the MFG-consistent fundamental track. + Steps down once, on the guidance shock (a Mode-I event + operator: genuine information, Theorem 3 irreversibility). + κ_t — institutional crowding (Level-1/2 mean-field concentration). + f_t — retail flow: momentum-chasing, LLM-homogenized (Day 16). + w_t — the behavioral wedge: dw = a·(κ + b·f⁺) dt − c·w dt + jumps. + Market price P_t = P_t^eq · (1 + w_t). + Signal — U_t fires when D_t > θ_D while the smoothed κ-trend has + turned negative for ≥3 consecutive sessions: the wedge is + large AND the agents holding it up are leaving. + +Run: python demo/denoised_price_2026.py → figures/denoised_price_demo.png +""" +import os +import numpy as np +import pandas as pd +import matplotlib +matplotlib.use("Agg") +import matplotlib.pyplot as plt + +HERE = os.path.dirname(os.path.abspath(__file__)) +FIGS = os.path.join(os.path.dirname(HERE), "figures") + +rng = np.random.default_rng(20260729) + +# ── calendar ─────────────────────────────────────────────────────────────────── +dates = pd.bdate_range(end="2026-07-29", periods=252) +n = len(dates) +t = np.arange(n) + +GUIDANCE = pd.Timestamp("2026-07-13") # HBM4-delay / guidance shock (Mode-I event) +UNWIND = pd.Timestamp("2026-07-24") # unwind accelerates (Korea selloff spillover) +CAPITUL = pd.Timestamp("2026-07-28") # capitulation session +i_guid = dates.get_loc(GUIDANCE) +i_cap = dates.get_loc(CAPITUL) + +# ── equilibrium (denoised) price ─────────────────────────────────────────────── +# Fundamental drift of an AI-memory franchise: strong but not manic. +mu_eq = 0.00055 +eq_shock = np.zeros(n) +eq_shock[i_guid:] = np.log(1 - 0.06) # −6% genuine re-rating at guidance +log_eq = np.log(100.0) + mu_eq * t + eq_shock \ + + np.cumsum(rng.normal(0, 0.0035, n)) # physical noise σ_τ only +P_eq = np.exp(log_eq) + +# ── institutional crowding κ_t (mean-field concentration, Level 1–2) ─────────── +# Logistic build-up through the AI-memory trade; institutions begin rotating +# out a few sessions BEFORE the guidance shock (they see the channel checks), +# and de-crowd hard after it. +kappa = 0.15 + 0.75 / (1 + np.exp(-(t - n * 0.55) / 18)) +i_rot = i_guid - 4 +decay = np.zeros(n) +decay[i_rot:] = np.linspace(0, 1, n - i_rot) ** 1.35 +kappa = kappa - 0.55 * decay * kappa +kappa = np.clip(kappa + rng.normal(0, 0.008, n), 0.05, 0.95) +# Post-capitulation: institutions buy the dip (the AlphaGBM observation) +kappa[i_cap:] += np.linspace(0, 0.05, n - i_cap) + +# ── retail flow f_t (LLM-homogenized momentum chase, Day 16) ─────────────────── +mom = pd.Series(log_eq).diff(20).fillna(0).to_numpy() +h = 0.85 # homogenization coefficient +f = h * np.tanh(35 * mom) + rng.normal(0, 0.06, n) +f[i_guid:i_cap] += 0.25 # retail buys the first dip… +f[i_cap:] = -0.9 + rng.normal(0, 0.05, n - i_cap) # …then capitulates at the low + +# ── behavioral wedge w_t ─────────────────────────────────────────────────────── +w = np.zeros(n) +a, b, c = 0.0030, 0.55, 0.0105 +for k in range(1, n): + drive = a * (kappa[k] + b * max(f[k], 0.0)) + jump = 0.0 + if rng.uniform() < 0.04 + 0.10 * w[k - 1]: # jumps cluster with crowding + jump = rng.choice([-1, 1], p=[0.35, 0.65]) * rng.exponential(0.006) + if k >= i_cap - 2: # forced unwind: 4 sessions + drive, c_eff = -0.055, 0.28 + jump = -abs(rng.normal(0.012, 0.006)) + else: + c_eff = c + w[k] = w[k - 1] + drive - c_eff * w[k - 1] + jump +w = np.clip(w, -0.08, 0.40) + +P = P_eq * (1 + w) +D = P / P_eq - 1 # divergence + +# ── the signal U_t ───────────────────────────────────────────────────────────── +THETA_D = 0.20 +kap_trend = pd.Series(kappa).ewm(span=10).mean().diff() +falling = (kap_trend < 0).rolling(3).sum() == 3 +armed = (pd.Series(D) > THETA_D) & falling.fillna(False) +sig_idx = armed[armed & (armed.index >= i_guid - 10)].index.min() +sig_date = dates[sig_idx] +lead_unwind = int(((dates > sig_date) & (dates <= UNWIND)).sum()) +lead = int(((dates > sig_date) & (dates <= CAPITUL)).sum()) + +peak = P.max() +trough = P[i_cap - 2:].min() + +print("=" * 70) +print("SYNTHETIC CONCEPT DEMO — denoised equilibrium price (To-C product line)") +print(f"scenario calendar : {dates[0].date()} → {dates[-1].date()} ({n} sessions)") +print(f"guidance shock : {GUIDANCE.date()} (Mode-I operator, eq −6%)") +print(f"signal U_t fires : {sig_date.date()} — D = {D[sig_idx]*100:+.1f}% above " + f"equilibrium, κ-trend negative 3 sessions") +print(f"lead time : {lead_unwind} trading days (≈2 weeks) before the " + f"{UNWIND.date()} unwind acceleration,") +print(f" {lead} trading days before the {CAPITUL.date()} capitulation") +print(f"peak → trough : {(trough/peak - 1)*100:.0f}% " + f"(market returns to its denoised track)") +print(f"at the trough : retail flow f = {f[i_cap+1]:+.2f} (panic sell) | " + f"institutional κ rising (buying the dip)") +print("=" * 70) + +# ── figure ───────────────────────────────────────────────────────────────────── +OK = dict(blue="#0072B2", orange="#E69F00", red="#D55E00", ink="#111827", + mute="#6B7280", grid="#E5E7EB") +plt.rcParams.update({"font.family": "sans-serif", "font.size": 10, + "axes.spines.top": False, "axes.spines.right": False, + "figure.facecolor": "white"}) +fig, axes = plt.subplots(2, 1, figsize=(13.2, 7.6), sharex=True, + gridspec_kw=dict(height_ratios=[1.15, 1.0], hspace=0.10)) +lo, hi = dates[0], dates[-1] + +ax = axes[0] +ax.plot(dates, P, color=OK["ink"], lw=1.5, label="market price $P_t$ (equilibrium + behavioral wedge)") +ax.plot(dates, P_eq, color=OK["blue"], lw=1.7, + label="denoised equilibrium price $P_t^{eq}$ (the product)") +ax.fill_between(dates, P_eq, P, where=P > P_eq, color=OK["orange"], alpha=0.25, + label="behavioral wedge $w_t$ (crowding, Thm 1)") +ax.axvline(sig_date, color=OK["orange"], lw=1.5, ls="--") +ax.axvline(UNWIND, color=OK["red"], lw=1.0, ls=":") +ax.axvline(CAPITUL, color=OK["red"], lw=1.4, ls="--") +ax.annotate(f"{sig_date.strftime('%b %d')} — signal: price {D[sig_idx]*100:+.0f}% above\n" + f"equilibrium while institutions rotate out.\n" + f"{lead_unwind} trading days (≈2 weeks) before the\n" + f"unwind accelerates on Jul 24.", + xy=(sig_date, P[sig_idx]), xytext=(dates[int(n*0.35)], peak * 0.915), + fontsize=9.5, fontweight="bold", color="#B45309", + arrowprops=dict(arrowstyle="->", color="#B45309", lw=1.2)) +ax.annotate("Jul 28 — capitulation:\nretail panic-sells at the low,\n" + "institutions buy; price re-joins\nits denoised track.", + xy=(CAPITUL, trough), xytext=(dates[int(n*0.46)], P_eq[0] * 0.985), + fontsize=9.5, fontweight="bold", color=OK["red"], + arrowprops=dict(arrowstyle="->", color=OK["red"], lw=1.2, + connectionstyle="arc3,rad=0.15")) +ax.axvline(GUIDANCE, color=OK["mute"], lw=0.9, ls=":") +ax.text(GUIDANCE - pd.Timedelta(days=3), 119.5, "guidance shock\nJul 13 (Mode-I)", + fontsize=7.8, color=OK["mute"], ha="right") +ax.set_ylabel("price (indexed)") +ax.legend(loc="upper left", fontsize=8.5, frameon=False) +ax.set_title("Denoised equilibrium price — SYNTHETIC concept demo of the To-C product line " + "(scenario shaped on the July 2026 memory unwind)", + fontsize=12.5, fontweight="bold", loc="left", pad=10) +ax.grid(alpha=0.3) + +ax = axes[1] +ax.plot(dates, D * 100, color=OK["ink"], lw=1.5, label="divergence $D_t = P_t/P_t^{eq}-1$") +ax.axhline(THETA_D * 100, color=OK["red"], ls=":", lw=1.3) +ax.text(dates[int(n*0.42)], THETA_D * 100 + 1.2, f"θ_D = {THETA_D*100:.0f}%", + fontsize=8.5, color=OK["red"]) +ax.fill_between(dates, THETA_D * 100, np.clip(D * 100, THETA_D * 100, None), + color=OK["orange"], alpha=0.45) +ax2 = ax.twinx() +ax2.plot(dates, kappa, color=OK["blue"], lw=1.3, alpha=0.85) +ax2.set_ylabel("institutional crowding κ_t", color=OK["blue"]) +ax2.tick_params(axis="y", labelcolor=OK["blue"]) +ax2.spines.top.set_visible(False) +ax.axvline(sig_date, color=OK["orange"], lw=1.5, ls="--") +ax.axvline(CAPITUL, color=OK["red"], lw=1.4, ls="--") +ax.set_ylabel("divergence (%)") +ax.set_xlim(lo, hi) +ax.legend(loc="upper left", fontsize=8.5, frameon=False) +ax.grid(alpha=0.3) + +fig.text(0.01, 0.005, + "SYNTHETIC DATA — a concept demo of the denoised-price product, not investment advice and not " + "a real-data hindcast. Signal U_t: D_t > θ_D while the EMA(10) κ-trend is negative 3 consecutive " + "sessions. Real-data version: experiment E7 (RESOURCES.md).", + fontsize=7.5, color=OK["mute"]) +fig.savefig(os.path.join(FIGS, "denoised_price_demo.png"), dpi=200, bbox_inches="tight") +print("✓ figures/denoised_price_demo.png") diff --git a/docs/JOURNEY.md b/docs/JOURNEY.md new file mode 100644 index 0000000..02ee41f --- /dev/null +++ b/docs/JOURNEY.md @@ -0,0 +1,155 @@ +# The Journey — From World-Model Skeptic to This Repository + +*HongJin HE · Alpha Flow Research · July 2026* + +*[README](../README.md) · [中文](../README_CN.md)* + +--- + +## I did not believe in world models + +For a long time I thought "world model" was a marketing term, and I had two +objections that felt unanswerable. + +**The precision objection.** Quantum mechanics puts a hard floor under how +precisely the state of a physical system can be known — and chaotic dynamics +amplify any microscopic uncertainty exponentially. If you cannot pin down the +state, how can you claim to model the world that generates it? + +**The storage objection.** Even granting perfect knowledge, a faithful +simulation of the world explodes combinatorially. Every naive estimate of +"simulate the environment" lands orders of magnitude beyond any hardware +roadmap. A world model, I concluded, was either a toy or a fantasy. + +I was wrong about both — but it took watching three fields succeed in +parallel to see why. + +## What changed my mind + +**Robots that dream.** The lineage that started with Ha & Schmidhuber's +[World Models](https://worldmodels.github.io/) (2018) — an agent learning +inside its own compressed dream of the environment — became Hafner's +[Dreamer](https://arxiv.org/abs/1912.01603) line, and by +[DreamerV3](https://github.com/danijar/dreamerv3) (2023) a single +configuration was mastering 150+ domains from pixels. The result that +genuinely shook me was +[DayDreamer](https://danijar.com/project/daydreamer/) (Wu, Escontrela, +Hafner, Abbeel & Goldberg, CoRL 2022): a *physical* quadruped learning to +walk in about one hour, because it practiced inside a learned latent model +instead of on its own legs. No atom of the robot's world was simulated. Only +what the task needed was kept. + +**Cars that predict.** Wayve's [GAIA-1](https://arxiv.org/abs/2309.17080) +(2023) and [GAIA-2](https://wayve.ai/thinking/gaia-2/) (2025) generate +coherent futures of driving scenes — not because they track every photon on +the road, but because they learned the *distribution of plausible futures* +at exactly the abstraction level where driving decisions live. Autonomous +driving stopped asking "what is the exact state of the world?" and started +asking "what happens next, at the resolution that matters?" + +**Agents that imagine.** DeepMind's [Genie](https://arxiv.org/abs/2402.15391) +(ICML 2024) through [Genie 3](https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/) +(2025) learn playable, interactive worlds from video; agents make decisions +by *imagining* rollouts. [NVIDIA Cosmos](https://github.com/NVIDIA/Cosmos) +(2025) industrialized the recipe into world foundation models for physical +AI. And Yann LeCun's position paper, +[A Path Towards Autonomous Machine Intelligence](https://openreview.net/forum?id=BZ5a1r-kVsf) +(2022), together with Meta's [V-JEPA 2](https://arxiv.org/abs/2506.09985) +(2025), articulated the principle underneath all of it: **predict in +representation space, not in pixel space**. You do not model the world. +You model the *sufficient statistics* of the world for the decisions you +need to make. + +That principle dissolved both of my objections at once: + +- The **storage objection** dies because compression is the whole point. + A world model is not a simulation of the world; it is the smallest state + that makes the future predictable. (In this repo, that state is 5 + dimensions per asset plus a distribution — not a tick-by-tick replay.) +- The **precision objection** dies because prediction never needed + microscopic precision. Boltzmann could not track a single molecule, and + kinetic theory works anyway: at the population level, dynamics become + *more* lawful as N grows, not less. Quantum indeterminacy lives twenty + orders of magnitude below the level where any decision — a lane change, a + portfolio weight — actually happens. + +I was an exchange student at Stanford while much of this was in the air — +the [spatial intelligence](https://www.ted.com/talks/fei_fei_li_with_spatial_intelligence_ai_will_understand_the_real_world) +conversation around Fei-Fei Li and [World Labs](https://www.worldlabs.ai/) +made it feel less like a research direction and more like a consensus +forming in real time: *world models are how agents will understand +everything*. The question I could not put down was: **everything — except +markets?** + +## The second thread: a violation of first principles + +At the same time I was working through quantitative finance coursework and +reading the empirical asset-pricing literature, and I hit something I still +find astonishing: **the dominant paradigm of quantitative finance is not +even trying to predict the market.** + +Factor models ask which characteristics *correlate* with cross-sectional +returns. The literature documented 600+ of them — Cochrane called it the +["factor zoo"](https://onlinelibrary.wiley.com/doi/10.1111/j.1540-6261.2011.01671.x) +in his 2011 AFA presidential address — and when +[Harvey, Liu & Zhu](https://academic.oup.com/rfs/article/29/1/5/1843824) +audited the zoo, most factors failed to replicate. +[López de Prado](https://www.wiley.com/en-us/Advances+in+Financial+Machine+Learning-p-9781119482086) +catalogued the same pathology from inside the industry: backtest overfitting +as standard practice. + +From first principles this is upside down. A model of the market should +model *the market* — the thing that generates prices — not mine correlations +from the residue prices leave behind. History's data does not repeat: every +factor decays the moment it is crowded (the Lucas critique, measured). +But history itself — the *structure* that generates the data: institutions, +constraints, incentives, the game — does repeat. If you want factors that +mean something, you have to go back to the world that produces them. + +That sentence, I realized, had a name. It was a world model. + +## The merge + +Two threads, one collision: + +- Robotics, driving, and agent research proved that **compressed world + models predict well** at the abstraction level that matters. +- Quantitative finance was stuck mining patterns precisely because it had + **no world model** — no representation of the thing generating the data. + +But markets add one twist that Dreamer never faced: **the "physics" of a +market is other agents' strategies.** A road does not replan when GAIA-1 +predicts it; a market does — every fund that discovers a pattern destroys +it by trading on it. So the transition network of a financial world model +cannot be learned dynamics; it must be an **equilibrium solver**. The +environment *is* the fixed point of every agent's best response to everyone +else. + +That single substitution — replace the learned transition model with a +hierarchical mean-field game — is this entire repository. The encoder +compresses the market panel into a latent state (the JEPA lesson). The Game +module solves for the equilibrium instead of extrapolating history (the +Lucas lesson). The controller acts on the equilibrium drift (the Dreamer +lesson). And the residual that no data can remove — behavioral noise — is +not swept under the carpet but bounded by theorem +([Theorem 1](../README.md#component-2--dual-noise-decomposition-theorem-1), +the dual Cramér-Rao bound): the honest descendant of my old precision +objection. + +## What I believe now + +World models are not a technique that happens to work in robotics. They are +the first-principles form of prediction in any domain: find the level of +description at which dynamics are lawful, compress to it, and model the +generator — not the residue it leaves in a dataset. Finance was simply the +domain where nobody had done it yet, because in finance the generator +fights back. + +MicroWorld is the bet that modeling the generator anyway — as a game, with +proofs — is worth more than one more decade of factor mining. + +--- + +*Back to the [README](../README.md) · the mathematics starts +[here](../README.md#the-mathematical-framework-two-threads-one-theory) · +what comes after the mathematics is [Phase 2](PHASE2_NEURAL_GAME.md).* diff --git a/docs/ONE_PAGER.md b/docs/ONE_PAGER.md new file mode 100644 index 0000000..38410eb --- /dev/null +++ b/docs/ONE_PAGER.md @@ -0,0 +1,70 @@ +# MicroWorld — One Pager + +**A world model for equity markets: model the players, not the patterns.** +*Alpha Flow Research · HongJin HE · HKUST / Stanford IHP · July 2026* +*[github.com/hongjin-he/MicroWorld](https://github.com/hongjin-he/MicroWorld) · MIT license* + +--- + +**The problem.** Quantitative finance's dominant paradigm — factor mining +and time-series ML — extracts patterns from historical data. But historical +*data* does not repeat: every discovered signal is destroyed by its own +adoption (alpha half-life: ~6 years in 1990 → ~11 months in 2023). What +does repeat is the *structure* that generates the data: institutions, +regulations, incentives, the game. + +**The approach.** MicroWorld models that structure directly: US equity +markets as a four-level hierarchical mean-field game — cross-market flows, +institution types, individual institutions, intra-institution desks — with +a 5-dimensional state space per asset, a dual decomposition of noise into +physical and behavioral components, and a 22-operator event algebra for +M&A/IPO/policy shocks. Predictions are equilibria, not extrapolations, so +they are designed to survive their own deployment. Seven theorems proved; +50 tests passing; every demo runs on a laptop with zero API keys. + +**Validation to date.** +- **2008 hindcast, public data only:** the stability indicator Λₜ entered + its sustained crisis regime on Aug 16, 2007 — **272 trading days before + Lehman** — with zero false alarms in 2005–06. Same signal: Feb 20, 2020. +- **Synthetic product demo:** denoised equilibrium price flags a + crowding-driven divergence ~2 weeks before a July-2026-shaped sector + unwind ([demo](../demo/denoised_price_2026.py)); real-data version is + specified as experiment E7. +- **Open-source traction:** 138+ GitHub stars in the first weeks, 17-notebook + tutorial series, interactive 3D market universe. + +**Two products, one engine.** +- **To-C — the denoised price.** The MFG equilibrium track P^eq and the + divergence D_t = P/P^eq − 1: a mid/long-horizon positioning research + signal for retail ("are you buying value or buying crowding?"). Not + personalized advice; an instrument-level research layer. +- **To-B — structural risk early warning.** Λₜ regime monitoring, crowding + decomposition, and event-operator scenario analysis for funds and risk + desks — the 2008-grade signal, live (pipeline scaffolded: Airflow + + Alpaca paper-trading). + +**The moat.** Causal explainability. Factor and time-series shops find +patterns they cannot explain and therefore cannot defend when regimes +break. Every MicroWorld output is attributable to agents, constraints, and +equilibrium conditions — the model's explanation *is* the model. The +framework is also the only one simultaneously offering strategic agents, +universe-changing events, noise decomposition with an estimation bound, +crisis early-warning, and a four-level hierarchy. + +**Roadmap.** Phase 1 (now): real-data validation E1–E7 → NeurIPS-2026 +workshop paper + the E7 product exhibit (~one quarter, ~$25k inc. compute). +Phase 2: the Neural Network Game Structure — every neuron an agent-network, +regulatory constraints as architecture ([design doc](PHASE2_NEURAL_GAME.md)); +pilots on one H100/H200, full campaign on an 8×H200 node (~$300k scenario). +Detail: [RESOURCES.md](../RESOURCES.md). + +**The ask.** Data access (WRDS-grade), compute (H100/H200 hours), or +pre-seed partnership — priced by scenario in RESOURCES.md. + +**Contact.** [LinkedIn](https://www.linkedin.com/in/hongjinhe-hkust-edu) · +[GitHub](https://github.com/hongjin-he) · [X](https://x.com/Mr_Abstractor) + +--- + +*This document describes research software. Nothing here is investment +advice or an offer of securities.* diff --git a/docs/PHASE2_NEURAL_GAME.md b/docs/PHASE2_NEURAL_GAME.md new file mode 100644 index 0000000..a46229c --- /dev/null +++ b/docs/PHASE2_NEURAL_GAME.md @@ -0,0 +1,171 @@ +# Phase 2 — The Neural Network Game Structure (NNGS) + +**Status: design document.** Nothing in this file is implemented, and that +is deliberate. Phase 2 begins when two resources exist that do not exist +today: the cleaned multi-type data panel and H200-class training compute +(see [RESOURCES.md](../RESOURCES.md)). Until then, this document is the +specification we hold ourselves to — written down in public, before the +hardware arrives, so the idea has a timestamp. + +*[README](../README.md) · [The journey that led here](JOURNEY.md)* + +--- + +## 1 · Why Phase 1 is a game-theory engine, not a neural network + +Phase 1 — everything currently in this repository — solves the market as a +hierarchical mean-field game: coupled HJB–FPK systems, an operator algebra +for events, seven proven theorems. It is deliberately *mathematical +structure first, learning second*, for one reason: **data scarcity is not a +temporary inconvenience; at today's access level it is binding.** When you +cannot estimate millions of parameters, you must get the same behavior from +structure — and Theorem 1 (the dual Cramér-Rao bound) says something +stronger: part of the residual can *never* be estimated away with more +data, only explained by a model of the mechanism. Phase 1 is the +mechanism, written in PDEs because PDEs are what run on a laptop. + +So the honest description of the current stage is: **the game-theoretic +core is the data-efficient regime of the world model.** It is not the +final form. + +## 2 · The Phase 2 thesis: neurons that play, not neurons that fire + +Everyone else who models markets builds a mathematical model *of* the +agents. Phase 2 builds the agents *as the network*: + +> **A neural network in which each neuron is itself a small neural +> network — one per institution or retail cohort — with its own objective, +> its own information set, and its own strategy. The connections between +> neurons are not weights that passively mix signals; they are strategic +> couplings: each unit's forward pass is a best response to the others. +> We call this the Neural Network Game Structure (NNGS).** + +The units are, in a precise sense, *personified*: a neuron here does not +"fire", it *decides*. Where a GNN node aggregates its neighbors' messages, +an NNGS node responds to its neighbors' strategies — the difference between +diffusion and game play. This is the same distinction that separates this +repo from swarm models (see the MicroFish comparison in the README), now +pushed down into the architecture itself. + +### Two representational choices, both admissible + +1. **Agents-as-neurons (the primary design).** Each of the ~11 agent + classes (6 institutional + 5 retail, [`agents/`](../agents/)) is + instantiated as a population of small networks, fully connected across + the graph, each carrying its own recurrent state. +2. **Levels-as-layers (the fallback).** The four levels L0–L3 become four + layers of one large network, with within-layer competition expressed via + lateral connections. Coarser, cheaper, less faithful — but trainable + sooner. + +These are two encodings of the same object; which one wins is an empirical +question (experiment **E9** below), not an aesthetic one. + +## 3 · The constraint structure — why this is not a free-form black box + +An unconstrained network-of-networks would forfeit the one thing Phase 1 +paid for: interpretability. NNGS keeps it through **constraints that are +facts about the world, imposed as architecture**: + +- **Regulatory weight sharing.** All agents of one type live under the same + regulator — every bank under Basel III, every mutual fund under UCITS/40-Act + limits, every insurer under Solvency-style capital rules. Architecturally: + agents of a type share a constraint module (projection layer onto the + feasible set), exactly as the Phase 1 taxonomy shares Merton-style + constraint sets. The shared module *is* the regulator. +- **Budget and leverage feasibility** as hard projection layers, not soft + penalties — an agent cannot learn its way out of a balance sheet. +- **Information stratification** ([`state/information.py`](../state/information.py)): + each neuron sees only its type's filtration. A retail cohort cannot + attend to order-flow features it would not observe in reality. +- **The mean-field anchor.** The Phase 1 equilibrium is retained as a + regularizer: population-level statistics of NNGS play must stay within a + Wasserstein ball of the MFG equilibrium μ*, unless data demands + otherwise. Phase 1 becomes the prior; Phase 2 the posterior. + +The result is a *constrained* neural network whose every departure from +equilibrium is attributable — to an agent type, a constraint, or an +information set. Explanations survive the scaling-up. + +## 4 · One network or two? — an experiment, not a debate + +Two candidate macro-architectures: + +| | **A — Unified** | **B — Dual (environment + population)** | +|---|---|---| +| Structure | One network: agents and environment dynamics entangled in a single graph | Network 1: a trained environment/world model (the neuralized **E**). Network 2: the population of agent-neurons playing inside it (the neuralized **Game**) | +| Lineage | End-to-end world models (Dreamer-style) | E-Game-C itself — B is its native neuralization | +| Risk | Attribution becomes murky; environment leaks into strategy | Interface mismatch: the population's actions must feed back into the environment model consistently (reflexivity must close the loop) | +| Prior | — | Phase 1 MFG initializes Network 2; the Phase 1 encoder initializes Network 1 | + +Design B is the default because it inherits E-Game-C directly and keeps the +reflexivity loop explicit (price-belief feedback as the interface between +the two networks). But the choice is assigned to experiment **E8**, run +first at toy scale — the answer we publish will be measured, not asserted. + +## 5 · The complexity wall — and the four tools against it + +Honesty first: training a model whose parameters are themselves models +roughly **squares the training difficulty**. N agents with pairwise +strategic coupling is O(N²) in interactions before anything recurses, and +each unit's learning signal depends on every other unit's current policy — +the non-stationarity that makes multi-agent RL notoriously unstable, at a +scale multi-agent RL has not attempted. This is exactly why Phase 2 waits +for hardware, and why the design leans on four reductions: + +1. **Mean-field factorization.** Within a type, agents couple to the + *distribution* of their type, not to each individual — O(N²) → O(N·K) + for K types. This is not an approximation bolted on: Phase 1 *proved* + (Theorem 7.4, propagation of chaos) that it is the correct N→∞ limit. +2. **Latent agent embeddings.** Each neuron's policy conditions on a + low-dimensional embedding of its identity (d ≈ 16–64), so populations + share one policy network modulated per agent — the DreamerV3 trick of + one configuration spanning many domains, applied within one market. +3. **Hierarchy as curriculum.** Train L1 (type-level) frozen-environment + first, unfreeze L2 (institution-level), then L3 — the four-level + structure is not just descriptive, it is the training schedule. +4. **Sparse strategic attention.** Full connectivity is the specification, + not the runtime: a learned top-k attention over counterparties captures + the empirically sparse strategic graph (a fund responds to its actual + competitors, not to all 50,000 institutions). + +## 6 · Compute and data triggers + +- **Compute.** The E8/E9 toy-scale ablations run on a single H100/H200. + A credible full-scale L1+L2 NNGS train (≈10³ neuron-networks, each + 10⁵–10⁶ parameters, adversarial co-training) is an H200-cluster problem + — the training-run budget lives in [RESOURCES.md](../RESOURCES.md). +- **Data.** The cleaned, typed panel of DATA_REQUIREMENTS.md (13F/COT + positioning for institutional ground truth, the E5 LLM-query atlas for + retail policy priors). NNGS without agent-level ground truth would be a + simulator, not a world model. + +## 7 · Phase 2 experiments + +Continuing the E-numbering from [DATA_REQUIREMENTS.md](../DATA_REQUIREMENTS.md) +(E1–E6) and [RESOURCES.md](../RESOURCES.md) (E7): + +| ID | Experiment | Question it settles | Scale | +|---|---|---|---| +| **E8** | Unified vs dual architecture on the synthetic market ([`demo/synthetic_market.py`](../demo/synthetic_market.py)) | One network or two? | 1 GPU | +| **E9** | Agents-as-neurons vs levels-as-layers, same data, same budget | Which encoding of the game? | 1 GPU | +| **E10** | MFG-distillation: initialize NNGS from the Phase 1 equilibrium vs cold start | Is Phase 1 a useful prior (we predict: decisively yes)? | 1 GPU → cluster | +| **E11** | Constrained vs unconstrained NNGS on real panel | Do the regulatory constraints help or hurt fit? (The thesis: they *are* the alpha) | cluster | + +## 8 · The larger claim + +If NNGS works for markets, nothing about it is market-specific. Any +real-world graph whose nodes have objectives — supply chains, electricity +markets, ecosystems of platforms, geopolitical blocs — admits the same +construction: **model each node as a small neural network with a stake, +and the graph as their game.** Markets are simply the best first target: +the players are catalogued, the constraints are written law, and the +scoreboard prints every millisecond. + +That is the Phase 2 bet. The mathematics of Phase 1 is how we earn the +right to place it. + +--- + +*Questions, objections, or compute to offer: see +[Partnerships](../README.md#partnerships--contact).* diff --git a/docs/i18n/README_JA.md b/docs/i18n/README_JA.md new file mode 100644 index 0000000..6440f54 --- /dev/null +++ b/docs/i18n/README_JA.md @@ -0,0 +1,104 @@ +