This study asks whether survival models can replace the fixed aggregation window in HMM-based regime-aware execution with a per-instance prediction of remaining bearish regime duration. The answer is no, and this repository documents precisely why.
This is Paper III in a trilogy. Paper I showed flat RL cannot exploit regime signals even with the true label in the state space. Paper II showed those signals require multi-day aggregation before they predict execution quality. This paper tests the natural next step: using the regime duration to calibrate that aggregation window adaptively.
- Can Weibull AFT survival models predict remaining bearish regime duration from HMM-derived covariates at regime start?
- Is the failure a data quantity problem (solvable with more instances) or a covariate adequacy problem (structural)?
- Do adaptive windows calibrated to predicted duration outperform the fixed W = 10 baseline from Paper II?
| Asset | Model type | C-index | Weibull shape |
|---|---|---|---|
| SPY | Per-asset | 0.200 | 0.711 |
| QQQ / IWM | Pooled (equity idx.) | 0.384 | 0.288 |
| BTC-USD | Pooled (crypto) | 0.376 | 0.149 |
| ETH-USD | Per-asset | 0.390 | 0.226 |
| GLD | Median fallback | — | — |
| TLT | Median fallback | — | — |
| AAPL | Per-asset | 0.315 | 0.129 |
C-index = 0.50 is chance; C-index > 0.70 is conventionally useful. All fitted models fall below chance. All Weibull shape parameters are below 1.0.
Core result: Survival models achieve C-indices of 0.20–0.39, uniformly near or below chance. A without-replacement subsampling simulation shows C-index remains flat at ≈ 0.48 from n = 4 to n = 45 training instances , ruling out data scarcity as the cause. Weibull shape parameters below 1.0 in every asset class produce decreasing-hazard distributions whose mean durations exceed the window cap, collapsing 60–89% of predictions to boundary values regardless of covariate values. There are no genuine adaptive wins over the fixed W = 10 baseline.
git clone https://github.com/satishgarg/regime-survival-execution
cd regime-survival-execution
pip install -r requirements.txtpython run.py # full pipeline: fits HMM, extracts instances, fits survival
# models, runs all analyses, writes results/ and figures/All data is downloaded automatically via yfinance (internet required). The five-year window (2020-01-01 to 2024-12-31) is fixed; results are reproducible against a specific historical period rather than a rolling window. Expected runtime: ~15–25 minutes on a standard CPU.
To reproduce individual analyses:
python survival_model.py # Table 2: C-index and Weibull shape per asset
python adaptive_execution.py # Table 3/4: window distribution and main result
python analysis.py # Tables 5–9: robustness, subgroup, simulation├── data/
│ ├── hmm_results.pkl # Fitted HMM and regime assignments
│ ├── regime_instances.csv # All 81 bearish instances (train + test)
│ ├── regime_instances_train.csv # 54 training instances (start ≤ 2022-12-31)
│ └── regime_instances_test.csv # 27 test instances (start ≥ 2023-01-01)
│ └── survival_models.pkl # Fitted Weibull AFT and fallback models
├── figures/
│ ├── fig1_duration_distributions.png
│ ├── fig2_survival_curves.png
│ ├── fig3_adaptive_window_dist.png
│ ├── fig4_window_comparison.png
│ ├── fig5_subgroup_analysis.png
│ ├── fig6_covariate_heatmap.png
│ └── fig7_sample_size_simulation.png
├── results/
│ ├── table1_survival_fit.csv
│ ├── table2_covariate_coefs.csv
│ ├── table3_window_comparison.csv
│ ├── table4_subgroup.csv
│ ├── table5_sample_size_simulation.csv
│ ├── survival_params.csv
│ ├── window_comparison.csv
│ ├── alternative_distributions.csv
│ ├── censoring_sensitivity.csv
│ ├── extended_covariate_models.csv
│ ├── robustness_bootstrap.csv
│ ├── subtype_models.csv
│ └── univariate_models.csv
├── adaptive_execution.py
├── analysis.py
├── config.py
├── data_and_hmm.py
├── plotting.py
├── regime.py
├── regime_instances.py
├── run.py
├── survival_model.py
├── survival_utils.py
└── requirements.txt
regime.py: 4-state Gaussian HMM fitting with BIC-based covariance
selection, state identification by return and trend score, and minimum-duration
smoothing to suppress single-day regime switches. Identical procedure to
Paper II except fitted over the full five-year window (2020–2024) rather
than one year, to accumulate sufficient bearish regime instances for
survival analysis.
data_and_hmm.py: Downloads daily OHLCV for all eight assets via
yfinance, fits the HMM, and computes per-day covariates (posterior entropy,
stay probability) and execution cost differences (regime-aware minus TWAP).
regime_instances.py: Extracts bearish regime instances from the
collapsed HMM label sequence, applies right-censoring rules (active at
sample end; exceeding 120-day threshold), and splits into train/test sets
by start date.
survival_model.py: Fits the Weibull AFT model hierarchy (per-asset
where n ≥ 8, pooled within asset class otherwise, median fallback for GLD
and TLT). Reports C-index and shape parameters. Also fits alternative
parametric distributions (log-normal, log-logistic) and Cox PH for
Section 5.4.
adaptive_execution.py: Generates per-day adaptive windows from
survival model predictions, computes variable-window rolling cost
differences, and evaluates Spearman correlation against entropy for all
fixed and adaptive windows over the 2023–2024 test period.
analysis.py: All robustness analyses: bootstrap stability (Section 5.1),
execution model sensitivity (Section 5.2), censoring threshold sensitivity
(Section 5.3), subgroup analysis (Section 4.5), sample-size simulation
(Section 4.6), univariate models, crash vs. bearish subtypes, and extended
OHLCV covariates.
survival_utils.py: Shared utilities for survival model fitting,
C-index computation, and without-replacement subsampling.
plotting.py: All seven publication figures.
config.py: Central configuration , asset list, date ranges, window
bounds, censoring threshold, model hierarchy thresholds.
run.py: Entry point. Runs the full pipeline end-to-end and writes
all outputs to results/ and figures/.
Eight assets via yfinance, fixed five-year window (2020-01-01 to 2024-12-31):
| Asset | Class | n (days) | Train instances | Test instances |
|---|---|---|---|---|
| SPY | Equity index ETF | ~1,237 | 10 | 6 |
| QQQ | Equity index ETF | ~1,237 | 5 | 1 |
| IWM | Equity index ETF | ~1,237 | 3 | 1 |
| BTC-USD | Cryptocurrency | 1,806 | 5 | 0 |
| ETH-USD | Cryptocurrency | 1,806 | 14 | 3 |
| GLD | Commodity ETF | ~1,237 | 5 | 5 |
| TLT | Bond ETF | ~1,237 | 4 | 8 |
| AAPL | Large-cap equity | ~1,237 | 8 | 3 |
Train/test split on instance start date: train ≤ 2022-12-31, test ≥ 2023-01-01. BTC-USD contributes no test instances; the pooled crypto model's test-period evaluation reflects ETH-USD instances exclusively.
All HMM fitting is in-sample over the full five-year window. Regime assignments in the test period (2023–2024) therefore reflect information from the full sample, which may overstate regime clarity relative to a production setting. Survival model evaluation uses a genuine temporal train/test split, but reported results should be treated as upper bounds on prospective predictive power. The sample-size simulation is bounded by the 54 available training instances and cannot extrapolate beyond n = 49.
This is Paper III in a trilogy:
Paper I: Garg, S. (2025). Regime Awareness in RL for Optimal Trade Execution: A Simulation Study. SSRN preprint 6559598.
https://papers.ssrn.com/abstract=6559598
Shows flat PPO cannot exploit regime signals even with the true label in the state space.
Paper II: Garg, S. (2025). When Do Regime Signals Work? HMM Uncertainty and Trade Execution Across Asset Classes. SSRN preprint 6733198.
https://papers.ssrn.com/abstract=6733198
Shows HMM uncertainty signals require 3–10 day aggregation before they predict execution quality; establishes the W = 10 baseline tested here.
- Amrouni et al. (2022): CTMSTOU simulation environment (JP Morgan AI Research)
- lifelines: Survival analysis in Python
- hmmlearn: HMM fitting
- yfinance: Market data