A complete, walk-forward-validated machine-learning pipeline that asks one question:
Can a model systematically pick 21-day winners from a basket of AI/semiconductor stocks — and does it beat simply holding the sector ETF?
The pipeline runs end to end (data → features → model → backtest → portfolio → options overlay) and is benchmarked honestly. The measured answer is no: the model's apparent performance is AI-sector beta, not stock-selection alpha.
| Metric | Model | Passive SMH (buy & hold) |
|---|---|---|
| Sharpe | 1.12 | 1.23 |
| Max drawdown | −25.8% | −18.6% |
| Rank IC (selection skill) | −0.021 ≈ 0 | — |
The final model posts a backtest Sharpe of 1.12 — a number that looks like success — but a one-click passive SMH hold beats it on a risk-adjusted basis with a shallower drawdown. The model's stock-selection skill, measured by Rank IC, is statistically zero. The apparent returns are sector beta amplified by position concentration, not alpha. Full numbers: results/RESULTS.md.
Hypothesis. AI/semiconductor names trend hard. If a model could rank which names will outperform over the next 3–4 weeks, those moves could be captured systematically.
v1 — cross-sectional ranker (LightGBM LambdaRank). Trained to rank ~60 names by 21-day forward return. Result: Rank IC −0.017 — worse than random. The names are too correlated (0.6–0.8 pairwise); there is little to rank between them.
v2 — per-stock binary classifier (XGBoost). Reframed as "P(this stock is up >3% in 21 days?)", trained on a wider 150-name, 7-sector universe to break the sector-only bias, with features curated by Information Coefficient (RSI/MACD/VWAP dropped — AUC ≈ 0.50, noise at this horizon). Result: a believable Sharpe 1.12 and positive trade stats — but Rank IC still −0.021.
Benchmark test. Compared to passive alternatives over the identical window with identical costs, SMH buy-and-hold won. The model's Sharpe was sector beta, amplified by 5-name concentration — not selection skill.
Conclusion. Systematic 21-day stock selection in a tightly-correlated AI universe, using freely-available data, is at or beyond the limit of predictability. The only genuine signal the model found was market-regime timing (VIX, yield curve, trend), not stock selection.
| Finding | Evidence |
|---|---|
| No stock-selection edge at the 21-day horizon | Rank IC −0.021, IC 0.003 (both ≈ 0), across two architectures |
| Apparent Sharpe is sector beta, not alpha | Model Sharpe 1.12 < SMH 1.23; deeper drawdown (−25.8% vs −18.6%) |
| Short-term technicals are noise at 21d | RSI/MACD/VWAP all AUC ≈ 0.50 in IC analysis |
| Predictive signal is macro/regime, not stock-specific | Feature importance dominated by VIX, yield-curve slope, regime score |
| Annualization bug caught and fixed | Period-vs-day error inflated Sharpe to a false 6.17; corrected to 1.12 |
- Universe correlation. 0.6–0.8 pairwise correlation among the names leaves almost no idiosyncratic return to predict — cross-sectional selection has nothing to grip.
- Horizon dead zone. 21 days is too long for microstructure/momentum signals and too short for fundamental factors to express in price.
- Look-ahead in fundamentals.
yfinance.inforeturns a current snapshot; broadcasting it across history leaks future knowledge. Even with this advantage the IC was ≈ 0 — the true point-in-time result would be worse. - Survivorship + single regime. The names are today's winners over a 2022–2026 window dominated by one AI bull market; the model never saw a tech bear.
- Point-in-time fundamentals (CRSP/Compustat) to remove the look-ahead leak
- Survivorship-free universe including delisted names
- Diversified, lower-correlation universe so selection has real dispersion
- Longer horizon (60d) or residualized targets (excess return vs sector ETF) to strip beta and isolate alpha
- Operationalize the one real signal — a regime-timed ETF strategy
- Walk-forward time-series CV — 24m train / 3m validation / monthly retrain → 45 out-of-sample folds, no look-ahead in the split
- ~108 engineered features across 6 families (price, technical, volume, volatility, fundamental, macro/regime, calendar), curated to ~33 by Information Coefficient
- Two model architectures — LightGBM LambdaRank ranker vs XGBoost binary classifier with isotonic probability calibration
- Realistic backtester — 15 bps cost + 10 bps slippage + drag, period-correct annualization, Sharpe/Sortino/Calmar/drawdown, IC/Rank IC/AUC
- Options-strategy engine — rule-based mapping of (direction, vol, IV regime, earnings proximity) → 7 strategies (bull call/put spreads, PMCC, short strangle, iron condor, earnings iron condor, calendar) with simplified Black-Scholes P&L
- Risk modules — rule-based regime detector + Kelly position sizer with caps
- 50+ unit tests covering features, metrics, strategy routing, and the metrics fix
Python · pandas · numpy · scikit-learn · XGBoost · LightGBM · scipy ·
yfinance · pyarrow
.
├── data/ # universe definitions + cached yfinance fetcher
├── features/ # 6 feature families + IC-curated feature set
├── models/ # XGBoost classifier, RF vol model, regime detector,
│ # Kelly sizer, 7-strategy options selector
├── backtest/ # walk-forward CV, portfolio sim, options overlay,
│ # cost model, metrics (IC, Sharpe, AUC, drawdown)
├── reports/ results/ # report generation + committed example outputs
├── tests/ # 50+ unit tests
├── train.py # fit model
├── predict.py # daily inference → picks + options recommendations
├── run_backtest.py # full walk-forward backtest
├── analyze_features.py # feature predictivity analysis (IC / AUC / MI)
└── benchmark.py # model vs passive buy-and-hold
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python tests/test_v2.py # unit tests
python run_backtest.py --years 5 # full backtest → output/backtest_report.md
python benchmark.py --years 5 # model vs SMH / SPY / equal-weight
python analyze_features.py # feature IC / AUC rankingResearch and educational project. Not investment advice. Backtested performance does not predict future results and — as this study demonstrates — can be misleading without proper benchmark validation.