wcpredict

中文文档: README.zh.md

2026 FIFA World Cup match predictor. Home / draw / away probabilities backed by a native Elo engine, XGBoost, and a Davidson ordered-logit ensemble — with live Elo updates between matches and honest 4-tournament LOTO (Leave-One-Tournament-Out) backtesting.

Two things it does:

• Rank teams — a native elo_own engine tuned against eloratings.net and FIFA, refreshed daily and reconciled with both external sources side-by-side.
• Predict matches — XGBoost + Davidson blend produces H/D/A probabilities for every 2026 WC fixture, per model, cached and refreshed as ESPN updates the schedule.

Screenshots

Snapshot taken 2026-06-30 from live data.

Top-N rankings — elo_own vs eloratings.net vs FIFA

2026 WC match table with per-model probabilities

Why wcpredict

Public World Cup predictors are usually one of two things: a static Elo table (fine, but never explains itself), or a black-box XGBoost that was trained once and never retuned. wcpredict tries to be honest about both:

• Multi-source Elo triangulation — the three columns on the ELO page (elo_own / eloratings.net / FIFA) sit side-by-side so you can see exactly where they disagree. elo_own's six hyperparameters (k_base, home_advantage, sigmoid_divisor, gd_blowup_step, new_team_boost_*) are Optuna-tuned to minimize rank-correlation loss against the two external references.
• XGBoost + Davidson ensemble — 44 hand-engineered features (Elo differentials, form, rest days, FIFA-points diff, shootout history) feed XGBoost; a Davidson ordered logit calibrates draw probability separately. Ensemble weights (w_xgb, w_fifa) are fit on val folds, not guessed.
• Multi-algorithm stacking — LightGBM, CatBoost, Random Forest, and OrderedLogit run alongside XGBoost through a shared wrapper interface. compare-models lines up all N on the 2026 WC holdout; train-stacking fits simplex blend weights on val for the final predictor.
• Post-hoc calibration — temperature scaling + draw boost is a first-class forge phase, so probabilities are honest and not just discriminative.
• 4-WC LOTO backtest — the LOTO framework holds out each of the last four World Cups in turn, retrains on the remaining three, and reports logloss / RPS / accuracy — with a frozen-vs-online-Elo A/B split in the same run.
• Live online updates during the tournament — as each 2026 match finishes, live-update atomically appends to results.csv, refreshes the elo_own cache, and invalidates the prediction cache. Frozen and online ratings are both scored so the online lift is measurable.
• SHAP + LLM explanation per match — every prediction ships with the Top-3 TreeSHAP drivers, translated to natural language by a configurable LLM provider (with a bet365 expected-value calculator in the same modal).

Features

• ELO — Top-N rankings across elo_own, eloratings.net, and FIFA, with per-source last-updated timestamps
• Predict — Every 2026 WC fixture with actual result · worldcupflow baseline · wcpredict-per-model probabilities; scored on finished matches by logloss / RPS / accuracy
• Group standings — Live group tables driven by finished matches
• Bracket — Deterministic single playout showing the argmax path to the final
• Championship odds — 10,000-run Monte Carlo marginalizing over all paths; recomputes when the schedule changes
• Match detail modal — Top-3 SHAP drivers, cross-model comparison, bet365 EV calculator, LLM natural-language explanation
• Live online updates — atomic live-update writes finished match into csv + refreshes Elo + invalidates the prediction cache

Quickstart

Requires uv:

# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# or: brew install uv

Scenario 1 — Use the shipped model (just serve)

For anyone who only wants to see the site running against the pre-trained model already in the repo. No training, no external data ingest.

git clone https://github.com/xiapuyang/worldcup
cd worldcup
uv venv --python 3.11
source .venv/bin/activate
uv sync --group dev

# Backend + frontend, then open http://localhost:5026 in a browser
uv run python serve.py

Script mode (background, logs to logs/):

scripts/run.sh      # start (also opens http://localhost:5026 in your browser)
scripts/stop.sh     # stop
scripts/restart.sh  # restart

Scenario 2 — Retrain the XGBoost model from scratch

The default algorithm is XGBoost. The pipeline is: tune Elo → train → LOTO backtest → publish predictions. See docs/cli-cookbook.md for the full recipe; the essentials:

# 0) Cold-start caches (one time only)
#    Historical CSVs are already vendored in data/raw/ — no copy needed.
#    Bulk-pull eloratings + FIFA caches for ~211 teams.
uv run wcpredict fetch-external

# 1) Tune the Elo engine's 6 hyperparameters against eloratings.net + FIFA.
#    Downstream features depend on this — rerun only when the Elo engine or data changes.
uv run wcpredict tune-elo --n-trials 200
cp data/elo_tuning/elo_params_*.json config/elo_params.json   # pick the latest

# 2) Smoke-test the full forge pipeline first (~5 min, deterministic).
uv run wcpredict train --smoke --task-id smoke-001

# 3) Real training run. The last log line prints the auto-generated task-id.
uv run wcpredict train --trials 50 --objective logloss --calibration temperature

# 4) Honest generalization backtest: retrains once per held-out WC.
TASK=<paste-task-id-from-log>
uv run wcpredict loto-report --task-id $TASK --retrain

# 5) Batch-predict the entire 2026 WC — this writes the cache the frontend reads.
uv run wcpredict predict-wc2026 data/models/active/$TASK

# 6) One-off match prediction.
uv run wcpredict predict Brazil Argentina --date 2026-07-04 --task-id $TASK

Main XGBoost training flags (all on uv run wcpredict train):

Flag	Options	Default	What it controls
--algorithm	xgboost / lightgbm / catboost / random_forest / ordered_logit	xgboost	Model wrapper. Leaving it at xgboost reproduces the shipped baseline; the other four exist for compare-models / stacking (see docs/cli-cookbook.md).
--trials	integer 1-500	20	Optuna searches over 8 XGBoost hyperparameters + sample-weight time-decay lam. 50 is a solid production setting; use 20-30 for daily cron.
--objective	logloss / rps	logloss	Optuna's search objective. logloss for calibrated probabilities; rps if you care more about the ordinal shape of the H/D/A distribution.
--calibration	temperature / sigmoid / isotonic / none	temperature	Post-hoc probability calibration head. temperature is safe under small val (≥10 samples). sigmoid needs 10+, isotonic needs 30+. none = raw model probs (ablation only).

After a successful train, the deploy phase promotes staging → data/models/active/ if val_logloss clears the floor in config/loto.yaml. Healthy XGBoost numbers land around logloss 0.85-1.05 · RPS 0.18-0.22 · accuracy 0.50-0.55 — all three should look sane, not just accuracy.

LLM configuration

The match-detail modal on the Predict page ships every SHAP explanation through a configurable LLM provider. The default is a local MLX server (omlx) — no cloud key needed if you just want the site running. Cloud providers only fire when you switch default_provider in config/llm_models.yaml or pick one via the per-request dropdown in the modal.

Providers live in config/llm_models.yaml. The catalog ships with four entries, all speaking the OpenAI /chat/completions shape:

Provider id	Kind	Model	Env var
omlx (default)	local	Qwen3.6-35B-A3B-oQ4-mtp	OMLX_API_KEY
deepseek-v4-flash	cloud	deepseek-v4-flash	DEEPSEEK_API_KEY
gemini-3-1-flash-lite	cloud	gemini-3.1-flash-lite	GEMINI_API_KEY
claude	cloud	claude-sonnet-4-6	ANTHROPIC_API_KEY

# Copy the env template and fill in only the providers you want to use.
cp .env.example .env
# then edit .env, e.g.:
#   ANTHROPIC_API_KEY=sk-ant-...
#   DEEPSEEK_API_KEY=sk-...

# Bypass the startup readiness check (CI / offline dev — the /predict endpoint
# still returns SHAP output, only the LLM narrative is skipped).
export WCPREDICT_SKIP_LLM_CHECK=1

When a provider entry declares api_key_env but the env var is unset, the server marks it "missing key" in GET /llm/providers; the frontend dropdown greys it out and the modal surfaces a "switch provider" hint. To add a new provider, append an entry to config/llm_models.yaml (same schema) and list its env var in .env.example.

Architecture

src/wcpredict/
├── data/         # provider abstraction (worldcupflow CSV, eloratings, FIFA, ESPN)
├── features/     # 44-dim feature engineering + sample weights
├── models/       # XGBoost / LightGBM / CatBoost / RF / OrderedLogit wrappers
├── forge/        # pipeline framework (spec, common, scoring, framework)
├── evaluation/   # walk-forward CV + LOTO + frozen/online A/B
├── ensemble/     # Davidson ordered logit + weight fitter + stacking
├── calibration/  # temperature scaling + draw boost
├── online/       # atomic live-update flow
├── explain/      # TreeSHAP + LLM translator
├── simulator/    # deterministic bracket + Monte Carlo tournament
├── serving/      # FastAPI backend
├── llm/          # provider abstraction for the match-detail explainer
└── cli/          # Typer entry points

Stack

Python	3.11+, uv
Backend	FastAPI · uvicorn · Typer
Models	XGBoost · LightGBM · CatBoost · scikit-learn · mord
Explainability	SHAP — TreeSHAP for per-match drivers
HPO & tracking	Optuna · MLflow
Data	eloratings.net · FIFA Wikipedia · ESPN 2026 WC scoreboard · worldcupflow historical CSV (vendored under data/raw/)
Frontend	React 18 · Vite · react-router · react-i18next