A Gymnasium environment for Bennet Foddy's game QWOP - a pure Python Box2D implementation forked from qwop-gym.
This project reimplements the qwop-gym environment and tooling in pure Python, running entirely headless (no browser, no chromedriver). Designed for high-throughput training with parallelization in mind.
- Pure Python + Box2D - No browser, WebGL, or chromedriver. Runs entirely in process.
- Headless by default - Training uses no rendering; play, spectate, and replay use Pygame.
- Parallelization-ready -
n_envsusesSubprocVecEnv(DummyVecEnv fallback); seedoc/PARALLEL_SWEEPS.mdfor multi-run CPU scouts. - Same interface - 60-dim observations, Discrete 9/16 actions, compatible reward model. Behavior matches qwop-gym.
Train in this Box2D env, then transfer to the real browser game to beat 45.530s HUD (kurodo1916). Browser keep: 45.167 — see docs/wr-hunt/WINNING_RECIPE.md.
- Finish clock = landed
user/time(game ended, jump landed, not fallen).user/split_100m_timeis torso-at-100m only — not a finish / not a WR claim. settle_spawnmatches browser first-contact after reset (seeconfig/env.yml,doc/SETTLE_SPAWN.md, PR #10).- How we built the loop:
docs/wr-hunt/METHODOLOGY.md. Assets from the WR video:docs/wr-hunt/kurodo-extract/. - Winning stack:
docs/wr-hunt/WINNING_RECIPE.md. Strategy notes:docs/wr-hunt/STRATEGIES.md, segment gapSEGMENT_GAP_POST_SETTLE.md, Kurodo splitsKURODO_WR_SPLITS.md, farm orchestrationORCHESTRATION.md.
Farm-facing YAML under config/ (examples):
| Family | Examples |
|---|---|
| Flex gait / fps2stride | train_ppo_gait_fps2.yml, train_ppo_gait_fps2stride.yml, *_cont, *_ent, *_softhold, *_grace |
| Start pace / early survival | train_ppo_gait_fps2stride_startpace.yml, train_ppo_gait_fps2stride_earlysurv.yml |
| Kurodo-shaped | train_ppo_kurodo_gait.yml, train_ppo_kurodo_fps2.yml, train_ppo_kurodo_speed.yml, train_ppo_kurodo_cadence.yml |
| Browser BC | train_bc_browser_fps2.yml, train_ppo_browser_bc_finetune.yml |
| Scout / eval / spectate | scout_qrdqn_*.yml, eval_wr.yml, spectate_fps2stride5.yml, spectate_kurodo_gait_a.yml |
qwop-python -c config/train_ppo_gait_fps2stride_startpace.yml train_ppo
qwop-python -c config/eval_wr.yml evaluatePython 3.10+:
pip install -e .For RL training (stable-baselines3, PyTorch, TensorBoard):
pip install -e ".[sb3]"The qwop-python executable mirrors qwop-gym's CLI. Run from a directory with config/ and data/ (or repo root).
First-time setup:
qwop-python bootstrapPlay the game (use Q, W, O, P keys):
qwop-python playExplore the other commands:
$ qwop-python -h
usage: qwop-python [options] <action>
options:
-h, --help show this help message and exit
-c FILE config file, defaults to config/<action>.yml
--run-id ID run id (train_*)
action:
bootstrap create config/ with templates
play interactive gameplay
record play with recording (-c config/record.yml)
replay replay recorded actions
spectate watch trained model play
benchmark measure env steps/sec
evaluate headless HUD-time eval of a saved model
record_model headlessly roll out a model into .rec demos
train_bc behavioral cloning from .rec demos
train_ppo train using PPO
train_dqn train using DQN
train_qrdqn train using QRDQN
train_rppo train using RPPO
train_a2c train using A2C
examples:
qwop-python bootstrap
qwop-python play
qwop-python -c config/record.yml play
qwop-python spectate
qwop-python -c config/eval_wr.yml evaluate
qwop-python train_ppoRecord your own gameplay:
qwop-python -c config/record.yml playHeadlessly record a saved model into .rec demos (expert zip for demos only):
qwop-python record_modelBehavioral cloning from recordings (then optional PPO finetune; WR QRDQN stays from-scratch):
qwop-python train_bcTrain a PPO agent (edit config/train_ppo.yml if needed):
qwop-python train_ppoVisualize TensorBoard logs:
tensorboard --logdir data/Live WR chase dashboard (local TB + optional GCS farm heartbeats):
python scripts/wr_dashboard.py --port 8787 --fixture-dir infra/gcp/fixtures
# live: --gcs-bucket gs://qwop-wr-training
# local: --local-onlySee doc/WR_DASHBOARD.md and infra/gcp/CONTROL_PLANE.md.
Grok Bot orchestrates the farm; this dashboard is read-only.
Configure model_file in config/spectate.yml and watch a trained agent:
qwop-python spectateReplay recorded episodes:
qwop-python replayBenchmark env throughput (steps/sec):
qwop-python benchmarkEvaluate a saved model with HUD-clock metrics (see doc/TRANSFER_AND_METRICS.md):
qwop-python -c config/eval_wr.yml evaluateimport gymnasium as gym
# Headless (default) - for training
env = gym.make("local/QWOP-v1", seed=42)
# With rendering - for play, spectate, replay
env = gym.make("local/QWOP-v1", render_mode="human", seed=42)
obs, info = env.reset()
obs, reward, terminated, truncated, info = env.step(0)
env.render()
env.close()qwop_python/ # Main package
qwop_env.py # Gymnasium environment
game.py # Game loop, physics integration
tools/ # CLI (play, train_*, spectate, replay, benchmark)
wrappers/ # VerboseWrapper, RecordWrapper
config/ # YAML configs (created by bootstrap)
data/ # Models, logs, checkpoints, recordings
scripts/ # Parallel sweep launcher + monitor
doc/PARALLEL_SWEEPS.md # CPU-safe parallel scout experiments
On limited machines (e.g. 8 vCPU / 16GB), run 2–4 short experiments in parallel:
python scripts/sweep_parallel.py --builtin -n 2 --max-timesteps 200000
python scripts/monitor_runs.py --latestSee doc/PARALLEL_SWEEPS.md. n_envs in train YAML is honored (SubprocVecEnv, with DummyVecEnv fallback).
Apache-2.0 (same as qwop-gym).
