Skip to content

Latest commit

 

History

History
222 lines (159 loc) · 7.03 KB

README.md

File metadata and controls

222 lines (159 loc) · 7.03 KB

QWOP Python

made-with-python Open Source Love License: Apache-2.0

A Gymnasium environment for Bennet Foddy's game QWOP - a pure Python Box2D implementation forked from qwop-gym.

qwop-python

This project reimplements the qwop-gym environment and tooling in pure Python, running entirely headless (no browser, no chromedriver). Designed for high-throughput training with parallelization in mind.

Differences from qwop-gym

  • Pure Python + Box2D - No browser, WebGL, or chromedriver. Runs entirely in process.
  • Headless by default - Training uses no rendering; play, spectate, and replay use Pygame.
  • Parallelization-ready - n_envs uses SubprocVecEnv (DummyVecEnv fallback); see doc/PARALLEL_SWEEPS.md for multi-run CPU scouts.
  • Same interface - 60-dim observations, Discrete 9/16 actions, compatible reward model. Behavior matches qwop-gym.

World-record hunt (HTML5)

Train in this Box2D env, then transfer to the real browser game to beat 45.530s HUD (kurodo1916). Browser keep: 45.167 — see docs/wr-hunt/WINNING_RECIPE.md.

Training configs (WR hunt)

Farm-facing YAML under config/ (examples):

Family Examples
Flex gait / fps2stride train_ppo_gait_fps2.yml, train_ppo_gait_fps2stride.yml, *_cont, *_ent, *_softhold, *_grace
Start pace / early survival train_ppo_gait_fps2stride_startpace.yml, train_ppo_gait_fps2stride_earlysurv.yml
Kurodo-shaped train_ppo_kurodo_gait.yml, train_ppo_kurodo_fps2.yml, train_ppo_kurodo_speed.yml, train_ppo_kurodo_cadence.yml
Browser BC train_bc_browser_fps2.yml, train_ppo_browser_bc_finetune.yml
Scout / eval / spectate scout_qrdqn_*.yml, eval_wr.yml, spectate_fps2stride5.yml, spectate_kurodo_gait_a.yml
qwop-python -c config/train_ppo_gait_fps2stride_startpace.yml train_ppo
qwop-python -c config/eval_wr.yml evaluate

Install

Python 3.10+:

pip install -e .

For RL training (stable-baselines3, PyTorch, TensorBoard):

pip install -e ".[sb3]"

The qwop-python tool

The qwop-python executable mirrors qwop-gym's CLI. Run from a directory with config/ and data/ (or repo root).

First-time setup:

qwop-python bootstrap

Play the game (use Q, W, O, P keys):

qwop-python play

Explore the other commands:

$ qwop-python -h
usage: qwop-python [options] <action>

options:
  -h, --help    show this help message and exit
  -c FILE       config file, defaults to config/<action>.yml
  --run-id ID   run id (train_*)

action:
  bootstrap      create config/ with templates
  play           interactive gameplay
  record         play with recording (-c config/record.yml)
  replay         replay recorded actions
  spectate       watch trained model play
  benchmark      measure env steps/sec
  evaluate       headless HUD-time eval of a saved model
  record_model   headlessly roll out a model into .rec demos
  train_bc       behavioral cloning from .rec demos
  train_ppo      train using PPO
  train_dqn      train using DQN
  train_qrdqn    train using QRDQN
  train_rppo     train using RPPO
  train_a2c      train using A2C

examples:
  qwop-python bootstrap
  qwop-python play
  qwop-python -c config/record.yml play
  qwop-python spectate
  qwop-python -c config/eval_wr.yml evaluate
  qwop-python train_ppo

Record your own gameplay:

qwop-python -c config/record.yml play

Headlessly record a saved model into .rec demos (expert zip for demos only):

qwop-python record_model

Behavioral cloning from recordings (then optional PPO finetune; WR QRDQN stays from-scratch):

qwop-python train_bc

See doc/DEMO_BOOTSTRAP.md.

Train a PPO agent (edit config/train_ppo.yml if needed):

qwop-python train_ppo

Visualize TensorBoard logs:

tensorboard --logdir data/

Live WR chase dashboard (local TB + optional GCS farm heartbeats):

python scripts/wr_dashboard.py --port 8787 --fixture-dir infra/gcp/fixtures
# live:  --gcs-bucket gs://qwop-wr-training
# local: --local-only

See doc/WR_DASHBOARD.md and infra/gcp/CONTROL_PLANE.md. Grok Bot orchestrates the farm; this dashboard is read-only.

Configure model_file in config/spectate.yml and watch a trained agent:

qwop-python spectate

Replay recorded episodes:

qwop-python replay

Benchmark env throughput (steps/sec):

qwop-python benchmark

Evaluate a saved model with HUD-clock metrics (see doc/TRANSFER_AND_METRICS.md):

qwop-python -c config/eval_wr.yml evaluate

Create an instance in code

import gymnasium as gym

# Headless (default) - for training
env = gym.make("local/QWOP-v1", seed=42)

# With rendering - for play, spectate, replay
env = gym.make("local/QWOP-v1", render_mode="human", seed=42)
obs, info = env.reset()
obs, reward, terminated, truncated, info = env.step(0)
env.render()
env.close()

Project structure

qwop_python/           # Main package
  qwop_env.py          # Gymnasium environment
  game.py              # Game loop, physics integration
  tools/               # CLI (play, train_*, spectate, replay, benchmark)
  wrappers/            # VerboseWrapper, RecordWrapper
config/                # YAML configs (created by bootstrap)
data/                  # Models, logs, checkpoints, recordings
scripts/               # Parallel sweep launcher + monitor
doc/PARALLEL_SWEEPS.md # CPU-safe parallel scout experiments

Parallel scout sweeps

On limited machines (e.g. 8 vCPU / 16GB), run 2–4 short experiments in parallel:

python scripts/sweep_parallel.py --builtin -n 2 --max-timesteps 200000
python scripts/monitor_runs.py --latest

See doc/PARALLEL_SWEEPS.md. n_envs in train YAML is honored (SubprocVecEnv, with DummyVecEnv fallback).

License

Apache-2.0 (same as qwop-gym).