GodotBench

GodotBench is a local, playable benchmark for comparing AI-generated Godot games. It includes Godot source projects generated from the same prompts, a Next.js arena for side-by-side play, pairwise voting, and Elo rankings calculated from votes on your machine.

The project is designed to run on your machine. Votes, seen matchups, sign-in details, and rankings are stored locally in SQLite, so the leaderboard reflects the votes recorded in your checkout.

Included Models

GodotBench currently includes generated game projects for:

• Claude 4.6 Opus Thinking (claude_4_6_opus_thinking)
• Claude 4.8 Opus Max (claude_4_8_opus_max)
• Claude 5.0 Fable Xhigh (claude_5_0_fable_xhigh)
• Composer 2.5 (composer_2_5)
• DeepSeek V4 Pro (deepseek_v4_pro)
• Gemini (gemini)
• Gemini 3.5 Flash (gemini_3_5_flash)
• GPT 5.5 Xhigh (gpt_5_5_xhigh)
• Grok 4.3 (grok_4_3)
• Kimi K2.6 (kimi_k2_6)
• MiniMax M3 (minimax_m3)
• Qwen 3.7 Max (qwen_3_7_max)

What Is Included

• games/: Godot source projects, organized as games/<prompt_id>/<model_id>/.
• web-app/: the Next.js voting arena and leaderboard.
• package_game.py: exports Godot projects to WebAssembly and registers them in local SQLite.
• prompts.md: benchmark prompts used to generate the projects.
• PROJECT_MANIFEST.md: source coverage and local export status snapshot.

Generated web exports are intentionally ignored by git because full Godot WebAssembly builds are large. Build them locally when you want to play the benchmark.

Requirements

• Node.js 20 or newer.
• npm.
• Python 3.
• Godot 4.x with Web export templates installed, required when packaging games.

If Godot is not on your PATH, set GODOT_BIN to the executable path. On macOS this is often:

export GODOT_BIN="/Applications/Godot.app/Contents/MacOS/Godot"

Quick Start

Install the web app dependencies:

cd web-app
cp .env.example .env.local
npm install

Package at least two games for the same prompt. This quick sample is enough to unlock one local arena matchup:

cd ..
python3 package_game.py --path games/one_button_hero/composer_2_5 --model composer_2_5 --prompt one_button_hero
python3 package_game.py --path games/one_button_hero/gpt_5_5_xhigh --model gpt_5_5_xhigh --prompt one_button_hero

Run the app:

cd web-app
npm run dev

Open http://localhost:3000.

When voting locally, use the "Local Sign-In" button. It is enabled by ENABLE_LOCAL_AUTH=true in .env.local and stores votes in local SQLite.

Package All Games

To build every checked-in Godot project into web-app/public/games/:

cd web-app
npm run package:games

That command runs:

python3 package_game.py --parent-dir games --prune-stale

The full export can take a while and may produce a large web-app/public/games/ directory. Those files stay local unless you choose to distribute them separately.

Local Data

The app uses SQLite by default. The database is created automatically at:

web-app/data/database.sqlite

This file is ignored by git. Delete it to reset users, seen matchups, votes, and Elo scores. While the database exists, the arena avoids serving the same matchup twice for the same local viewer.

Optional environment variables:

• ENABLE_LOCAL_AUTH=true: enables the local sign-in button.
• JWT_SECRET: required for production-style npm run start; use a long random value.
• SQLITE_DATABASE_PATH: stores SQLite somewhere other than web-app/data/database.sqlite.
• NEXT_PUBLIC_APP_URL: sets canonical metadata and sitemap URLs.
• COOKIE_SECURE=true: mark cookies as HTTPS-only when running behind HTTPS.

Development Commands

From web-app/:

npm run dev
npm run lint
npm run build
npm run package:games
npm run package:prune

From the repo root:

python3 package_game.py --path games/<prompt_id>/<model_id> --model <model_id> --prompt <prompt_id>
python3 package_game.py --parent-dir games --prune-stale

Adding Or Updating A Game

1. Add the Godot project at games/<prompt_id>/<model_id>/.
2. Make sure it has project.godot.
3. Package it:

python3 package_game.py --path games/<prompt_id>/<model_id> --model <model_id> --prompt <prompt_id>

4. Start the web app and confirm the game appears in the arena.
5. Commit source files and metadata, but leave generated exports in web-app/public/games/ untracked.

Open Source Notes

The app is local-first on purpose. Shared public deployments can still be built by contributors, but this repository does not include provider-specific deployment scripts or credentials.

Generated files that should not be committed:

• web-app/public/games/
• web-app/data/*.sqlite
• games/**/export/
• .env.local and other local env files

License

MIT. See LICENSE.