Pyxle Framework Benchmark Suite

Transparent, reproducible benchmarks comparing Pyxle against popular web frameworks.

Every framework lives in its own frameworks/<name>/ directory and implements identical endpoints with the same business logic, database schema, and response format. There are no submodules and nothing fetched at runtime — clone the repo, run one script, and you get the same numbers we publish.

Quick start

./bench.sh                 # API throughput suite (default), all 7 frameworks
./bench.sh --suite=ssr     # SSR latency: Pyxle SSR vs Next.js
./bench.sh --suite=all     # both suites

bench.sh creates a local .venv, installs every dependency, builds the Pyxle and Next.js apps, starts the servers each suite needs, health-checks them, runs the benchmark, and tears everything down. The first run takes a few minutes (installs + builds); pass --skip-install --skip-build on later runs to reuse the prepared tree.

The two suites

API throughput (--suite=api)

Measures requests/second across seven frameworks hitting identical API endpoints.

Framework	Language	Category	Port	Server
Pyxle	Python	Full-stack (SSR + API)	8001	pyxle serve (uvicorn)
FastAPI	Python	API	8002	uvicorn
Django	Python	Full-stack	8003	uvicorn (ASGI)
Flask	Python	Micro	8004	gunicorn (4 gthread workers)
Express	Node.js	API	8005	built-in
Hono	Node.js	Ultralight API	8006	@hono/node-server
Next.js	Node.js	Full-stack (SSR)	3000	next start (production)

Endpoints (identical across every framework):

Endpoint	Method	Description
/api/json	GET	Return a static JSON object (pure serialization overhead)
/api/db	GET	Read one random row from SQLite (framework + DB)
/api/queries?n=5	GET	Read 5 random rows from SQLite (query loop)
/api/queries?n=20	GET	Read 20 random rows (heavier workload)
/api/form	POST	Parse JSON body, validate, return response
/health	GET	Minimal health check (raw routing overhead)

SSR latency (--suite=ssr)

Measures dynamic server-side rendering — the full page re-rendered on every request — for Pyxle SSR (frameworks/pyxle-ssr/, port 8011) vs Next.js (frameworks/nextjs/, port 3000), across byte-for-byte identical pages:

Page	Path	What it stresses
Landing	/	Baseline SSR overhead
Heavy	/heavy	A 300-row data table
Complex	/complex	A nested dashboard

Both apps use the same SQLite data and the same component structure, so the comparison isolates the rendering pipeline.

All suites use a SQLite database with 1,000 deterministically-seeded rows (WAL mode), identical across frameworks.

CLI options

./bench.sh [options]

  --suite=api|ssr|all     Which suite to run (default: api)
  --frameworks=a,b,...    (api) Only benchmark these frameworks
                            pyxle,fastapi,django,flask,express,hono,nextjs
  --duration=N            Seconds per test (default: 10)
  --connections=N,M,...   Concurrency levels (e.g. 10,50,100)
  --warmup=N              (api) Warmup requests per endpoint (default: 3)
  --output=FILE           Save JSON results to FILE
  --skip-install          Don't (re)install dependencies
  --skip-build            Don't (re)build the Pyxle / Next.js apps
  -h, --help              Show this help

Examples:

# Quick Pyxle vs FastAPI, high concurrency
./bench.sh --frameworks=pyxle,fastapi --connections=10,50,100 --duration=15

# SSR comparison only, longer runs
./bench.sh --suite=ssr --duration=15

# Re-run without reinstalling/rebuilding
./bench.sh --skip-install --skip-build

The runners under bench/ can also be invoked directly once servers are up:

cd bench && node api.mjs --only=pyxle,hono --duration=5
cd bench && node ssr.mjs --pyxle-url=http://127.0.0.1:8011 --duration=15

Methodology

• Tool: autocannon v8 (HTTP/1.1 benchmarking)
• Warmup: each endpoint receives warmup requests before measurement
• Duration: configurable (default 10 seconds per test)
• Concurrency: configurable connection levels
• Database: SQLite with WAL mode, 1,000 pre-seeded rows, identical schema across frameworks
• Hardware: results vary by machine; always compare on the same hardware

Fairness

• The async Python frameworks (Pyxle, FastAPI, Django) run on uvicorn (single ASGI worker); Flask runs on gunicorn with gthread workers, since it's synchronous — a single-worker sync config would reject connections under concurrency rather than serve them.
• All Node.js frameworks run on their default/recommended server; Next.js runs next start (production build).
• Pyxle runs via pyxle serve (production mode) with CSRF disabled for a fair POST comparison.
• Each framework's code is idiomatic — not artificially optimized or handicapped.
• Response bodies are identical across all frameworks for each endpoint.
• SSR suite: measures dynamic SSR — the page re-rendered on every request (Pyxle's @server vs Next.js force-dynamic). For static or cacheable content, Next.js pre-renders to HTML and serves it at static-file speed; this benchmark deliberately targets pages that must render per request.

Project structure

benchmarks/
├── README.md
├── LICENSE
├── bench.sh               # One-command runner (CLI-arg driven)
├── .gitignore
├── bench/                 # Load-test runners
│   ├── api.mjs            # API throughput suite (autocannon)
│   ├── ssr.mjs            # SSR latency suite (Pyxle SSR vs Next.js)
│   └── package.json       # autocannon
├── frameworks/            # One self-contained app per framework
│   ├── pyxle/             # Pyxle API app          → :8001
│   ├── pyxle-ssr/         # Pyxle SSR pages         → :8011
│   ├── fastapi/           # FastAPI                 → :8002
│   ├── django/            # Django (ASGI)           → :8003
│   ├── flask/             # Flask                   → :8004
│   ├── express/           # Express.js              → :8005
│   ├── hono/              # Hono                    → :8006
│   ├── nextjs/            # Next.js (API + SSR)     → :3000
│   └── shared/            # Shared DB seed helpers
└── results/               # Timestamped JSON per run (gitignored)

Reproducibility

Everything needed to reproduce the published numbers is in this one repository — no submodules, no separately-cloned apps. A fresh clone plus ./bench.sh --suite=all rebuilds and re-runs every comparison shown on pyxle.dev/benchmarks. Numbers will differ from ours in absolute terms (different hardware) but the relative standings are what the page reports.

Contributing

To add a new framework to the API suite:

1. Create frameworks/<name>/ with the app code.
2. Implement all six endpoints with identical logic and response format.
3. Add the framework (name, URL, port) to the FRAMEWORKS registry in bench/api.mjs.
4. Add it to the server-start block in bench.sh.
5. Update this README.

License

MIT