Benchmarking PostgreSQL 18, MySQL 9.0, SQL Server 2022, and SQLite 3.45 under identical AI agent workloads using the Model Context Protocol (MCP).
Every benchmark operation is a tools/call JSON-RPC 2.0 message sent through Faucet v0.1.12's MCP server. This measures what AI agents actually experience when querying databases.
- Indexing matters more than database choice. Filtered query improvements after adding 10 indexes: MySQL 670x, SQL Server 74x, SQLite 85x, PostgreSQL 43x.
- MySQL's defaults are secretly excellent. InnoDB's Adaptive Hash Index self-optimized during warmup — explicit tuning yielded near-zero improvement on point reads and session discovery.
- MCP protocol overhead is negligible. Session initialization (initialize + tools/list) takes <1ms across all databases.
- PostgreSQL leads concurrent scaling at 15,222 MCP calls/sec (c=25) with 0 errors at all concurrency levels.
- There's an optimization floor. Once queries drop below ~0.5ms, further database tuning is invisible to the agent — the MCP protocol layer becomes the dominant cost.
| Workload | PostgreSQL 18 | MySQL 9.0 | SQL Server 2022 | SQLite 3.45 |
|---|---|---|---|---|
| Session Discovery | 249 | 322 | 180 | 1,254 |
| Point Reads | 1,315 | 1,017 | 1,462 | 1,152 |
| Filtered Queries | 646 | 433 | 622 | 268 |
| Analytical | 1,129 | 702 | 1,169 | 464 |
| Concurrent (c=25) | 15,222 | 10,982 | 12,037 | 10,028 |
| Database | Workload | Before | After | Improvement |
|---|---|---|---|---|
| MySQL | Filtered Queries | 0.6/s | 433/s | 670x |
| SQL Server | Concurrent c=25 | 19/s | 12,037/s | 646x |
| SQLite | Concurrent c=100 | 17/s | 9,996/s | 592x |
| PostgreSQL | Concurrent c=25 | 30/s | 15,222/s | 511x |
See RESULTS.md for full data and METHODOLOGY.md for detailed methodology.
| Component | Detail |
|---|---|
| Host | AWS EC2 c6a.4xlarge (16 vCPU AMD EPYC, 32 GB RAM) |
| OS | Ubuntu 24.04 LTS |
| Docker | 29.2.1, Compose v5.1.0 |
| MCP Server | Faucet v0.1.12 |
| Transport | HTTP Streamable (JSON-RPC 2.0) |
| Dataset | ~25.6M rows across 8 tables (identical per database) |
Each database runs in a Docker container with 4 CPUs and 8 GB RAM.
AI Agent (Go harness) → JSON-RPC 2.0 / HTTP → Faucet MCP Server → Database
The harness implements a full MCP client:
- Establishes MCP sessions (
initialize+notifications/initialized) - Discovers available tools (
tools/list) - Executes database operations via
tools/call(faucet_query,faucet_insert, etc.) - Measures latency percentiles (P50/P90/P95/P99) and throughput
- Runs 3 iterations per workload, reports median
mcp-db-benchmark/
├── README.md
├── RESULTS.md # Full results with tables and analysis
├── METHODOLOGY.md # Detailed methodology
├── LICENSE
├── docker-compose.yml # Database containers (4 CPUs / 8 GB each)
├── configs/
│ ├── mysql/tuning.cnf # MySQL default + optimized configs
│ └── postgres/tuning.conf.optimized
├── scripts/
│ ├── datagen/ # Go data generator (deterministic, seed 42)
│ ├── schema_*.sql # DDL per database
│ ├── load_*.sh # Data loading scripts
│ ├── optimize_*.sh # Optimization scripts
│ ├── verify_mcp.sh # MCP connectivity verification
│ └── analyze.py # Analysis & chart generation
├── harness/ # Go MCP benchmark harness
│ ├── main.go # CLI entry point
│ ├── mcp_client.go # MCP JSON-RPC 2.0 client
│ ├── session.go # MCP session management
│ ├── runner.go # Benchmark orchestration
│ ├── workloads.go # 6 workload implementations
│ ├── metrics.go # Latency collection
│ └── reporter.go # JSON/CSV output
└── results/
├── system_info.txt # Hardware/software specs
├── round1/ # Baseline results (JSON)
├── round2/ # Optimized results (JSON)
├── optimization/ # Per-database optimization docs
└── analysis/
├── summary.csv # All metrics in one CSV
├── comparison.md # Auto-generated comparison
└── charts/ # PNG visualizations
- Docker & Docker Compose
- Go 1.24+
- Python 3 with matplotlib/numpy (for charts)
- Faucet v0.1.12+
# Clone
git clone https://github.com/faucetdb/mcp-db-benchmark.git
cd mcp-db-benchmark
# Start databases
docker compose up -d
# Generate and load data
cd scripts/datagen && go build -o datagen . && ./datagen && cd ../..
bash scripts/load_postgres.sh
bash scripts/load_mysql.sh
bash scripts/load_sqlserver.sh
bash scripts/load_sqlite.sh
# Install and configure Faucet
npm install -g @faucetdb/faucet@0.1.12
faucet service add bench-postgres --driver postgres --dsn "postgres://bench:benchpass@localhost:5432/benchmark?sslmode=disable"
faucet service add bench-mysql --driver mysql --dsn "bench:benchpass@tcp(localhost:3306)/benchmark"
faucet service add bench-sqlserver --driver mssql --dsn "sqlserver://sa:BenchPass%23123@localhost:1433?database=benchmark&encrypt=disable"
faucet service add bench-sqlite --driver sqlite --dsn "./data/benchmark.sqlite3"
# Start MCP server
faucet mcp --transport http --port 3001
# Build and run harness
cd harness && go build -o harness . && cd ..
# Verify MCP connectivity
./harness/harness verify --endpoint "http://localhost:3001/mcp" \
--services "bench-postgres,bench-mysql,bench-sqlserver,bench-sqlite"
# Round 1: Baseline
./harness/harness run-all --endpoint "http://localhost:3001/mcp" \
--services "bench-postgres,bench-mysql,bench-sqlserver,bench-sqlite" \
--round baseline --iterations 3 --output results/round1
# Apply optimizations
bash scripts/optimize_postgres.sh
bash scripts/optimize_mysql.sh
bash scripts/optimize_sqlserver.sh
bash scripts/optimize_sqlite.sh
# Round 2: Optimized
./harness/harness run-all --endpoint "http://localhost:3001/mcp" \
--services "bench-postgres,bench-mysql,bench-sqlserver,bench-sqlite" \
--round optimized --iterations 3 --output results/round2
# Generate analysis and charts
python3 scripts/analyze.pyRead the full analysis: Your Indexes Matter 10x More Than Your Database — And MySQL Knew That All Along
MIT. See LICENSE.