Skip to content

Add API benchmark suite#283

Open
JunkingA1 wants to merge 2 commits into
SecureBananaLabs:mainfrom
JunkingA1:bounty-30-api-benchmarks
Open

Add API benchmark suite#283
JunkingA1 wants to merge 2 commits into
SecureBananaLabs:mainfrom
JunkingA1:bounty-30-api-benchmarks

Conversation

@JunkingA1
Copy link
Copy Markdown

@JunkingA1 JunkingA1 commented May 19, 2026

/claim #30

Summary

Adds a reproducible API benchmark suite under benchmarks/ for the current Express API surface.

What changed:

  • Added npm run benchmark and npm run benchmark:smoke.
  • Added a dependency-free Node benchmark runner that can start the local Express app or target BENCHMARK_BASE_URL.
  • Covered all 20 current /api/ endpoints with realistic synthetic payloads and a benchmark auth token for protected routes.
  • Captured p50/p95/p99 latency, p50/p95/p99 TTFB, sustained/peak RPS, status counts, and error rate.
  • Wrote JSON and Markdown reports under benchmarks/results/.
  • Added reviewable p99/error-rate thresholds in benchmarks/thresholds.json.
  • Added a pull-request smoke benchmark workflow that uploads benchmark results.
  • Fixed the API test script so Node 26 runs the existing *.test.js test file instead of treating src/tests as an entry file.
  • Added a reviewer demo walkthrough at benchmarks/demo-walkthrough.md with exact replay/recording steps.

Reviewer Demo

Short demo walkthrough:

  • benchmarks/demo-walkthrough.md

Replay commands:

npm ci
npm test
npm run benchmark:smoke
sed -n '1,80p' benchmarks/results/api-benchmark-latest.md

The smoke benchmark starts the local API automatically, exercises all configured routes, checks benchmarks/thresholds.json, and writes both JSON and Markdown reports. The API Benchmark Smoke workflow also uploads benchmarks/results/ as the api-benchmark-results artifact for PR review.

Validation

npm_config_registry=https://registry.npmmirror.com npm ci
npm test
npm run benchmark:smoke

Results from the latest local verification:

✔ GET /health returns ok payload (33.807166ms)
tests 1
pass 1
fail 0

Wrote /private/tmp/securebanana-bounty-30-pr283/benchmarks/results/api-benchmark-latest.json
Wrote /private/tmp/securebanana-bounty-30-pr283/benchmarks/results/api-benchmark-latest.md

Smoke benchmark summary:

  • Routes covered: 20
  • Requests per endpoint: 1
  • Concurrency per endpoint: 1
  • Threshold result: passed
  • Error rate: 0% on every endpoint

Benchmark Environment

Hardware

  • CPU model & core count: Apple M5, 10 logical cores
  • RAM (total & available during benchmark): 16 GiB total, benchmark runner recorded 105 MiB free at report time
  • Storage type (SSD / NVMe / HDD): local Apple SSD / APFS
  • Network interface (Ethernet / WiFi / loopback): loopback (127.0.0.1)
  • Machine type (local workstation / cloud VM / CI runner - include instance type if cloud): local Mac workstation
  • OS & version: macOS 26.4.1, Darwin 25.4.0 arm64

Runtime

  • Node.js version (or relevant runtime): Node.js v26.0.0
  • Any resource limits applied (Docker memory cap, cgroup limits, etc.): none intentionally applied
  • Other significant processes running during benchmark (yes / no - if yes, describe): yes, normal desktop/Codex/background processes

If submitted by or with an AI agent

  • Agent or tool name (e.g. Claude Code, Devin, Copilot Workspace, AutoGPT): OpenAI Codex
  • Underlying model and version (e.g. claude-sonnet-4-5, gpt-4o - if known): GPT-5 Codex-family coding agent
  • Inference provider (e.g. Anthropic, OpenAI, Azure, self-hosted): OpenAI
  • Orchestration framework if any (e.g. LangChain, AutoGen, custom): none
  • Execution mode (fully autonomous / human-supervised / human-initiated per step): human-initiated, agent-executed
  • Did the agent have shell/tool access during execution (yes / no): yes
  • Did the agent have internet access during execution (yes / no): yes
  • Were benchmark commands run by the agent directly or handed off to the human to run: run by the agent directly; dependency installation used the npm mirror registry because the local network previously had npm registry timeouts
  • Any known agent constraints or sandboxing that may have affected execution: local sandbox required normal host permissions for tests/benchmarks that bind a local port; final verification was run successfully with local port access

@Adamchaua
Copy link
Copy Markdown

Monitoring this PR. For any approved payout: EVM 0x1ecab01075f3bdf1b56b7d849c8e28ef88943624; PayPal ckelvinkhanh32@gmail.com.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants