Distributed, Tiered Rate Limiter (Full Setup + Guide)

A production-style rate limiting system using:

Node.js + Express (backend)
Redis + Lua (atomic logic across 3 algorithms: sliding window counter, sliding window log, token bucket)
Tiered, per-API-key limits with Prometheus metrics, health/readiness probes, and graceful shutdown
React + Vite (dashboard)
k6 (load testing)

This project shows how real systems handle concurrent traffic safely across multiple servers, and how different clients (anonymous vs. paying tiers) can be rate limited differently without duplicating the underlying enforcement logic.

What’s Implemented

Three interchangeable rate limiting algorithms (Sliding Window Counter, Sliding Window Log, Token Bucket) - all Redis + Lua atomic, not just single-process toys
Tiered, per-API-key rate limits (anonymous/free/pro), each tier deliberately using a different algorithm so the tradeoffs are observable, not just described
Redis-based distributed state
Lua scripting for atomicity (no race conditions)
Middleware-based rate limiting
Prometheus /metrics endpoint (request + rate-limit-check histograms/counters)
/health (liveness) and /ready (readiness) probes, graceful shutdown on SIGTERM/SIGINT (drains in-flight requests, closes Redis cleanly)
Structured JSON logging (pino) with per-request logs
OpenAPI/Swagger docs at /docs
Real-time dashboard (logs + status + tier simulation buttons)
k6 load testing setup, with real captured benchmark numbers (see below)

⚙️ Architecture

sequenceDiagram
    participant Client
    participant Node API (PM2)
    participant Redis

    Client->>Node API (PM2): HTTP Request
    Node API (PM2)->>Redis: Execute Lua Script (Current & Prev Keys)
    Note over Redis: Atomic execution:<br/>1. Read counts<br/>2. Calculate weighted avg<br/>3. Increment
    Redis-->>Node API (PM2): Return { allowed: 1/0, remaining }

    alt Allowed
        Node API (PM2)-->>Client: 200 OK (X-RateLimit Headers)
    else Blocked
        Node API (PM2)-->>Client: 429 Too Many Requests
    end

Rate Limiting Tiers & Algorithms

Requests are resolved to a tier via the X-Api-Key header (falls back to IP-based anonymous if the header is missing or unrecognized). Each tier is configured in core/src/config/tiers.ts with its own algorithm, limit, and window - so the three algorithms aren't just implemented, they're actually running side by side in the same process:

Tier	How to select it	Algorithm	Limit	Why this algorithm
`anonymous`	no header (uses IP)	Sliding Window Counter	10 / 60s	Cheap, O(1) memory per identifier - fine for casual/unauthenticated traffic
`free`	`X-Api-Key: demo-free-key`	Sliding Window Log	30 / 60s	Exact enforcement (no boundary-burst approximation), worth the extra memory once a client is identified
`pro`	`X-Api-Key: demo-pro-key`	Token Bucket	300 / 60s	Allows short bursts as long as the long-run average holds - better fit for bursty API clients

# anonymous tier
curl -i http://localhost:5001/test-hit

# free tier
curl -i -H "X-Api-Key: demo-free-key" http://localhost:5001/test-hit

# pro tier
curl -i -H "X-Api-Key: demo-pro-key" http://localhost:5001/test-hit

Every response carries X-RateLimit-Tier, X-RateLimit-Limit, and X-RateLimit-Remaining headers. The dashboard's "Simulate" buttons exercise all three tiers directly so the behavior is visible without touching curl.

In a real deployment, API_KEYS in tiers.ts would be a database lookup (customer -> plan) instead of a hardcoded map - the two demo keys exist purely so this is runnable and demoable out of the box.

Backend Setup (core)

1 Install dependencies

pnpm install

2 Environment setup

Create .env:

API_PORT=5001
REDIS_HOST=localhost
REDIS_PORT=6379
LOG_LEVEL=info

Tier limits/windows/algorithms aren't env-driven - they live in core/src/config/tiers.ts (see Rate Limiting Tiers & Algorithms above).

3 Start Redis

If installed locally:

redis-server

Or using Docker:

docker run -p 6379:6379 redis

4 Build project

pnpm run build

5 Run server

pnpm run prod

Or dev mode:

pnpm run dev

Backend Scripts

"scripts": {
  "test": "vitest run",
  "test:watch": "vitest",
  "test:load": "k6 run dist/tests/k6-test.js",
  "build": "tsc",
  "dev": "nodemon dist/api.js",
  "prod": "node dist/api.js",
  "watch": "tsc -w"
}

Docker Setup

The whole system (Redis + backend + frontend) can be run with Docker Compose, no local Node/pnpm/Redis install required.

docker compose up --build

Backend: http://localhost:5001
Frontend: http://localhost:8080
Redis: localhost:6379

Services:

redis — Redis 7, with a healthcheck the core service waits on
core — builds core/Dockerfile (multi-stage: pnpm install → tsc build → slim runtime image), listens on API_PORT (default 5001)
client — builds client/Dockerfile (multi-stage: Vite build → static files served by nginx)

The frontend is a Vite app, so VITE_API_URL is baked in at build time, not read at container runtime. Since the browser (not the client container) calls the API directly, it's set to the host-exposed URL (http://localhost:5001/) via a build arg in docker-compose.yml, not the internal core service name. Override it if you deploy behind a different host/port:

docker compose build --build-arg VITE_API_URL=https://api.example.com/ client

To build/run a single service image directly:

docker build -t rate-limiter-core ./core
docker build -t rate-limiter-client ./client

Frontend Setup (client)

1 Install

pnpm install

2 Run dev server

pnpm run dev

Open:

http://localhost:5173

Dashboard Features

Live logs, including tier per request
Allowed / Blocked requests
Remaining quota
Simulate buttons for all three tiers (Anonymous / Free Key / Pro Key), so tiered rate limiting is demoable without curl

k6 Setup (Load Testing)

Install k6

Linux:

sudo apt update
sudo apt install k6

If not available:

sudo snap install k6

Run load test

pnpm run test:load

This executes:

k6 run dist/tests/k6-test.js

Optional dashboard

K6_WEB_DASHBOARD=true k6 run dist/tests/k6-test.js

Open:

http://localhost:5665

Benchmark Results

k6 runs its scripts in its own restricted JS runtime, not Node - no process.env, no npm module resolution. Env vars come in via __ENV (pass with -e KEY=value). The script hits /test-hit for 20 VUs over 10s. Numbers below are from an actual local run (single machine, Redis + API on localhost, not tuned for peak throughput - just to have real numbers instead of guesses):

Anonymous tier (sliding-window-counter, limit 10/60s) — k6 run -e API_PORT=5001 dist/tests/k6-test.js:

http_reqs......................: 1920   190.28/s
http_req_duration..............: avg=3.77ms   p90=7.45ms   p95=12.37ms   max=100.79ms
http_req_failed.................: 99.47% (1910/1920 correctly rejected with 429)

The limiter let the first ~10 requests through, then correctly rejected everything else with 429 for the rest of the 10s run - proving atomic enforcement holds even under 20 concurrent VUs hammering the same identifier, not just under light load.

Pro tier (token-bucket, capacity 300, refill 5/s) via X-Api-Key: demo-pro-key:

http_reqs......................: 2793   278.28/s
http_req_duration..............: avg=20.21ms  p90=48.65ms  p95=61.88ms   max=141.45ms
http_req_failed.................: 87.50% (~349 allowed: the 300 burst capacity + refill accrued over the run)

The allowed count (~349) lines up with theory: 300 capacity plus ~5 tokens/sec refilling over the remaining ~9s once the bucket was drained - a nice sanity check that the Lua implementation matches the documented algorithm, not just its happy path.

How Sliding Window Counter Works

We approximate a continuous sliding window using:

estimated = (prevCount * overlap) + currCount

Where:

overlap = 1 - (time_into_current_window / window_size)

Why It’s Not Perfectly Accurate

Because we assume:

requests in previous window were uniformly distributed

Reality:

requests can be bursty
timing info is lost

Example Problem

Previous window:

10 requests all happened at end

System assumes:

spread evenly → incorrect weighting

Tradeoffs

Pros

Fast
Memory efficient
Works in distributed systems
Prevents burst attacks

Cons

Approximation (not exact)
Floating point precision issues
Depends on Redis
Slight timing inaccuracies

Consistency Model

Strong Consistency (per request)

Lua script runs atomically inside Redis
No race conditions

Eventual Consistency (system-wide)

Multiple servers rely on Redis
Network delays possible

Why Lua Script?

Without Lua:

GET → CHECK → INCR race condition

With Lua:

ALL INSIDE REDIS atomic

Testing Scenarios

Manual frontend clicks (simulate any of the 3 tiers from the dashboard)
curl with/without X-Api-Key to hit different tiers directly
k6 concurrent load (20 VUs) against any tier

Automated Tests

Backend (`core`)

Uses Vitest:

src/tests/SlidingWindowLimiter.test.ts, src/tests/TokenBucketLimiter.test.ts, src/tests/SlidingWindowLogLimiter.test.ts — integration tests against a real Redis instance for each of the three algorithms (limit enforcement, per-identifier isolation, window/refill behavior, shared log recording). Needs Redis reachable via REDIS_HOST/REDIS_PORT (defaults to localhost:6379) — run docker run -p 6379:6379 redis or docker compose up redis -d first. Each suite uses its own Redis logical DB (db: 0/1/2) so flushdb() in one suite can't race with another suite running in a parallel test file.
src/tests/limiter.test.ts — unit tests for the Express middleware with mocked limiters (no Redis needed): allowed/blocked/error paths, rate-limit headers, and tier resolution from the X-Api-Key header.

cd core
pnpm test          # single run
pnpm run test:watch

Frontend (`client`)

Uses Vitest + Testing Library with jsdom:

src/App.test.tsx — renders the dashboard, verifies the empty state and that fetched logs render with the correct Allowed/Dropped status.

cd client
pnpm test
pnpm run test:watch

Monitoring & Observability

Dashboard shows:

Timestamp
User/IP or API key
Tier (anonymous/free/pro)
Remaining quota
Allowed/Blocked status

Helps visualize limiter behavior in real-time.

Prometheus metrics (`/metrics`)

Exposes, in Prometheus text exposition format:

rate_limiter_checks_total{tier, result} - counter of allowed/blocked checks per tier
rate_limiter_check_duration_seconds{tier} - histogram of the Redis round-trip for each check
http_request_duration_seconds{method, route, status_code} - histogram for every HTTP request
default Node.js process metrics (event loop lag, heap, GC) via prom-client's collectDefaultMetrics

Point Prometheus/Grafana at http://localhost:5001/metrics to build real dashboards on top of this instead of just eyeballing the frontend.

Health & readiness

GET /health - liveness probe, always 200 once the process is up. Doesn't check dependencies, so it won't flap just because Redis has a blip.
GET /ready - readiness probe, pings Redis and returns 503 if it's unreachable. Use this as the one a load balancer/k8s readiness check should hit.

Structured logging

All logs are structured JSON (via pino), including a per-request log line (method, route, status, duration) from pino-http. In non-production (NODE_ENV !== "production"), logs are pretty-printed for local dev instead.

API Documentation

Interactive OpenAPI/Swagger docs are served at http://localhost:5001/docs, covering /test-hit, /logs, /health, /ready, and /metrics - including the X-Api-Key header and the tier-related response headers.

Important Design Decisions

`/logs` is NOT rate limited

Why:

monitoring should always be available
needed during debugging/high load

Rate limiting only applied to:

/test-hit or specific endpoints

Graceful shutdown

On SIGTERM/SIGINT (e.g. docker compose stop, a Kubernetes pod eviction), the server stops accepting new connections, lets in-flight requests finish, closes the Redis connection, and only then exits - with a 10s force-exit safety net in case something hangs. This avoids dropped requests during rolling deploys.

CI Pipeline

.github/workflows/ci.yml runs on every push/PR to master:

core — installs deps, tsc build, vitest run against a real redis:7-alpine GitHub Actions service container
client — installs deps, eslint, vitest run, vite build
docker (after both pass) — builds the core and client Docker images to catch any Dockerfile breakage

TL;DR

Redis → shared state
Lua → atomic execution across 3 algorithms (sliding window counter/log, token bucket)
Tiered API keys → per-customer limits, not just one-size-fits-all
Prometheus + health/ready + structured logs → actually observable in production
k6 → load testing, with real captured numbers, not just a script that exists
Dashboard → visibility, with live tier simulation
Docker Compose → one-command local stack, graceful shutdown on stop
CI → build + test on every push/PR

What Makes This Project Strong

Handles real-world concurrency, with three algorithms implemented atomically (not just described) so their tradeoffs are directly comparable
Uses distributed system design, including per-tenant (API key) rate limits like a real SaaS product, not just IP throttling
Includes observability (Prometheus metrics, health/readiness probes, structured logs) and testing (25 automated tests across 4 suites)
Production-conscious: graceful shutdown, OpenAPI docs, no hand-waving on shutdown or deployment behavior
Covers tradeoffs (not just implementation) - and backs claims with real k6 numbers instead of assertions

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
.github/workflows		.github/workflows
client		client
core		core
.gitignore		.gitignore
Readme.md		Readme.md
docker-compose.yml		docker-compose.yml

Folders and files

Latest commit

History

Repository files navigation