gateway-pro

A from-scratch HTTP API gateway written in Go.

This is a portfolio / learning project, not a production system. It was built to understand how an API gateway works by implementing the core pieces — routing, load balancing, circuit breaking, rate limiting, health checking, and observability — directly against the Go standard library instead of reaching for an existing framework. It is not production-proven or battle-tested. See Status & limitations.

gateway-pro is a single-binary gateway that sits in front of backend services. It routes incoming requests to the right upstream, spreads load across replicas, sheds traffic when a backend is failing, enforces per-client rate limits, and exposes Prometheus metrics — with the whole request path readable in a few small packages.

Architecture

Request path

Every proxied request flows through a middleware chain assembled with a small Chain helper. The outermost middleware runs first:

1. Recovery — recovers from panics and returns a JSON 500 instead of dropping the connection.
2. Request ID — attaches an X-Request-ID (reusing an inbound one if present) and echoes it back.
3. Tracing — W3C traceparent propagation with sampling; spans are exported asynchronously and kept in an in-memory ring buffer surfaced at the /traces admin endpoint.
4. Structured logging — one JSON line per request (method, path, status, duration, request ID) via zap.
5. Metrics — Prometheus request counter and latency histogram, labelled by route.
6. Auth — optional RS256 JWT validation.

After the chain, the request is matched to a route and proxied to a chosen backend.

Routing

Routes are matched by longest-prefix: the route whose path_prefix is the longest match for the request path wins. An optional strip_prefix removes the matched prefix before the request is forwarded upstream.

Authentication

JWTs are validated using RS256 (RSA + SHA-256), parsed directly against Go's stdlib crypto packages rather than a JWT library. The gateway loads a PEM-encoded RSA public key at startup (failing fast on misconfiguration), and on each request checks the algorithm (only RS256 is accepted — none and symmetric algorithms are rejected), the exp claim, and the signature. On success it forwards the token subject as X-User-ID so upstreams need not re-parse the token. Configurable skip_paths bypass auth (e.g. health/metrics).

Load balancing

Four strategies, selected per route, all goroutine-safe:

Strategy	Behaviour
round_robin	Even rotation across healthy backends (default).
least_conn	Fewest in-flight requests, tracked atomically.
weighted	Smooth weighted round-robin (nginx-style).
ip_hash	Sticky routing by client IP.

Circuit breaking

A per-backend three-state circuit breaker (closed → open → half-open → closed). In the closed state it tracks successes/failures over a rolling window; once the failure ratio crosses the threshold (with a minimum request count to avoid tripping on noise) it opens and fast-fails. After a cooldown it moves to half-open, admitting a limited number of probe requests before deciding whether to close again or re-open.

Rate limiting

Per-key limits, keyed by IP, user ID, or API key. Three implementations:

• Local token bucket — in-process, good for bursty traffic.
• Local sliding window — in-process, precise per-window counting via timestamp eviction.
• Distributed (Redis) sliding window — a Redis sorted set per key, driven by a Lua script so the prune-count-add sequence is atomic across gateway instances. This lets a fleet of gateways share one limit.

Health checks

An active health checker probes each backend's health endpoint on an interval (default 10s) and flips an atomic alive flag. The load balancer only selects live backends; if none are healthy it returns an error rather than routing into a black hole.

Config hot-reload

Config lives in a YAML file watched with fsnotify. On change, a new config is parsed and delivered over a channel, and routes/limiters/backends are rebuilt without restarting the process.

Trade-offs

• Fail-open on Redis unavailability. If the distributed limiter can't reach Redis (or times out — calls are bounded at 50ms), the request is allowed rather than rejected. This chooses availability over strict enforcement: a Redis outage degrades rate limiting instead of taking down the gateway. A stricter deployment might prefer fail-closed.
• No external JWT library. Implementing RS256 by hand keeps dependencies minimal and the code auditable, at the cost of maintaining crypto-adjacent code yourself — a reasonable trade for a learning project, less so for a large team.
• In-memory trace store. Traces are kept in a bounded ring buffer for the admin endpoint, not shipped to a full tracing backend by default. It's a debugging aid, not a system of record.

How to run

Requires Go 1.22+ (no C dependencies).

# Build
make build            # produces ./bin/gateway-pro

# Run
./bin/gateway-pro -config configs/gateway.yaml

Docker and Kubernetes manifests, plus a Docker Compose example with Redis, Prometheus, and Grafana, live under deploy/ and examples/:

make docker-build     # build the image locally
cd examples/docker-compose && docker compose up

Configuration

Routes, load-balancing algorithm, rate limits, and circuit-breaker thresholds are all defined in configs/gateway.yaml:

server:
  addr: ":8080"
admin:
  addr: ":9090"

routes:
  - path_prefix: /api/users
    lb_algorithm: round_robin      # round_robin | least_conn | weighted | ip_hash
    backends:
      - url: http://user-svc-1:8080
      - url: http://user-svc-2:8080
    rate_limit:
      algorithm: sliding_window    # token_bucket | sliding_window
      rate: 500
      window: 1m
      key_by: ip                   # ip | user | api_key
    circuit_breaker:
      failure_threshold: 50
      open_duration_seconds: 30

See configs/gateway.yaml for the fully annotated example.

Admin endpoints (default :9090)

Endpoint	Description
GET /metrics	Prometheus metrics
GET /healthz	Liveness check
GET /readyz	Readiness check
GET /backends	Live backend + circuit-breaker status
GET /traces	Recent request traces (in-memory ring buffer)

Project structure

cmd/gateway/          Entry point
internal/
  config/             YAML loader + fsnotify hot-reload
  loadbalancer/       Round-robin, least-conn, weighted, IP-hash
  ratelimiter/        Token bucket + sliding window (local and Redis/Lua)
  circuitbreaker/     Three-state circuit breaker
  health/             Active HTTP health checks
  middleware/         Recovery, request ID, tracing, logging, metrics, JWT auth
  admin/              Trace store + admin handlers
  proxy/              Gateway wiring, longest-prefix routing, admin server
deploy/               Dockerfile + Kubernetes manifests
examples/             Docker Compose stack (Redis, Prometheus, Grafana)
configs/              Annotated example config

Status & limitations

This is deliberately honest — the point of the project is the implementation, not a launch:

• Test coverage is uneven. Some packages (auth, tracing, admin) have tests; the core routing, load-balancing, circuit-breaker, and rate-limiter paths still need more coverage.
• No load testing or benchmarks have been run. There are no measured throughput or latency numbers, and none are claimed. The bench / load-test Makefile targets are scaffolding, not results.
• Not battle-tested in production. It has not run under real traffic, and edge cases around connection handling, backpressure, and failure modes have not been hardened.

If you're evaluating a gateway for real use, reach for a mature project (Kong, Traefik, Envoy). gateway-pro exists to be read and understood.

License

Apache 2.0 — see LICENSE.