SBproxy

Last modified: 2026-06-24

The AI gateway built like a real proxy.

Install · Quick start · Examples · Docs

---

Why SBproxy

Most teams run one tool for HTTP traffic and another for LLM traffic. That's two systems to configure, deploy, and monitor. SBproxy handles both in one binary.

• One config file replaces your reverse proxy, AI gateway, and the middleware glue between them.
• 200+ LLM models behind an OpenAI-compatible API, with fallback chains, guardrails, and budgets.
• Local semantic cache. Near-duplicate prompts replay from cache. Vectorize them on-box with a local embedder, so there is no per-call cost and the prompt never leaves your network.
• Observable out of the box. Metrics, logs, and traces from one process. AI spans follow the OpenTelemetry GenAI and OpenInference conventions and carry per-request token counts and USD cost, ready for Phoenix, Langfuse, Grafana, or Datadog.
• Secure by default. Auth, rate limiting, WAF, DDoS, and CSRF are built in.
• Hot reload with no dropped connections.
• Sub-millisecond p99 overhead. Idle RSS in single-digit megabytes.

---

Install

curl (macOS / Linux):

curl -fsSL https://download.sbproxy.dev | sh

The script detects your OS and architecture, fetches the matching release binary from GitHub, and drops it in ~/.local/bin. Override with SBPROXY_INSTALL=<dir> for a custom location or SBPROXY_VERSION=<tag> to pin a release.

Homebrew (macOS / Linux):

brew tap soapbucket/tap
brew install sbproxy

Docker:

docker pull ghcr.io/soapbucket/sbproxy:latest

From source (needs Rust 1.82+):

git clone https://github.com/soapbucket/sbproxy
cd sbproxy
make build-release

---

Quick start

We host a public HTTP echo service at test.sbproxy.dev (request inspection, like httpbin) so you can wire up a real upstream without leaving the SoapBucket ecosystem. Try it directly:

curl https://test.sbproxy.dev/get

Now run the gateway in front of it. Drop this into sb.yml:

proxy:
  http_bind_port: 8080

origins:
  "myapp.example.com":
    action:
      type: proxy
      url: https://test.sbproxy.dev

make run CONFIG=sb.yml
curl -H "Host: myapp.example.com" http://127.0.0.1:8080/get

myapp.example.com is the host your client sees; SoapBucket matches it against origins: and forwards to the upstream. Use any hostname you want here; example.com is reserved (RFC 2606), so it never collides with anything real.

That's a reverse proxy. Add AI routing, auth, and rate limiting in the same file. See examples/ for runnable end-to-end configurations covering each feature.

---

See it in action

Each clip is recorded against the release binary running a real example config. Regenerate them with scripts/record-tapes.sh.

Failover across providers — the primary is down; the backup answers, transparently. (config)

Semantic cache — a reworded prompt is served from cache (x-semcache: HIT), skipping the billable completion. (config)

Guardrails — prompt-injection and PII are blocked before any provider is called. (config)

---

Documentation

The full documentation lives in docs/README.md: manual, configuration reference, AI gateway guide, scripting reference, performance, troubleshooting, architecture, and more. Running the operator for the first time? Start with docs/quickstart-operator.md.

For contributors: CONTRIBUTING.md.

---

Community

• Issue Tracker for bug reports and feature requests.
• Looking for a managed offering? SBproxy Enterprise.

---

Upgrading from v0.1.x (Go)

SBproxy v1.0 is a Rust rewrite. The Go implementation that previously occupied this repository is archived at soapbucket/sbproxy-go and tagged v0.1.2-go-final. New work happens here. See MIGRATION.md for the upgrade path; existing sb.yml files should compile unchanged.

---

License

Licensed under the Apache License 2.0. Free for any use, including production and commercial, with no field-of-use restriction.

See also NOTICE and TRADEMARKS. A Soap Bucket LLC project.