Enterprise AI Gateway — reliable, unified, OpenAI-compatible infrastructure to connect, manage, route, and observe multiple AI providers.
North Star: First become an excellent Gateway; let real usage data tell us what the Control Plane should become.
Enterprises already run OpenAI, Anthropic, Gemini, domestic models, and internal AI services side by side. What they often lack is a single access layer that is:
- Unified — one OpenAI-compatible endpoint for
/v1/chat/completions(stream SSE) and/v1/embeddings - Reliable — 3-state circuit breaking (Closed/Open/Half-Open), exponential backoff retries, Redis distributed state, and same-model failover
- Enterprise Financial Guardrails — Requests Per Minute (RPM) rate limiting and Monthly Budget Quotas with Soft (80%) warning and Hard (100%) block limits
- Secure & Compliant — Enterprise OIDC / OAuth2 Single Sign-On (SSO), AES-256-GCM encrypted provider credentials at rest, SHA-256 hashed API keys, and Prompt Injection / Jailbreak Guardrails
- Real-Time Alerts — HMAC-SHA256 signed Webhook notification engine for circuit trips, budget alerts, and guardrail blocks
- Honest about capability — Model Capability Is Sacred: no silent downgrades from premium models to cheaper/weaker ones
┌─────────────────────────────────┐
│ Enterprise Web Dashboard │
│ (Next.js 14 — Dark Mode GUI) │
└────────────────┬────────────────┘
│ :8081 Admin REST API & SSO
┌────────────────▼────────────────┐
│ BettaGate Gateway Core │
│ Auth · Guardrail · Routing │
│ (7-Step Data Pipeline — Go) │
└────────┬───────────────┬────────┘
│ │
PostgreSQL 16 & Redis ▼
(pgx Pool & Stores) AI Providers
(OpenAI / Anthropic / Gemini / Compatible)
Client Request
├── 1. Validation & Auth Context Injection (SHA-256 API Key & Quota Check)
├── 2. Security & Validation (Prompt Injection Guardrails & PII Anonymization)
├── 3. Route Resolution (Priority + Nginx Smooth WRR)
├── 4. Provider Lookup & Credential Decryption (AES-256-GCM)
├── 5. Outbound Execution (Retryable 429/500-504 Backoff & Same-Model Failover)
├── 6. PII Deanonymization (Streaming & Non-Streaming Chunk Restoration)
└── 7. Async Usage Event & Telemetry Emission (Non-blocking Channel Drain & Webhooks)
| Capability | Detail |
|---|---|
| OpenAI Compatibility | Full /v1/chat/completions (streaming SSE + non-streaming), /v1/embeddings (RAG vectors), and /v1/models support |
| Provider Adapters | Native OpenAI (tool calls + vision), Anthropic Messages API (tool use + vision + streaming tool events), Google Gemini & Vertex AI (function calling + vision + service-account auth), DeepSeek, and OpenAI-Compatible upstreams; plus Azure OpenAI, AWS Bedrock (SigV4 + ConverseStream SSE), Qwen, Zhipu, Kimi, Qianfan, Bailian, Volcengine, Mistral, Cohere, Ollama, Yi, Baichuan, Hunyuan |
| Same-Model Failover | Automatic rotation across provider targets for identical models; zero silent model degradation |
| Conditional Routing & Canary | Target-level conditions matched against X-BettaGate-* request headers (tier/canary/etc.), plus weighted / round_robin / canary strategies for gradual traffic splits |
| Distributed Rate Limiting & Circuit State | Redis-backed sliding window RPM limiter and shared 3-state circuit state across replicas, with graceful in-memory fallback and health probing |
| Monthly Budget Quotas | Per-Organization budgets via /api/v1/budgets (Soft 80% warning header + Hard 100% block), backed by model-price cost estimation on every usage event |
| Prompt Injection Guardrails | Built-in heuristic jailbreak defense (DAN, system prompt leak) plus persisted custom rules, applied on both streaming and non-streaming paths with block, log, and sanitize actions |
| Semantic Cache | Cosine-similarity response cache (in-memory or Redis-shared) that short-circuits deterministic, tool-free, PII-free requests — lower cost and latency; configurable threshold/TTL, cache-hit usage events and metrics |
| Enterprise SSO Login | Single Sign-On via GitHub, Google, and Custom OIDC with Just-In-Time (JIT) user provisioning and JWT cookies |
| Webhook Notifications | HMAC-SHA256 signed real-time Webhook alert engine with exponential backoff retries |
| Web Dashboard Console | Modern Next.js 14 Dark Mode GUI for Providers, API Keys, Routing Rules, Models, Guardrails, Webhooks, Budgets, Prompts, Projects, and Usage Audit |
| i18n | Lightweight en/zh console switching (no dependencies), persisted per browser |
| Security & Privacy | AES-256-GCM provider key encryption, SHA-256 hashed API keys, zero-prompt storage default |
git clone https://github.com/betta-labs/bettagate.git
cd bettagate
# Configure environment (optional; sensible local-dev defaults exist)
cp .env.example .env
# Start Postgres 16, Redis, BettaGate Gateway (:8080/:8081), and Web Console (:3000)
docker-compose up -dAccess the Web Console at http://localhost:3000.
⚠️ Production: never deploy with the development defaults. SetBG_ENV=productionand a strong, backed-upBG_SECURITY_MASTER_SECRET(≥ 32 bytes) — it encrypts provider credentials at rest and signs SSO sessions. See Configuration and Security.
Open the Web Console at http://localhost:3000 and follow the onboarding
guide on the Overview page — it marks each step as complete:
- Configure Provider —
/providers: add an upstream (e.g. DeepSeek) and a real API key. Selected models and 1:1 routing rules are auto-registered. - Create Application —
/applications: an application is the attribution target for per-app cost tracking. - Create API Key —
/api-keys: create a key and bind it to the application (keys without an explicit application attach to the org's default application automatically). - Run a Live Test — use the Live Playground on the Overview page to send
a request, then check
/usage: the model name, the application name, success/error status and estimated cost must match what you sent.
Acceptance: after step 4, the Usage page shows the real model you called, an
application name (never unknown for new keys), and an explainable error code
for any failed request. Run make verify-attribution against the compose stack to
check this automatically.
# Run tests
make test
# Start Gateway Backend
make runfrom openai import OpenAI
# Point OpenAI client to BettaGate Gateway
client = OpenAI(
base_url="http://localhost:8080/v1",
api_key="sk-betta-live-key123"
)
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello BettaGate!"}]
)
print(response.choices[0].message.content)v0.1 scope: BettaGate is an excellent gateway first. Complex budgets, SSO/SCIM, full RBAC, MCP gateway and SaaS billing are deliberately out of scope until real usage data says otherwise.
- OpenAPI 3.0 Spec: Available at
pkg/api/openapi/openapi.yaml - Data Plane:
POST /v1/chat/completions,/v1/embeddings,/v1/images/generations,/v1/audio/speech,/v1/audio/transcriptions,/v1/messages(+/count_tokens),/v1/responses,GET /v1/models - Control Plane:
http://localhost:8081/api/v1(Providers, API Keys, Routing Rules, Models, Guardrails, Webhooks, Auth/SSO, Usage) - Metrics & Health:
GET http://localhost:8081/metrics,GET http://localhost:8081/healthz
BettaGate is configured exclusively through environment variables — no config
file to maintain. Copy .env.example to .env and adjust.
docker-compose up currently wires a small development subset directly in
docker-compose.yml; for full configuration (or bare-metal runs) load .env
into your shell with set -a && source .env && set +a before make run.
| Variable | Purpose | Default |
|---|---|---|
BG_ENV |
development | staging | production | test |
development |
BG_GATEWAY_ADDR / BG_ADMIN_ADDR |
Data plane / control plane listen addresses | :8080 / :8081 |
BG_DATABASE_URL |
PostgreSQL connection (auto-migrated at startup) | local dev DSN |
BG_REDIS_ADDR |
Redis for distributed rate limit, circuit & cache state | redis://localhost:6379 |
BG_SECURITY_MASTER_SECRET |
AES-256 credential encryption + SSO JWT signing (required in prod, ≥ 32 bytes) | dev-only default |
BG_ADMIN_TOKEN |
Bearer token for control-plane API automation | empty (dev bypass) |
BG_LOG_LEVEL / BG_LOG_DEV |
Log level and console vs JSON output | info / true |
BG_OTEL_EXPORTER_OTLP_ENDPOINT |
OpenTelemetry OTLP/HTTP endpoint | empty (log-only tracing) |
BG_CACHE_ENABLED / BG_CACHE_TTL / BG_CACHE_SIMILARITY_THRESHOLD |
Exact/semantic response cache | off |
BG_CACHE_EMBEDDING_* |
Embedding endpoint for semantic cache | empty (semantic off) |
BG_RETENTION_DAYS |
Prune usage/audit events older than N days (GDPR) | 0 (disabled) |
BG_SSO_* |
GitHub / Google / OIDC SSO credentials and domain allow-list | empty (SSO off) |
BG_WEB_URL |
Web console origin for SSO redirects | http://localhost:3000 |
- Credentials at rest: provider API keys are encrypted with AES-256-GCM
using
BG_SECURITY_MASTER_SECRET; raw keys never appear in logs, traces or API responses, and the UI only ever shows masked values. - API keys: gateway keys are stored as SHA-256 hashes with optional expiry and per-key/app/org budget quotas.
- Secrets in logs: full prompts, responses and credentials are not logged by default; PII anonymization and redaction are applied on both streaming and non-streaming paths.
- Reporting: if you find a vulnerability, report it privately (see Contributing) — do not open a public issue with exploit details.
- Capability honesty: failover only happens between same-model targets; BettaGate never silently downgrades a premium model.
Contributions are welcome. The architecture and boundaries are documented in
AGENTS.md and docs/architecture.md —
please read them before opening a PR.
- Design principles: vertical slices over horizontal abstraction, capability-sacred routing, explicit configuration, security by default.
- Tests: table-driven unit tests for pure logic, contract tests for every
provider adapter, and the existing integration/E2E suites must stay green
(
make test). - Documentation: keep
README,README-zh,docs/domain-model.mdand the gap analysis (docs/GAP-ANALYSIS.md) accurate when behavior changes.
BettaGate is licensed under the Apache License 2.0.
Attribution and copyright notices are maintained in NOTICE.
Third-party libraries and components retain their own licenses (see
go.mod and web/package-lock.json).