AI Proxy Operator

Kubernetes operator for AI Proxy — manage channels, models, groups, and API keys declaratively via CRDs.

Deploy

Option 1: Remote (no clone)

kubectl create namespace aiproxy-system
kubectl apply -k 'https://github.com/anas1412/aiproxy-operator//deploy?ref=main'

Option 2: Clone and deploy

git clone https://github.com/anas1412/aiproxy-operator.git
cd aiproxy-operator
kubectl create namespace aiproxy-system
kubectl apply -k deploy/

Both commands provision everything into the aiproxy-system namespace:

Layer	Resources
CRDs	channels, models, groups, apikeys
Dependencies	Postgres 17 (StatefulSet + Service), Redis 7 (Deployment + Service), postgres-creds Secret
AIProxy	Deployment + Service on port 3000
Operator	Deployment (pulls ghcr.io/anas1412/aiproxy-operator:latest), ServiceAccount, ClusterRole, ClusterRoleBinding

Customizing

Fork or clone if you need to change defaults before deploying:

File	Field	Default
deploy/dependencies/postgres-secret.yaml	stringData.password, stringData.dsn	aiproxy / postgres://postgres:aiproxy@postgres:5432/aiproxy
deploy/operator/deployment.yaml	--aiproxy-endpoint	http://aiproxy.aiproxy-system:3000
deploy/dependencies/postgres.yaml	storage	10Gi

CLI (aiproxyctl)

Generate, validate, and inspect CRD YAML files — no hand-writing needed. Included in the operator image.

kubectl exec -n aiproxy-system deploy/aiproxy-operator -- aiproxyctl scaffold channel openai -k my-secret -m gpt-4,gpt-4o

scaffold

Generate valid CRD YAML for GitOps. Write to stdout or -o file.yaml:

aiproxyctl scaffold channel openai -k my-secret -m gpt-4,gpt-4o -b https://api.openai.com
aiproxyctl scaffold model gpt-4 -o openai -t chat --rpm 500 --input-price 2.50 --output-price 10.00
aiproxyctl scaffold group my-team -s default
aiproxyctl scaffold apikey my-key -g my-team --quota 100

validate

Check files for required fields, enums, and ranges before applying:

aiproxyctl validate channel.yaml model.yaml -v

status

Inspect deployed resources via the admin API:

aiproxyctl status             # overall summary
aiproxyctl status channels    # table: ID, name, type, balance
aiproxyctl status models      # table: model, owner
aiproxyctl status groups      # table: ID, status, sets
aiproxyctl status apikeys     # table: ID, name, group, status

test

Run live channel tests via the admin API:

aiproxyctl test channel 1        # test all models on channel ID 1
aiproxyctl test channel 1 gpt-4  # test a specific model
aiproxyctl test all              # test every channel

Global flags: --api-server (default http://localhost:3000), --api-key.

CRDs

API group: aiproxy.labring.io/v1alpha1. All are namespaced.

CRD	Description
Channel	Upstream LLM provider endpoint — type, base URL, API key ref, models, priority, error-rate thresholds. Supports 50+ providers (OpenAI, Anthropic, Gemini, Azure, DeepSeek, etc.)
Model	Model definition — owner, type (chat/embedding/image/audio/video), pricing per 1K tokens, RPM/TPM limits, timeouts, capabilities config
Group	Tenant — routing sets, RPM/TPM ratio multipliers, balance alert threshold
APIKey	Token linked to a Group — creates a Secret with the generated key in the same namespace

The operator writes directly to the AIProxy database. Channel and Model controllers also watch Secrets — when a referenced API key Secret changes, the channel is re-reconciled. APIKey CRs can be created in any namespace; Channel, Model, and Group CRs must be in the operator namespace (aiproxy-system by default).

Operator flags

Flag	Env var	Default
--aiproxy-endpoint	AIROXY_ENDPOINT	http://aiproxy:3000
--namespace	OPERATOR_NAMESPACE	aiproxy-system
--metrics-addr	METRICS_ADDR	:8080

Health probes at :8081 (/healthz, /readyz). Database connection via SQL_DSN env var (same Postgres AIProxy uses).

Examples

Channel

apiVersion: aiproxy.labring.io/v1alpha1
kind: Channel
metadata:
  name: my-openai
  namespace: aiproxy-system
spec:
  type: openai
  baseURL: https://api.openai.com
  keySecretRef:
    name: openai-api-key
  models:
    - gpt-4o
    - gpt-4o-mini
  priority: 100
  sets:
    - default

Model

apiVersion: aiproxy.labring.io/v1alpha1
kind: Model
metadata:
  name: gpt-4o
  namespace: aiproxy-system
spec:
  owner: openai
  type: chat
  channelRefs:
    - my-openai
  rpm: 500
  tpm: 100000
  price:
    inputPrice: 2.50
    outputPrice: 10.00

Group

apiVersion: aiproxy.labring.io/v1alpha1
kind: Group
metadata:
  name: my-team
  namespace: aiproxy-system
spec:
  status: Enabled
  sets:
    - default
  rpmRatio: 1.0
  tpmRatio: 1.0

APIKey

apiVersion: aiproxy.labring.io/v1alpha1
kind: APIKey
metadata:
  name: my-key
  namespace: my-app
spec:
  groupRef: my-team
  status: Enabled
  models:
    - gpt-4o
  quota: 100.0

The operator generates a 48-character alphanumeric key and creates a Secret in the same namespace:

apiVersion: v1
kind: Secret
metadata:
  name: my-key
  namespace: my-app
  labels:
    app.kubernetes.io/managed-by: aiproxy-operator
  ownerReferences:
    - apiVersion: aiproxy.labring.io/v1alpha1
      kind: APIKey
      name: my-key
      controller: true
data:
  AIROXY_KEY: c2st...       # 48-char generated token
  AIROXY_ENDPOINT: aHR0cC4. # aiproxy service URL
  AIROXY_GROUP: bXktdGVhbQ== # groupRef value

Architecture

The operator and AIProxy share the same Postgres database but run as independent containers. An operator crash does not affect in-flight proxy requests.

aiproxy-system namespace:
┌───────────┐  ┌──────────────────────┐
│  AIProxy   │  │  Operator             │
│  :3000     │  │  :8080 metrics        │
└─────┬─────┘  │  :8081 health          │
      │        └───────────┬───────────┘
      │                    │
      └─────────┬──────────┘
                │
         ┌──────▼──────┐
         │  PostgreSQL  │
         └──────┬──────┘
                │
         ┌──────▼──────┐
         │    Redis     │
         └─────────────┘

Any namespace:
┌──────────────┐
│  APIKey CR    │──► Secret (AIROXY_KEY, AIROXY_ENDPOINT, AIROXY_GROUP)
└──────────────┘     with controller ownerReference

Local Testing (k3d)

Spin up a local cluster and deploy the operator in under 2 minutes:

# 1. Create a local k3d cluster
k3d cluster create aiproxy-dev \
  --k3s-arg "--disable=traefik@server:0"

# 2. Deploy
sudo kubectl create namespace aiproxy-system
sudo kubectl apply -k 'https://github.com/anas1412/aiproxy-operator//deploy?ref=main'

# 3. Access the dashboard
sudo kubectl port-forward -n aiproxy-system svc/aiproxy 3000:3000
# → http://localhost:3000 (admin key: aiproxy)

# 4. Test the CLI
kubectl exec -n aiproxy-system deploy/aiproxy-operator -- aiproxyctl help

Clean up:

k3d cluster delete aiproxy-dev

License

MIT — see LICENSE.