Skip to content

Latest commit

 

History

History
41 lines (32 loc) · 2.17 KB

api.md

File metadata and controls

41 lines (32 loc) · 2.17 KB

API and administration

fastllm-proxy --config litellm_config.yaml --host 127.0.0.1 --port 4000

Point clients at it as an OpenAI endpoint:

curl http://localhost:4000/v1/chat/completions \
  -H 'Authorization: Bearer sk-...' \
  -H 'content-type: application/json' \
  -d '{"model":"Qwen/Qwen3-1.7B","stream":true,"messages":[{"role":"user","content":"hi"}]}'

Reload after the model set changes — no restart, no dropped streams:

kill -HUP $(pgrep -x fastllm-proxy)

The machine-readable version of everything below is openapi.json, served by the running control plane at GET /openapi.json with Swagger UI at /docs. It is checked against the router by tests/openapi.rs in both directions — a route without a spec entry fails the build, and so does a spec entry whose route no longer exists.

In this section

Interactive API reference Swagger UI over openapi.json, browsable here without a deployment
The endpoints clients call The proxied surface, what is not proxied, the response cache, rate-limit headers and retries
Admin API Models, backends, keys, principals, prices, live health, and the audit log
Routing rules The rule grammar, and the dry-run that answers which rule would decide
Authentication, sessions and TLS Sessions, per-route permissions, encryption at rest, and which listener must be TLS
The control-plane protocol /usage, /health-report, budgets and rate-limit reconciliation

Provider base URLs and the translation limits moved to Providers. Every flag is in the command-line reference.