API and administration

fastllm-proxy --config litellm_config.yaml --host 127.0.0.1 --port 4000

Point clients at it as an OpenAI endpoint:

curl http://localhost:4000/v1/chat/completions \
  -H 'Authorization: Bearer sk-...' \
  -H 'content-type: application/json' \
  -d '{"model":"Qwen/Qwen3-1.7B","stream":true,"messages":[{"role":"user","content":"hi"}]}'

Reload after the model set changes — no restart, no dropped streams:

kill -HUP $(pgrep -x fastllm-proxy)

The machine-readable version of everything below is openapi.json, served by the running control plane at GET /openapi.json with Swagger UI at /docs. It is checked against the router by tests/openapi.rs in both directions — a route without a spec entry fails the build, and so does a spec entry whose route no longer exists.

In this section


Interactive API reference	Swagger UI over `openapi.json`, browsable here without a deployment
The endpoints clients call	The proxied surface, what is not proxied, the response cache, rate-limit headers and retries
Admin API	Models, backends, keys, principals, prices, live health, and the audit log
Routing rules	The rule grammar, and the dry-run that answers which rule would decide
Authentication, sessions and TLS	Sessions, per-route permissions, encryption at rest, and which listener must be TLS
The control-plane protocol	`/usage`, `/health-report`, budgets and rate-limit reconciliation

Provider base URLs and the translation limits moved to Providers. Every flag is in the command-line reference.