Providers

Adding a provider is a row in a table — not a code change, not a release. Anything speaking the OpenAI API is already supported whether or not it is on the list below; the list exists so you do not have to go and find the base URL.

The Providers screen groups what you are actually talking to, by host. A provider is a grouping this screen invents rather than a record the API models — the database has models and backends, and "OpenRouter" is what a human calls every backend pointing at openrouter.ai. What each card answers is the question you actually have: how many models ride on it, whether a credential is set (never the credential itself), and how many of its backends are up.

The catalogue

80 providers work today — 78 reached as-is, 2 through their own wire format. The count and these tables are checked against each other by tests/doc_claims.rs, so the number cannot drift away from the rows.

A caveat these tables are explicit about, because the number is otherwise a boast: "works" here means "is a configuration row that this proxy will forward to correctly", which follows from the endpoint being OpenAI-shaped. The ones exercised against real traffic in this repo's tests and on its dev cluster are marked ✓. The rest carry the base URL their vendor documents — check it against their docs before pasting it into production, because vendors move them and this file cannot notice.

Adding one

Two ways, same result. A backend belongs to the model it serves, so you add a model first and then a backend under it.

In the UI, on Models:

By API:

curl -sk -b /tmp/ck -X POST https://control:4001/admin/models \
  -H 'content-type: application/json' -d '{"name":"kimi-k2"}'    # -> {"id":7}

curl -sk -b /tmp/ck -X POST https://control:4001/admin/models/7/backends \
  -H 'content-type: application/json' -d '{
    "api_base": "https://api.moonshot.ai/v1",
    "upstream_model": "moonshot-v1-128k",
    "upstream_api_key": "sk-..."
  }'

Two backends under one model become one load-balanced pool — and two entries sharing a model_name in a LiteLLM config import to exactly that. It is the whole mechanism behind failover and traffic splitting — see virtual models for routing between different models.

Credentials

upstream_api_key is encrypted at rest with FASTLLM_ENCRYPTION_KEY before it reaches Postgres, and the admin API never reads one back — the UI shows whether a credential is set, never what it is.

Two knobs exist because not every vendor puts the key in authorization: Bearer:

Azure OpenAI is the case that needs both: auth_header: api-key and auth_scheme: "". Amazon Bedrock, despite the reputation, needs neither — its OpenAI-compatible endpoint takes a Bedrock API key as an ordinary bearer token, so it is a plain row like any other and there is no request signing.

Two providers speak their own language

Anthropic and Gemini do not expose an OpenAI-shaped endpoint, so they are reached through a translator rather than a base URL:

flowchart LR
    C["client<br/>OpenAI request"] --> P{"backend<br/>protocol?"}
    P -->|openai| B1["upstream<br/>bytes forwarded unchanged"]
    P -->|anthropic| T1["translate →<br/>Messages API<br/>x-api-key"] --> B2["api.anthropic.com"]
    P -->|gemini| T2["translate →<br/>generateContent<br/>model in the URL"] --> B3["generativelanguage<br/>.googleapis.com"]
    B1 --> R1["response returned<br/>byte-for-byte"]
    B2 --> R2["SSE re-framed<br/>to OpenAI chunks"]
    B3 --> R2

Tool calling translates in both directions, streaming included, as do image and audio inputs. Translation is opt-in per backend: it costs parsing, and the whole latency argument for this gateway rests on not parsing. An openai backend's response body is never deserialised, which is why the two paths in that diagram are drawn differently — one forwards bytes, the other builds them.

The translation limits, field by field, are in the API reference.

Verified base URLs

Most providers are OpenAI-compatible, so they need no code at all — just a backend row pointing at their base URL. That includes OpenRouter, which itself fronts Anthropic, Gemini and several hundred other models in OpenAI format:

curl -X POST https://control/admin/models/$MODEL_ID/backends \
  -H 'content-type: application/json' -b "$SESSION" \
  -d '{"api_base":"https://openrouter.ai/api/v1",
       "upstream_model":"anthropic/claude-sonnet-4",
       "upstream_api_key":"sk-or-..."}'

Verified base URLs for the OpenAI-compatible set:

Bedrock needs no request signing. Its OpenAI-compatible endpoint takes a Bedrock API key as an ordinary bearer token, so it is a plain backend row like any other — create the key in the Bedrock console and put it in upstream_api_key.

What is deliberately absent

Deliberately absent, and not counted: providers whose API is not OpenAI-shaped and would need a fourth translator in src/protocol/ — Replicate, Predibase, Petals, Triton, WatsonX, OCI Generative AI, AWS SageMaker. Also absent are the non-LLM services a gateway has no business proxying blind: speech (Deepgram, ElevenLabs), image generation (Stability, Black Forest Labs, Recraft, Fal, RunwayML), vector stores (Milvus), and other people's gateways (Helicone, LiteLLM itself). Counting those would inflate the number without making anything work.

Where next