One integration
Applications talk to a single endpoint instead of carrying a separate client for every provider.
LLM Router
LLM Router sits between your applications and the model providers. You integrate once; routing, fallback, keys, limits and cost tracking are handled centrally, and swapping the underlying model stops being an application change.
Routing and fallback
Send each request to the model that suits it, and fail over automatically when a provider degrades.
Spend you can see
Usage and cost are tracked per key, per team and per route, with budgets and rate limits.
Why a gateway
Teams accumulate model integrations quickly: one provider for chat, another for embeddings, something open-weight self-hosted for sensitive data. Each brings its own SDK, keys and failure modes. Centralising that means an application asks for a capability and the gateway decides how to serve it.
Routing rules
Route by task, cost ceiling, latency target, or data sensitivity. A summarisation call can go to a cheaper model while a reasoning-heavy call goes to a stronger one, and anything touching regulated data can be pinned to a self-hosted model.
Operational control
Per-key budgets and rate limits, request and response logging with redaction, and usage reporting by team. When a provider has an incident, traffic moves to the fallback instead of failing.
The numbers.
Providers
- Hosted models
- Claude, GPT and other major APIs
- Open models
- Self-hosted through vLLM or Ollama
- Interface
- OpenAI-compatible REST API
Controls
- Budgets
- Per key, team and route
- Reliability
- Automatic failover and retries
- Deployment
- Managed or on your own infrastructure
Straight answers.
Do we have to rewrite our application?
No. The API is OpenAI-compatible, so in most cases you change the base URL and the key.
Can we keep some traffic on our own models?
Yes. That is a common configuration: sensitive requests go to a self-hosted open model and everything else uses hosted APIs.
What happens if a provider goes down?
The request fails over to the next model in the route. You can define the order per route.