Open source LLM gateway

One model gateway.Run it yourself.

Connect your apps, scripts, and agents through one endpoint. Manage model mappings, upstream providers, and access from the included console.

View setup guideDocker / Cloudflare Workers
LutraGate console · Example data
The actual LutraGate console overview with example request counts, token usage, and latency.

Gateway features

Manage routing and access in one place.

Keep one client endpoint

Map a client model name to upstream models. Use OpenAI Chat Completions, Responses, Anthropic Messages, or native Gemini requests.

Switch channels when upstream fails

Route across eligible channels by weight. Retry on another eligible channel after rate limits, server errors, timeouts, or network errors.

Share access and track usage per user

Grant household members access to specific upstream keys. Each user sees their permitted models and can track usage by access key.

How a request reaches its provider

Model mappings and user grants determine which channels can handle a request. Another eligible channel can take over after a retryable error.

ClientPOST /v1/chat/completions
LutraGateFilter by grant, then route by weight
Channel AFirst attempt429 · Retry triggered
Channel BEligible channelRetry channel

From deployment to first request

  1. Start the gateway

    Run Docker with SQLite on your server, or deploy on Cloudflare Workers with D1.

  2. Configure the console

    Create the first administrator, add providers, upstream keys, and model mappings, then grant users access.

  3. Connect a client

    Set the gateway as the client API base URL, enter the user’s access key, and send a request with a configured model name.

Read the Workers setup guide

Questions before you deploy

Where can I run LutraGate?

Run it with Docker and SQLite on your own server, or deploy on Cloudflare Workers with D1. Both options include the web console.

Can I connect an existing OpenAI compatible client?

Yes. Point the client API base URL at the gateway, provide an lgk_ access key, and select a configured client model name. Chat Completions supports streaming.

What happens if an upstream request fails?

For 429 and 5xx responses, timeouts, and network errors, the gateway tries another eligible channel. It passes through other upstream 4xx responses.

Can my household share one gateway?

Yes. An administrator can create individual users and grant access to specific upstream keys. Users manage their own access keys and see the models they can reach.

Where can I see usage?

The console shows requests, token usage, and latency. Users can see their own usage by access key, and administrators can see activity across the gateway.

Which open source license applies?

The server, Worker, and admin console use AGPL 3.0. The shared and core packages use the MIT license.

Run your gateway on your own terms.

Start with Docker or Cloudflare Workers, then connect your clients using the provider API keys you already have.

View setup guide