LiteLLM Proxy gateway details

An open-source LLM gateway/proxy layer that unifies many providers into an OpenAI-style interface; ideal for teams that want to self-host a relay and control their own keys and logs.

Best for: For dev teams with ops capacity who want control over API keys, logs, routing, and permissions.

Current status

Self-hosted

Latency reference

Depends on your deployment

/models check

Self-hosted configuration

Last verified

2026-07-01

Status notes

An open-source proxy layer; real latency and availability depend on your deployment and upstream providers.

Pricing mode

Open-source and self-hosted; cost comes from servers and the underlying model providers.

Price tags

High stability, Self-hosted

Free quota

No unified free quota; depends on the providers you connect.

Usually you create an API key in the platform console, then replace the SDK's baseURL, model name, and auth details per the docs. Test streaming, function calling, error codes, and rate limits before launch.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: "see the platform docs"
});

const response = await client.chat.completions.create({
  model: "gpt-4.1-mini",
  messages: [{ role: "user", content: "Test one call through an AI gateway." }]
});

• Supported models: OpenAI, Claude, Gemini, DeepSeek, Grok, Qwen, Moonshot, Doubao and more.
• Capabilities: OpenAI-compatible, Streaming, Function calling, Embeddings, Log query.
• Documentation completeness: High; editorial stability score 4.0 / 5.

• Self-hosting means taking on ops, security, rate-limiting, and key management; the learning curve is high for individuals.
• Third-party relay pricing, models, and availability can change, so re-check them periodically.

Is LiteLLM Proxy compatible with the OpenAI API?

This entry is marked as supporting the OpenAI-compatible format, but the exact base_url, model names, and parameters still follow the official docs.

Is LiteLLM Proxy ready for production?

Run a small-traffic load test, rate-limit test, and error-code test first, and confirm the billing rules; keep a backup provider for important workloads.