Flip through the API documentation of any model provider—DeepSeek, Tongyi Qianwen, Kimi, Zhipu—or local deployment tools like vLLM, Ollama, or LM Studio, and you will almost always find this statement: "We provide an OpenAI-compatible interface." Behind these words lies one of the most significant de facto standards in AI engineering over the past two years: everyone has independently shaped their interfaces to look like OpenAI's. Consequently, code written for OpenAI can often be adapted to call other providers' models simply by changing the endpoint URL and API key.
This article clarifies exactly what this means, how far compatibility extends, and where its boundaries lie—because the term "compatible" is frequently understood with excessive optimism.
The OpenAI-compatible interface is an industry-wide de facto standard: no committee mandated it, yet everyone follows suit.
It's Not a Standard; It's a "De Facto Convention"
First, let's dispel a misconception: The OpenAI-compatible interface is not an open standard officially published by OpenAI for others to follow. OpenAI has never defined anything called a "compatibility specification." Its emergence is the result of market selection:
OpenAI's API was the first to gain widespread adoption. A massive ecosystem of tools, SDKs, and frameworks (such as LangChain and various clients) were built around its /v1/chat/completions endpoint. For newcomers who want this existing ecosystem to work directly with their models, the most efficient path is simple: make your interface identical to OpenAI's. Thus, the messages array, the model field, the stream parameter, SSE streaming format, and Authorization: Bearer authentication became the default structure. This is a classic case of "the first mover defines the interface shape; later movers actively align."
The benefits are tangible: Code you've written using OpenAI's official SDK usually requires only two changes to switch providers—
from openai import OpenAI
# Calling OpenAI
client = OpenAI(api_key="sk-openai-xxx")
# Switching to DeepSeek: change only base_url and key
client = OpenAI(
api_key="sk-deepseek-xxx",
base_url="https://api.deepseek.com/v1",
)
# Every client.chat.completions.create(...) call below stays exactly the sameThis explains why AI gateways universally choose to "expose a unified OpenAI format internally"—it is currently the interface dialect with the broadest ecosystem coverage.
What Are the Core Elements of Compatibility?
When the industry refers to "OpenAI compatibility," it implicitly means aligning on these most commonly used capabilities:
- Chat Completions Interface:
POST /v1/chat/completions, using amessagesarray structured withrole(system/user/assistant) andcontent; - Streaming Responses: When
stream: true, returning SSE-formatted chunks prefixed withdata:and ending with[DONE]; - Authentication: HTTP header
Authorization: Bearer <key>; - Basic Parameters:
temperature,max_tokens,top_p,stop, etc.; - [Tool Calling](/wiki/tool-calling): The
tools/tool_callsfields (compatibility here varies significantly, see below); - Usage Reporting: The
usageobject in the response (prompt_tokens/completion_tokens).
Covering these points allows most applications written for OpenAI to run without modification. Precisely because of this, local deployment frameworks treat compatibility as a standard feature—vLLM, Ollama, and LM Studio launch services that speak the OpenAI format by default. Your application remains unaware whether it is running against cloud-based GPT or an open-source model on your local machine.
"Compatible" Does Not Mean "Equivalent": Where Are the Boundaries?
This is where we need to temper expectations. Compatibility applies to interface structure, not model behavior, and certainly not full feature parity. The most common incompatibility pitfalls during actual migration include:
| Domain | Common Differences |
|---|---|
| Tool Calling | Some models differ from OpenAI in tool calling format or stability; support for parallel tool calls and streaming tool calls varies. |
| Structured Output | response_format / JSON mode is not implemented by all providers, or behaves differently. |
| Multimodal | Field extensions for image and audio inputs vary significantly across vendors, leading to poor compatibility. |
| Parameter Details | Ranges for temperature, limits on max_tokens, and special parameters (e.g., reasoning control) differ from provider to provider. |
| Proprietary Capabilities | Vendor-specific features like caching markers, internet access, or built-in tools are entirely absent in the compatible interface layer. |
| Model Behavior | With identical prompts, output styles and instruction-following capabilities vary wildly between models—this is a model issue, not an interface one. |
In short: "Changing just the base_url makes it run" is often true; "changing just the base_url yields the same results" is almost never true. After migration, you must re-test with your own evaluation dataset, especially for applications relying on tool calling and structured output.
Switching to a compatible interface means the code runs, but not that performance is identical—prompts often require retuning for the new model.
How to Use It Effectively
- Use the Official SDK with a Configurable `base_url`: Avoid manually constructing HTTP requests. Instead, leverage the official OpenAI SDK and configure
base_url,api_key, andmodelas variables; this minimizes switching costs. - Abstract Your Own Model Client Layer: Even if APIs are compatible, it is recommended to wrap a thin abstraction layer within your application. This isolates minor differences between providers (such as tool invocation formats), meaning you only need to modify this single layer when changing vendors.
- Delegate to an AI Gateway: For multi-model scenarios, use an AI gateway as a unified entry point. Your application speaks "OpenAI dialect" exclusively, while the gateway handles translation and routing. See "Designing an AI API Gateway from Scratch".
- Save Costs During Local Development: Use Ollama to run local open-source models (which are OpenAI-compatible) during development and debugging phases. Your code remains unchanged; simply switch to cloud flagship models when deploying to production.
Configuring `base_url`, `api_key`, and `model` as variables means switching models is just a matter of changing configuration—this is the primary benefit of compatible interfaces.
Target Audience and Alternatives
This approach suits developers who need to switch between multiple models or wish to avoid locking their code into a single vendor. Keep these alternatives and exceptions in mind:
- Anthropic has its own native API format (Messages API). While fields differ slightly from OpenAI's, the official provider offers an OpenAI-compatible endpoint, and major gateways can handle conversion, making integration straightforward.
- When leveraging deep proprietary capabilities, compatible interfaces become a constraint—they only cover common denominators and exclude vendor-specific "killer features." In such cases, use the native SDK.
- Framework Abstractions (e.g., LangChain, LlamaIndex) add another layer of abstraction on top of compatible interfaces, making multi-model switching even easier. However, introducing these frameworks incurs its own costs; choose based on your specific needs.
Frequently Asked Questions
Q: Do I need to use a proxy/vpn to access domestic models via the compatible interface? A: No. API services for domestic models like DeepSeek, Tongyi (Tongyi Qianwen), and Kimi are hosted within China. You can directly access their base_url. This is one of the major conveniences they offer compared to OpenAI's official interfaces in mainland China.
Q: Why do tool invocations sometimes fail with certain "compatible" models? A: Tool invocation support varies significantly across implementations. While interface fields may be aligned, different models have varying levels of training for tool usage. Consequently, the output tool_calls might be non-standard or unstable. Always specifically test your tool invocation pipeline against the target model before going to production.
Q: Is the OpenAI-compatible interface for local models reliable? A: The interface layer itself is mature (vLLM, Ollama, etc., have been widely used). However, note that local open-source models are typically weaker than cloud flagship models in areas like tool invocation, long-context handling, and instruction following. Interface compatibility cannot bridge the gap in underlying capabilities.
Summary
The OpenAI-compatible interface is a de facto standard formed by market forces: newcomers actively align their API structures with pioneers to gain the convenience of "switching models just by changing an address." However, remember that this compatibility applies to the shape of the interface, not model behavior—while your code will likely run after migration, performance must be re-validated. By configuring base_url, wrapping a custom client layer, or delegating entirely to a gateway, you can enjoy ecosystem benefits without being locked into any single vendor.