For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

API types

Supported LLM API endpoint types and route configurations

Agentgateway natively supports multiple LLM API endpoint types. These are automatically exposed on the gateway, and translated as appropriate based on the provider.

The following API types have dedicated guides:

• Chat completions: The OpenAI /v1/chat/completions endpoint. This is the most widely used API type for text generation and chat applications.
• Responses: The OpenAI /v1/responses endpoint for stateful, multi-step model interactions.
• Messages: The Anthropic /v1/messages endpoint for Claude models.
• Embeddings: The OpenAI-compatible /v1/embeddings endpoint for creating vector representations of text.
• Realtime: The OpenAI Realtime API for low-latency, streaming voice and text interactions over WebSockets.
• Rerank: The Cohere-compatible /v2/rerank endpoint for ranking documents by relevance to a query.
• Models: The OpenAI-compatible /v1/models endpoint for listing available models.
• Token count: The Anthropic /v1/messages/count_tokens endpoint for estimating input tokens.
• Passthrough: Forwards requests directly to the backend provider without transformation.