For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Custom

Configure agentgateway for providers without built-in support that implement the OpenAI API format.

Use this page for providers that implement the OpenAI API format but do not have a first-class provider: support yet. For built-in providers such as Baseten, Cerebras, Cohere, DeepInfra, DeepSeek, Fireworks AI, Groq, Hugging Face, Mistral, OpenRouter, Together AI, xAI, and Ollama, use the dedicated provider pages instead.

Before you begin

You also need the following prerequisites.

• An API key for your chosen provider, unless you are pointing to a local endpoint such as vLLM or LM Studio.

Configuring a custom provider

With a custom provider, you provide the API endpoint and a list of formats it supports. Agentgateway will automatically handle mapping between the incoming format and the supported formats.

Below shows an example of connecting to Perplexity, which exposes an OpenAI-compatible API for search-augmented models and does not currently have a first-class provider.

cat > /tmp/test-perplexity.yaml << 'EOF'
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "*"
    provider:
      custom:
        formats:
          # Indicate this provider supports the completions API. With no `path` specified, this defaults to <baseUrl>/chat/completions
          - type: completions
          # Indicate this provider supports the messages API, on a custom path /messages-api
          # - type: messages
          #   path: /messages-api
          # All possible APIs:
          # - type: embeddings
          # - type: responses
          # - type: realtime
          # - type: anthropicTokenCount
          # - type: rerank
    params:
      apiKey: "$PERPLEXITY_API_KEY"
      model: llama-3.1-sonar-large-128k-online
      baseUrl: "https://api.perplexity.ai"
EOF