For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
LLM consumption
Overview of LLM provider integration and consumption patterns
Agentgateway can act as a feature rich AI/LLM gateway, acting as a proxy between your applications and LLM providers. This enables connecting to thousands of LLM model through a unified interface providing governance, observability, and reliability controls.
About
Overview of supported LLM providers and their capabilities
API types
Supported LLM API endpoint types and route configurations
Providers
Configure agentgateway for first-class, OpenAI-compatible, and self-hosted LLM providers
Virtual key management
Issue API keys with per-key token budgets and cost tracking (also known as virtual keys).
Virtual models
Configure virtual models with weighted, failover, and conditional routing in simplified LLM mode.
Model costs
Price LLM requests with a model cost catalog and expose realized USD costs in logs, traces, metrics, …
Transform requests
Dynamically compute and set LLM request fields using CEL expressions.
Guardrails
Protect LLM interactions with prompt guards that evaluate and filter requests and responses for …
Observe traffic
Get prompt logging, cost tracking, and an audit trail for LLM traffic.
LLM playground
Send a test chat completion through the gateway in the agentgateway admin UI.
Routing-based configuration for LLMs
Use traditional HTTP routing configuration for advanced use cases like path-based routing and …
