Kong AI Gateway is the AI extension of Kong Gateway — the open-source API gateway used by ~33% of Fortune 500 companies. It adds LLM routing, token rate limiting, semantic caching, prompt decoration, and policy-as-code guardrails to Kong's existing enterprise infrastructure. Kong v3.14 (April 2026) added A2A (Agent-to-Agent) protocol support. If your organization already runs Kong, this is the lowest-friction path to an enterprise-grade LLM control plane.
Why It's in Assess
Kong AI Gateway is powerful but expensive to operate for teams starting fresh. Its value proposition is strongest for organizations already running Kong's control plane — adding AI routing becomes a configuration change, not a new service. For greenfield agentic engineering platforms, purpose-built options (Bifrost, Portkey, LiteLLM) are simpler to stand up:
- Policy-as-code governance: Declarative YAML/Terraform IaC for AI routing rules, rate limits, cost policies, and guardrails — enforced org-wide via Kong's existing federation model
- Performance: Kong claims 65% lower latency than Portkey and 86% lower than LiteLLM at comparable loads, though these are vendor-published benchmarks
- Token rate limiting: Per-consumer, per-route token-per-minute (TPM) and request-per-minute (RPM) limits, the same primitive Kong applies to traditional API traffic
- Semantic caching: Cache LLM responses at the gateway; configurable similarity threshold
- A2A support (v3.14, April 2026): Native Agent-to-Agent protocol routing — agents can invoke other agents through Kong's control plane with the same auth and observability as model requests
- Content moderation plugins: PII detection, prompt injection detection, content filtering — pluggable, open-source
- Unified API + AI management: Single control plane for REST, gRPC, WebSocket, and LLM traffic — audit logs, RBAC, and OIDC/SSO already present from the API gateway layer
Limitations
- Operational complexity: Kong requires its own deployment (DB-backed or DB-less), admin API, and portal. For teams that just want an LLM proxy, this is significant overhead.
- No MCP gateway: As of April 2026, Kong does not ship a first-class MCP server registry or per-team MCP tool access control. A2A support was just added in v3.14.
- Enterprise features behind paywall: Multi-tenant isolation, advanced RBAC, and FIPS compliance require Kong Enterprise (commercial). The open-source tier covers routing and basic rate limiting.
- Provider breadth: Kong AI plugins cover the major providers (OpenAI, Anthropic, Azure, Cohere, Mistral, Llama) but not the long tail that LiteLLM reaches.
When to Choose Kong AI Gateway
- Your org already runs Kong and wants to avoid a second gateway for LLM traffic
- You need unified API and LLM governance under one audit log and RBAC system
- You have a dedicated platform team that can operate Kong's control plane
- You require FIPS 140-2 compliance or other enterprise certifications (Kong Enterprise)
When NOT to Choose Kong AI Gateway
- Greenfield: simpler options (Bifrost, Portkey) take minutes to deploy vs. Kong's hours
- You need MCP tool governance today — Kong has A2A but no MCP registry
- Cost-conscious: Kong Enterprise pricing is significant; open-source Kong lacks the features that make AI governance viable at scale
Key Characteristics
| Property | Value |
|---|---|
| License | Apache 2.0 (OSS), Commercial (Enterprise) |
| Language | Lua / Go |
| GitHub | Kong/kong |
| A2A support | v3.14, April 2026 |
| Website | konghq.com/products/kong-ai-gateway |
| Provider | Kong Inc. |