Control CoreDocs

AI Gateway

AI Gateway is Control Core's enterprise layer for governing LLM, MCP, RAG, and agent traffic. It is built on AI Pilot (aigateway.envoyproxy.io/v1) and ships inside every Bouncer — there is no separate gateway product to install. Operators configure routes, credentials, guardrails, and resilience from the Control Plane /pilot cockpit (sidebar: AI Gateway).

Previously documented as "AI Pilot." The product surface, env vars, and audit events retain pilot / AI_PILOT identifiers in APIs; customer docs use AI Gateway as the primary name.

What the gateway delivers

CapabilityWhere it is managed
Multi-provider LLM routing (OpenAI, Anthropic, Azure, Bedrock, Vertex, OpenAI-compatible)AI Gateway → Routing & models
Token budgets, rate limits, circuit breakersRouting & models + Resilience
Prompt guard, DLP, PII redaction (Rego + Bouncer intercept)Guardrails & DLP + Controls
Intelligent MCP proxy (passthrough, registry, broker)MCP Proxy tab
Per-request token ledger and cost analyticsToken Ledger, Analytics
Compiled AI Pilot CRDs pushed to BouncersControl Plane compiler → Bouncer runtime

The Control Plane compiles UI inputs into nine CRD kinds (Gateway, HTTPRoute, AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, BackendTrafficPolicy, AIGatewayFilter, MCPRoute, AIGatewayUpstreamAuth, AIGatewayMetricsPolicy). Your applications stay provider-agnostic; the Bouncer remains transport-only.

Open the /pilot cockpit (~2 min)

  1. Sign in to the Control Plane as an operator with AI governance permissions.
  2. Open AI Gateway in the top navigation (route: /pilot).
  3. Select a Bouncer from the environment selector (sandbox or production).
  4. Confirm the Overview tab shows Managed status and the bouncer inventory is non-empty.

Troubleshooting: If the bouncer list is empty, verify the bouncer registered in Settings → PEPs for the selected environment and that the bouncer heartbeat is fresh. See AI Gateway troubleshooting.

Typical operator flow (~15 min)

  1. Register a Bouncer in front of your LLM or MCP upstream (Bouncer placement).
  2. In AI Gateway → Routing & models, add AIServiceBackend entries and map routes to models.
  3. Enable guardrails and DLP profiles your organization requires (Guardrails & DLP).
  4. Send test traffic through the Bouncer listener (port 8080 by default).
  5. Inspect Analytics, Traffic log, and Audit for AI_TRAFFIC_LOG, AI_POLICY_VIOLATION, and AI_PII_REDACTION events.
  6. Promote configuration from sandbox to production when acceptance criteria pass.

Advanced global baselines, credential vaults, cache/rate-limit topologies, and enterprise onboarding live under Settings → AI Gateway and the deep-dive guides linked below.

Deployment modes

ModeUse when
Reverse proxy / gatewayOne Bouncer governs many LLM, MCP, and API targets from a shared ingress
SidecarWorkload-local enforcement with smallest blast radius
App-embedded OEMParent platform drives generate → validate → deploy via API only

See Deployment modes for topology diagrams and verification checks.

Verification

curl -fsS -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $BOUNCER_API_KEY" \
  "$PAP_API_URL/health/live"

Expect 200. In the Control Plane, AI Gateway → Overview should show the protecting Bouncer with gateway mode badge and SIEM outbox health.

Troubleshooting: If routes compile but traffic bypasses policy, confirm all client traffic routes through the Bouncer listener — direct upstream paths skip governance. See /troubleshooting.

Next steps