Control CoreDocs

Token cost circuit-breaking

Autonomous agents can exhaust token budgets in minutes. Control Core combines AI Pilot BackendTrafficPolicy, Redis-backed token accounting, and Rego obligations to circuit-break spend before invoices spike.

Architecture

Click to enlarge

LayerResponsibility
BackendTrafficPolicy (AI Pilot CRD)Rate limits, model routing, fallback clusters
RedisRolling token counters per subject, tenant, model
Rego controlsDeny when input.agent_context.tokens_remaining <= 0

How to use

  1. Enable Redis in Bouncer stack — REDIS_URL must match Control Plane or dedicated FinOps instance (~5 min).
REDIS_URL=redis://:password@redis:6379/1
AI_GATEWAY_ENABLED=true

  1. Apply BackendTrafficPolicy from Control Plane AI Pilot UI or Helm values — set per-model token rate and burst (~10 min). See Cache & rate limits.

  2. Deploy token budget control — template AI Cost Guardrail or Rego:

deny if {
  input.agent_context.estimated_tokens > data.budgets.max_per_request
}

deny if {
  input.agent_context.daily_tokens_used > data.budgets.daily_cap
}
  1. Configure circuit breaker fallback — route to cheaper model or return 429 with audit event AI_TOKEN_BUDGET_EXCEEDED.

How to review / audit

  • Grafana cc-03-ai-cost-controls — token rate, circuit-break trips, model fallback count
  • Audit Logs — filter AI_TOKEN_BUDGET_EXCEEDED, AI_TRAFFIC_LOG
  • FinOps weekly: compare Redis counters to provider billing API

Review checklist:

  • Daily caps set per environment (sandbox lower than production)
  • Break-glass service accounts excluded via explicit Rego exception
  • Fallback model documented in runbook

Troubleshooting

Troubleshooting: Token counts always zero?

  • Verify Redis connectivity from Bouncer: docker exec bouncer redis-cli -u $REDIS_URL PING
  • Confirm Bouncer intercept emits token metadata — check AI Pilot troubleshooting

Troubleshooting: Circuit breaker never trips despite high usage?

  • BackendTrafficPolicy may be mis-scoped — kubectl describe backendtrafficpolicy
  • Rego budget data stale — force Policy Bridge sync: Controls → Sync to environment

Troubleshooting: Legitimate traffic blocked?

  • Raise daily_cap in PIP budget data or promote exception allowlist control
  • Use simulator with recorded agent_context JSON

Next steps: Cost optimization · Resilience & circuit breaker · Grafana dashboards