🛠️ AI Pilot Troubleshooting (AI Gateway)

This guide covers common issues when configuring AI Pilot for per-bouncer LLM controls.

📌 Quick triage

# Control Plane API
curl -sf http://localhost:8082/api/v1/health

# Bouncer health
curl -sf http://localhost:8080/health

# Plugin admin (live config source on bouncer)
curl -sf http://localhost:9998/pilot/config

🛠️ Common issues

Symptom	Likely cause	Resolution
AI Pilot page loads but bouncer data is empty	Bouncer not discovered/linked	Verify bouncer registration and `/peps` entries
Save override succeeds but runtime behavior unchanged	Config not pushed/applied on target bouncer	Validate `/pep-config/pilot/bouncer/{id}/ai-management` and bouncer `/pilot/config`
Prompt controls over-block legitimate traffic	Rules too broad or missing allowlist	Add allowlist terms; move some controls from `block` to `annotate`
Exfiltration control blocks approved traffic	Trusted domains not configured	Add exact trusted domains in content safety
Token limits appear inconsistent across replicas	Missing shared Redis	Configure `REDIS_URL` for shared counters
PII redaction not visible in tests	Action set to `annotate`/`block` only	Switch relevant control to `redact` and retest
Smart Control Core says draft generation failed	LLM provider key/model rejected at runtime	Check Smart CC settings provider, bouncer/provider credentials, and provider error details in API logs
Smart Control Core action buttons become unresponsive after route changes	Stale chat state in long-lived session	Re-open agent from current page; if stale session persists, refresh once and retry
Code assist in Policy Builder is not auto-available	Not in Code Editor mode, or builder context not active yet	Open Controls → Policy Builder → Code Editor tab, then reopen Smart CC
Code assist ignores PIP signals	PIP mapping exists but lacks relevant attributes	Map principal/resource/context attributes in Data Sources and retry with explicit attribute hints

📌 Smart Control Core Agent quick checks

# 1) API health
curl -sf http://localhost:8000/health

# 2) Login and get token (replace creds)
TOKEN=$(curl -s http://localhost:8000/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"<user>","password":"<password>"}' | jq -r '.access_token')

# 3) Verify Smart CC settings
curl -s http://localhost:8000/v1/smart-cc/settings \
  -H "Authorization: Bearer $TOKEN" | jq .

# 4) Run one Smart CC turn
curl -s http://localhost:8000/v1/smart-cc/turn \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"create draft for payroll MFA","context":{},"confirm_action":false}' | jq .

# 5) Run one code-assist turn with builder context
curl -s http://localhost:8000/v1/smart-cc/turn \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"add principal and action checks using PIP risk score","context":{"active_page":"policy_lifecycle","editor_mode":"code","resource_id":"1","bouncer_id":"1","current_policy":{"name":"Draft","rego_excerpt":"package controlcore\n\ndefault allow = false"}},"confirm_action":false}' | jq .

Check for:

intent and response_cards present.
requires_confirmation=true for mutating draft flows.
No provider/auth errors in evidence[] for normal draft generation.
Code assist turns return intent=assist_policy_code with snippet/rationale evidence.

🔧 API-level validation

# Effective AI management for a bouncer
curl -H "Authorization: Bearer <token>" \
  http://localhost:8082/pep-config/pilot/bouncer/<pep_id>/ai-management

# Observability payload for charts
curl -H "Authorization: Bearer <token>" \
  "http://localhost:8082/pep-config/pilot/bouncer/<pep_id>/observability?window=24h"

# Recent payload stream
curl -H "Authorization: Bearer <token>" \
  "http://localhost:8082/pep-config/pilot/bouncer/<pep_id>/recent-payloads?limit=8"

Check that:

use_global matches your UI state
local_override exists when expected
effective_config includes all prompt security fields

📌 Runtime compile checks

AI Pilot compiles per-bouncer controls into bouncer runtime config.

Expected keys in effective pilot payload:

ai_management
compiled_targets.ext_proc.content_safety
compiled_targets.xds.cost_optimization

curl -sf http://localhost:9998/pilot/config | jq .

📌 Prompt control tuning strategy

Start with strict controls in sandbox.
Use annotate mode for uncertain detections.
Promote high-confidence patterns to block.
Add explicit allowlist/trusted-domain exceptions.
Re-check audit events and false-positive rates weekly.