AI Pilot — Observability Dimensions
Audience: Platform engineers, FinOps, AI governance owners Time: ~6 min read
The Observability tab in /pilot and /settings/pilot lets you slice token usage, cost, latency, and decisions by every meaningful dimension — not just global aggregates.
All available dimensions
Every endpoint under /pep-config/pilot/bouncer/{id}/observability and /analytics accepts these query parameters:
| Dimension | Query param | Source field | Examples |
|---|---|---|---|
| Time window | window | (computed) | 1h, 24h, 7d, 30d, custom |
| Bouncer | bouncer_id | AuditLog.pep_id | 42 |
| Application / Caller | app | AuditLog.actor_app | marketing-portal, agent-runner |
| User / Subject | user | AuditLog.user_subject | u_123, alice@acme.com |
| Resource type | target_type | AuditLog.target_type | llm, mcp, rag, agent, api, data |
| Provider | provider | AuditLog.ai_provider | openai, azure-openai, anthropic, bedrock, vertex |
| Model | model | AuditLog.ai_model | gpt-4o, claude-3-5-sonnet, gemini-1.5-pro |
| MCP server | mcp_server | AuditLog.mcp_server_id | internal-tools, weather-api |
| MCP tool | mcp_tool | AuditLog.mcp_method | get_record, delete_record |
| AI agent | agent_id | AuditLog.agent_id | sf-agentforce-1, foundry-2 |
| RAG service | rag_service | AuditLog.rag_service | kb-finance, kb-product |
| Decision | decision | AuditLog.outcome | allow, deny, mask, redact, 4xx, 5xx |
| Correlation id | correlation_id | AuditLog.correlation_id | for transaction-level drill-down |
Multiple dimensions stack: ?provider=openai&model=gpt-4o&user=alice@acme.com&decision=deny.
Filter bar in the UI
/settings/pilot -> Observability and /pilot -> Analytics both expose the same filter bar. Each filter is multi-select (except correlation id). The filter bar persists per-tab in the URL so you can share a filtered view by copying the URL.
Worked queries
| Question | Filter |
|---|---|
| Which user is burning the most tokens this hour? | window=1h&target_type=llm then sort by tokens_in+tokens_out per user |
| Did Anthropic on Bedrock deny anything yesterday? | window=24h&provider=bedrock&model=claude-*&decision=deny |
| What did the marketing portal spend on AI yesterday? | window=24h&app=marketing-portal |
What MCP tools did agent foundry-2 call? | agent_id=foundry-2&target_type=mcp |
| Show me the slowest p95 on GPT-4o today | window=24h&provider=openai&model=gpt-4o then look at TTFT/ITL |
| Did anyone hit an MFA wall? | decision=deny + drill-down into AI_DECISION_LINEAGE for step_up_required |
Token Ledger drill-down
The Token Ledger tab in /pilot is the same data narrowed to per-transaction granularity. Each row links to /audit?correlation_id=... for the full lineage view.
Where the data comes from
All Observability data is computed from AuditLog rows the bouncer's Bouncer intercept already emits today. The 2026-Q2 schema additions:
AuditLog.target_type,target_id,ai_provider,ai_model,mcp_server_id,mcp_method,agent_id,rag_service,actor_app,user_subject
Existing rows are backfilled lazily; old rows missing dimensions show as unknown in filters.
Performance
For high-cardinality fields (e.g. user), the API returns the top-N results plus an other bucket. Default N is 25; tune with top=N (max 200).
Three observability layers
Control Core separates observability into three non-overlapping layers. Each store answers a different question — do not expect the same payload in Audit, Langfuse, and Grafana/SIEM.
| Layer | Store | Control Plane surface | Purpose |
|---|---|---|---|
| Control Core Audit Trail | Control Plane audit store | /audit, AI Pilot Audit tab | Authoritative PBAC decisions (allow / deny / mask), Policy Bridge sync, Control Plane admin actions |
| Langfuse | Langfuse (ClickHouse) | AI Pilot Activity → AI Traces (metadata only) + deep-link | LLM traces — tokens, latency, model — without storing prompt bodies in the PAP |
| Grafana or customer SIEM | Loki / OTLP / Splunk / Datadog | External link (CC_GRAFANA_URL) or SIEM outbox | Infrastructure metrics and enterprise correlation |
Use this page and the filter bar for Audit-derived dimensions. Use Langfuse for content-level LLM observability. Use Grafana or your SIEM for platform SRE alerting. For the agent-security workflow that ties these layers together, see AI Agent Security.