Audit vs Diagnostic Logs

Control Core intentionally separates audit and diagnostic data so each can be governed correctly.

Audit Logs (Compliance Record)

Audit logs answer who did what, when, and with what outcome.

Database-backed event history for user/admin/system actions
Used for compliance controls, investigations, and evidence
Includes policy change traceability (version and commit context when available)
Includes transaction-integrity fields (request correlation, decision source, latency, actor, and AI guardrail metadata)

Implementation note: By default, Control Core stores audit records in relational storage with retention controls.
Tamper-evident evidence mode (signed export manifests and immutable/WORM targets) is an enterprise deployment option and must be explicitly configured.

Single source of truth for decision history

Every PBAC decision — bouncer enforcement (AI_TRAFFIC_LOG, AI_POLICY_VIOLATION, AI_PII_REDACTION, generic allow/deny), Control Plane evaluation (POST /v1/decisions/evaluate), policy authoring/promotion, Control Bridge sync lifecycle, approval gates, and SIEM delivery — is written to the audit_logs table. It is the single surface that the audit UI, SIEM outbox, retention job, and long-term evidence exports read from.

Earlier internal builds carried a separate decision_requests table as a half-built parallel decision store; it was never wired to a router and was removed in the multi-environment overhaul. If you are upgrading from a pre-2026.04 build, the scrum_drop_decision_requests Alembic migration removes the legacy table; nothing in your retention, SIEM, or BI pipeline reads from it.

Per-environment retention (sandbox vs production vs custom envs) is configured under Settings → Logging & Retention and applies directly to audit_logs rows scoped by the environment column.

Diagnostic Logs (Behavior Trace)

Diagnostic logs answer how the system behaved during a request/incident.

High-fidelity operational and policy-evaluation context
Used for debugging, performance tuning, and PBAC root-cause analysis
Shared as secure packages or via approved external connectors (for example, Grafana/Splunk)
Off by default. Administrators turn subsystems on per-level from Settings → Diagnostic Logs. See Diagnostic Logs for the full workflow, export format, and purge semantics.

Data Handling Expectations

Treat audit logs as long-lived compliance evidence.
Treat diagnostics as scoped troubleshooting artifacts with bounded retention.
Apply redaction/masking policies consistently before sharing with support.
For SIEM pipelines, use durable outbox delivery and monitor pending/failed/dead-letter queue states.

SHADOW_WOULD_DENY event anatomy

Shadow decisions are audit records (compliance evidence), not diagnostic traces. Annotated sample:

{
  "event_type": "SHADOW_WOULD_DENY",   // Audit event class — filter on this
  "mode": "shadow",                    // Enforcement posture of the Bouncer
  "evaluation": "would_deny",          // Counterfactual: what enforce would do
  "matched_rule": "data.controlcore.wire_cert.allow",  // Rego rule path
  "risk_level": "high",                // Gap / pack severity hint
  "details": {
    "path": "/api/v1/wires",           // Request path (no body / PII)
    "method": "POST",
    "decision_id": "dec_01J9EXAMPLE",  // Correlate with diagnostics if enabled
    "reason": "aml_cert expired",      // Non-PII reason code / short text
    "control_id": "OSFI-B13-IA-WIRE-01",
    "upstream_forwarded": true         // Proof shadow did not block
  },
  "environment": "sandbox",
  "correlation_id": "req_01J9EXAMPLE",
  "actor": { "subject_id": "user-hash-or-id" }  // Prefer stable id, never full JWT
}

Field	Audit use
`mode` + `evaluation`	Prove shadow vs enforce posture in evidence packs
`matched_rule` / `control_id`	Trace which control would fire
`risk_level`	Prioritize gap remediation
`details.upstream_forwarded`	Demonstrate non-blocking pilot to auditors
`correlation_id`	Join to diagnostic packages when troubleshooting

Related: Shadow discovery quickstart sample JSON.

Filter recipes

UI (Audit logs):

Goal	Filters
All shadow counterfactuals	`event_type` = `SHADOW_WOULD_DENY` or `SHADOW_WOULD_ALLOW`
High-risk would-denies	`event_type` = `SHADOW_WOULD_DENY` + `risk_level` in `high,critical`
Single control	`details.control_id` = `<id>` or search `matched_rule`
Environment slice	`environment` = `sandbox` / `production`
Post-enforce comparison	Same path window: `SHADOW_WOULD_DENY` (before) vs `DENY` / `AI_POLICY_VIOLATION` (after)

API:

curl -s "$PAP_API/v1/audit?event_type=SHADOW_WOULD_DENY&environment=sandbox&limit=100" \
  -H "Authorization: Bearer $TOKEN" \
  | jq '.items[] | {created_at, risk_level, matched_rule: .event_context.matched_rule, path: .event_context.details.path}'

Export tip

For auditor packages, prefer Compliance Dashboard → Export PDF/JSON (Compliance report) which already maps clauses. For raw event extract:

Apply the filter recipe above.
Export audit CSV/JSON from Audit UI (respects retention and redaction).
Never include Authorization headers, tokens, or response bodies in tickets — use correlation_id + decision_id only.
Ship durable copies to SIEM via the audit outbox; monitor pending/failed/DLQ.

Diagnostic logs remain off by default and are for root-cause only — do not submit diagnostic packages as SOC 2 evidence unless your procedure explicitly allows scoped troubleshooting artifacts.