Control Testing with the Control Impact Simulator — Control Manager Guide

Introduction

The Control Impact Simulator (CIS) is your primary tool for testing control logic before deployment. It evaluates one control against one input payload per run, using the same OPA engine the Bouncer uses at runtime. You see the decision, the reason, and the full OPA trace without needing to read Rego or hand-craft JSON for every run.

This guide walks you through every aspect of testing controls effectively.

Scope note: The simulator is a per-input decision tool. It is not a blast-radius, traffic-replay, or fleet-impact tool. Use it alongside shadow mode and staged rollout for production promotion.

Getting Started - Your First Test

Step 1: Navigate to the Control Simulator

1. Open the Control Core PAP interface
2. Click Control Simulator in the Operations section of the sidebar
3. You'll see a three-panel interface:
   • Left: Control selection and environment context
   • Center: Automated test input builder
   • Right: Results and trace visualization

Step 2: Select a Control to Test

1. In the left panel, you'll see controls filtered by your current environment

   • Sandbox environment: Shows Draft and Sandbox controls you can modify
   • Production environment: Shows Production controls (read-only testing)

2. Select a control from the dropdown

   • Large lists (10+ controls): Search box automatically appears
   • Small lists: Standard dropdown for quick selection
   • Search by name, description, resource, or status

3. The panel automatically displays:

   • Resource: The API or application being protected
   • Bouncer: The PEP enforcing this control
   • Target Actions: HTTP methods (GET, POST, DELETE, etc.)

Tip: This automatic context helps you understand what the control protects without digging through configuration.

Step 3: Configure Test Inputs (The Smart Part!)

The center panel is where Control Core's intelligence shines.

What happens automatically:

1. Control Core parses your control's Rego code
2. Extracts all input.* attribute references (e.g., input.user.role, input.resource.owner)
3. Generates a custom form with user-friendly labels
4. Infers data types (string, number, boolean, array)
5. Pre-fills sensible default values

What you do:

1. Simply fill in the form fields with your test values
2. Each field shows its data type for guidance
3. No need to understand JSON structure or Rego syntax

Two ways to configure inputs:

Visual Form (default):

• Auto-generated form fields for each input attribute
• User-friendly labels and data type hints
• Scrollable list for policies with many attributes

JSON View:

• Click the JSON View tab to see the structured JSON
• Editable: You can edit JSON directly and changes sync to the form
• Useful for complex nested structures or bulk editing
• Real-time validation with error messages for invalid JSON

Example:

If your control references:

input.user.role == "admin"
input.resource.sensitivity == "high"
input.context.authenticated == true

You'll see:

• User Role (string): text input
• Resource Sensitivity (string): text input
• Context Authenticated (boolean): toggle switch

Step 4: Run the Test

1. Choose Test Type:

   • Decision: Fast evaluation for ALLOW/DENY result
   • Full Response: Complete evaluation with all data

2. Click Run Controls Impact Test

3. Within seconds, see the result in the right panel

Step 5: Interpret Results

The results panel shows:

Result Card:

• Large, color-coded badge: ALLOW (green), DENY (red), MASK (blue), or NO_MATCH (gray)
• Human-readable reason explaining the decision
• Execution time (typically < 100ms)

Control Trace Timeline:

• Step-by-step visualization of how the decision was made
• Shows which conditions passed or failed
• Expandable details for each step

Raw Data Tabs:

• Input JSON: The exact data sent to OPA
• Output JSON: The complete OPA response
• Debug View: (Enterprise feature) Line-by-line code trace

Advanced Features

Smart Control Selection

For organizations with many controls (10+), the Control Simulator automatically provides a searchable dropdown:

Features:

• Automatic search mode: Activates when you have 10+ controls
• Multi-field search: Search by control name, description, resource name, or status
• Real-time filtering: Results update as you type
• Result counter: Shows "X of Y controls" matching your search
• Scrollable results: Handles hundreds of controls efficiently

Usage:

1. Click the control dropdown
2. Start typing to search (e.g., "api", "admin", "production")
3. Select from filtered results

JSON Editing Mode

The JSON View tab provides direct JSON editing with real-time synchronization:

Features:

• Editable textarea: Edit JSON directly instead of using form fields
• Bidirectional sync: Changes in JSON update the form, and vice versa
• Real-time validation: Invalid JSON shows error messages
• Smart error handling: Only shows errors for clearly invalid JSON (not while typing)
• Full nested structure: Edit complex nested objects easily

When to use:

• Bulk editing multiple attributes
• Complex nested structures
• Copy-pasting test data from other sources
• Advanced users who prefer JSON

Example:

{
  "user": {
    "id": "user-001",
    "roles": ["demo_user"],
    "tenant_id": "tenant-1"
  },
  "resource": {
    "type": "api",
    "tenant_id": "tenant-1"
  },
  "action": {
    "name": "GET"
  }
}

Saving Test Scenarios

Why save scenarios?

• Regression Testing: Quickly retest after control changes
• Documentation: Preserve test cases as control documentation
• Consistency: Share test configurations with your team

How to save:

1. Configure your test inputs
2. Click Save as Scenario in the center panel
3. Give it a descriptive name (e.g., "Admin Access Happy Path")
4. Add an optional description

How to use:

1. Select a control
2. Choose a saved scenario from the Load Saved Scenario dropdown
3. All inputs are auto-filled—just click Run Test

Exporting Results

Export simulation results for:

• Documentation in change requests
• Compliance evidence
• Stakeholder communication
• Training materials

Export formats:

• JSON: Complete data with full trace (best for re-import or technical review)
• CSV: Tabular format (good for spreadsheets and reporting)
• PDF: Formatted report (best for sharing with non-technical stakeholders)

How to export:

1. Run a test and get results
2. Click Export in the results panel
3. Choose your format
4. File downloads automatically

Quick Test Templates

For common testing scenarios, use pre-configured templates:

Available Templates:

• Admin Full Access: Test with admin user having full permissions
• Regular User Access: Test with regular user having limited permissions
• Unauthenticated Request: Test public endpoint access
• Cross-Tenant Access: Test access attempt across different tenants
• Rate Limit Test: Test with high request rate to trigger rate limiting
• Blocked Route Test: Test access to blocked/internal routes

How to use:

1. Select a control
2. Scroll down in the left panel to see Quick Test Templates
3. Click a template to load its attributes
4. Modify if needed, then run the test

Test History

Every simulation is saved to test history for:

• Reviewing past tests
• Tracking policy changes over time
• Rerunning previous tests
• Audit compliance

Using test history:

1. Select a control
2. Expand the Test History panel at the bottom
3. See all past simulations with:
   • Timestamp
   • User who ran it
   • Decision result
   • Execution time
   • Environment

History actions:

• View Details (eye icon): See full trace and inputs
• Export (download icon): Export that specific test
• Rerun (play icon): Load inputs and run again

Environment-Specific Testing

Testing in Sandbox

When you're in Sandbox environment:

• Test Draft and Sandbox controls
• Make changes and immediately retest
• Save scenarios for later use
• Full read/write access to controls

Testing in Production

When you switch to Production environment:

• Test Production controls (read-only)
• Simulations are logged for audit
• Cannot modify control code
• Use to verify behavior matches expectations

Production warning banner appears to remind you testing in production is logged.

Interpreting the Control Trace

The trace timeline is your window into control decision-making.

Trace Structure

Step 1: Control Matched

• Confirms the control was selected for evaluation
• Shows control name and environment

Step 2-N: Condition Evaluation

• Each condition in your control is evaluated
• Shows TRUE or FALSE for each condition
• Hover or expand for details

Final Step: Final Decision

• Shows how conditions combined to produce the decision
• Explains the final ALLOW or DENY

Reading Conditions

Each condition step shows:

• Description: What was being checked
• Result: Pass or fail
• Details: Expandable raw trace data

Example trace:

Step 1: Control Matched ✓
  Control 'Admin Access Control' selected for evaluation

Step 2: Condition Evaluation ✓
  user.role == "admin" → TRUE

Step 3: Condition Evaluation ✗
  context.time < "17:00:00" → FALSE

Step 4: Final Decision ✗
  Access DENIED - One or more conditions not met

From this trace, you can see:

• The user has the admin role
• But the request was after 5:00 PM
• Therefore, access was denied

Best Practices

1. Test Before You Deploy

Always test controls in Sandbox before promoting to Production.

• Create control in Draft
• Test with multiple scenarios
• Save successful test cases
• Enable in Sandbox
• Test again with production-like data
• Promote to Production

2. Build a Test Scenario Library

For each critical control, save 3-5 scenarios:

• Happy Path: Expected success case
• Deny Case: Expected denial case
• Edge Cases: Boundary conditions
• Error Cases: Invalid inputs

This becomes your regression test suite.

3. Use Meaningful Scenario Names

Good names:

• "Admin Access - Business Hours"
• "Guest User - Public Resource - ALLOW"
• "Manager Approval - High Value Transaction"

Bad names:

• "Test 1"
• "My test"
• "Scenario"

4. Review Test History Periodically

Check test history to:

• Verify controls are being tested regularly
• Identify problematic controls (many failed tests)
• Track decision patterns over time

5. Export for Documentation

When submitting control changes:

• Export successful test results
• Include with change request
• Demonstrates due diligence
• Provides evidence for compliance

Understanding Test Results

ALLOW Decision

Meaning: Access would be granted by this control

What to check:

• All conditions passed in the trace
• Result aligns with control intent
• No unintended side effects

DENY Decision

Meaning: Access would be denied by this control

What to check:

• Which condition(s) failed in the trace
• Is the denial intentional?
• Review failed conditions for logic errors

MASK Decision

Meaning: Access allowed but data will be masked

What to check:

• Masking rules are correctly applied
• Sensitive fields are protected
• User still gets necessary data

NO_MATCH Decision

Meaning: Control didn't match, default action applied

What to check:

• Control matching conditions
• Resource default security posture
• May indicate control needs broader scope

Troubleshooting

"No attributes extracted"

Possible causes:

• Control has no input.* references
• Logic has syntax errors
• Control is empty or placeholder

Solution:

• Review control code
• Add input references to conditions
• Validate syntax

"Simulation failed"

Possible causes:

• Invalid input data format
• OPA evaluation error
• Control logic error

Solution:

• Check error message in results
• Review trace for clues
• Simplify test inputs
• Contact admin if OPA service is down

"Export failed"

Possible causes:

• No simulation ID (test wasn't logged)
• Network interruption
• Server error

Solution:

• Rerun the test
• Try different export format
• Check network connection

FAQ

Q: Are test simulations using real OPA or mocked results?

A: All simulations use the actual OPA engine bundled with Control Core. Results are identical to production evaluation.

Q: Can I share my saved scenarios with teammates?

A: Currently, scenarios are per-user. Share by exporting the test result and sending the JSON file.

Q: How long is test history retained?

A: Test history is retained indefinitely by default. Your admin can configure retention policies in settings.

Q: Can I test multiple controls at once?

A: No, the Control Simulator tests one control at a time for focused, detailed results.

Q: What's the difference between Decision and Full Response test types?

A: Decision only evaluates the allow/deny outcome (faster). Full Response evaluates all control rules including masking, logging, and response modifications (more complete).

Next Steps

• Explore API Reference for programmatic testing
• Review Admin Guide for team management
• See Quick Start: Testing Controls for a 5-minute intro
• Review Simulator & Testing Guide for workflow reference