AI Guardrails

What Guardrails Do

Guardrails evaluate requests and responses against content policies before they reach your application or the end user. They run at two points in the handler chain:

• Pre-request: Evaluate user prompts before they are sent to the LLM provider. Block prompt injection, enforce token budgets, and redact sensitive data from outbound requests.
• Post-response: Evaluate LLM responses before they reach the client. Flag unsafe content, redact PII in completions, and enforce output policies.

CEL Selectors

Target guardrail rules to specific consumers, models, or routes using CEL expressions:

ai:
  guardrails:
    rules:
      - name: block-profanity-for-public-api
        selector: "auth.consumer == 'public-app'"
        action: block

Selectors have access to the full request context: auth.consumer, request.model, request.path, session, and all other CEL variables. A rule without a selector applies to every request.

Content Policies

Each rule specifies an action when a violation is detected:

• block: Reject the request or response with a 403 status and an error message.
• flag: Allow the request to proceed but add a flag header and emit a webhook event for review.
• redact: Replace matched content with placeholder text and allow the request to continue.

PII Redaction

Built-in detectors identify and redact common PII patterns:

• Credit card numbers (Visa, Mastercard, Amex, Discover)
• Social Security numbers (XXX-XX-XXXX format)
• Email addresses
• Phone numbers (US and international formats)

ai:
  guardrails:
    rules:
      - name: redact-pii
        type: pii
        detectors: [credit_card, ssn, email, phone]
        action: redact
        replacement: "[REDACTED]"
        phase: both

Streaming Behavior

Guardrails evaluate streaming responses using a rolling window approach. As SSE chunks arrive, the evaluator buffers content and checks policies against the accumulated text.

If a violation is detected mid-stream, SBproxy closes the connection with an error frame. The client receives all content up to the violation point, then an SSE error event with the guardrail rule name and action. Your application should handle this by checking for error frames in the stream.

Config Reference

ai:
  guardrails:
    enabled: true
    fail_open: false
    rules:
      - name: rule-name
        type: content | pii | token_budget
        selector: "CEL expression"
        action: block | flag | redact
        phase: pre | post | both
        patterns: ["regex patterns"]
        detectors: [credit_card, ssn, email, phone]
        replacement: "[REDACTED]"
        max_tokens: 4096
        message: "Custom error message"

Set fail_open: false (the default) to reject requests when the guardrail engine encounters an error. Set fail_open: true to allow requests through if evaluation fails.

Examples

Block Profanity

ai:
  guardrails:
    rules:
      - name: block-profanity
        type: content
        action: block
        phase: both
        patterns:
          - "\\b(badword1|badword2|badword3)\\b"
        message: "Content policy violation: inappropriate language detected"

Redact Credit Card Numbers

ai:
  guardrails:
    rules:
      - name: redact-credit-cards
        type: pii
        detectors: [credit_card]
        action: redact
        replacement: "[CARD REDACTED]"
        phase: both

Reject Requests Over a Token Budget

ai:
  guardrails:
    rules:
      - name: enforce-token-budget
        type: token_budget
        action: block
        phase: pre
        max_tokens: 8192
        message: "Request exceeds the maximum token budget of 8192"