Request Logging

Structured JSON logs with multi-sink dispatch, field and PII redaction, and reversible masking for AI origins

Overview

SBproxy emits structured JSON logs, one object per line, UTF-8. Every request-scoped line carries request_id, trace_id, span_id, tenant_id, and route, so logs correlate with metrics and traces (see Monitoring & Observability). Access lines split latency into pipeline phases (auth_ms, upstream_ttfb_ms, response_filter_ms) alongside the standard HTTP fields.

Multi-sink dispatch

Each emitted record fans out to one or more declared sinks. A sink chooses its channel, destination, wire format, and redaction profile. With no sinks declared, logs go to stdout as before.

proxy:
  observability:
    log:
      sinks:
        - name: stdout
          target: access_log        # access_log | error_log | audit_log | trace_exporter | external_log
          format: json
          output: { type: stdout }
          profile: internal          # internal keeps fingerprints and raw query; external strips them
        - name: file-archive
          target: audit_log
          format: json
          output:
            type: file
            path: /var/log/sbproxy/audit.json
            max_size_mb: 100
            max_backups: 7
            compress: true
        - name: otel-collector
          target: access_log
          format: json
          output:
            type: otlp
            endpoint: http://otel-collector:4318/v1/logs
            transport: http

Sinks can also be declared per tenant (tenants[].observability.log.sinks) and per origin (origins[].observability.log.sinks). A tenant-scoped sink only ever sees records whose resolved tenant matches, so one tenant's logs can ship to its own Loki instance and never cross into another's.

Redaction

Sensitive values are matched by field key, not by guessing at content. A built-in denylist runs first and cannot be disabled: authorization, cookie, set-cookie, *_secret, *_token, *_key, prompt, messages, ja3, ja4, and more. Each match becomes a [REDACTED:<NAME>] marker. Operators layer their own field keys and regex masks on top:

proxy:
  observability:
    log:
      redact:
        fields:
          - x-internal-token
          - internal_account_id
        patterns:
          - name: customer_uuid
            pattern: 'cust_[a-z0-9]{20}'
            replacement: '[REDACTED:CUSTOMER_UUID]'

Built-in PII detector

An optional fourth pass runs the rule-driven PII detector against the rendered JSON. It ships rules for email, US SSN, credit card (Luhn-validated), US phone, IPv4, IBAN, and common API-key shapes (OpenAI, Anthropic, AWS, GitHub, Slack). Enable all defaults or select a subset, and override per tenant or origin for stricter regulated workloads:

proxy:
  observability:
    log:
      redact:
        pii:
          enabled: true
          rules: [email, us_ssn, credit_card]
          disable: [ipv4]

Reversible masking for AI origins

A customer copilot needs the model to answer with the same name or order number the user typed, but you do not want that value written to any log. Reversible masking replaces the value with a placeholder before the prompt is forwarded, lets the model echo the placeholder, and restores the original on the response before it reaches the client. The original lives in memory for the request lifetime only and is never written to the access log, audit log, or trace span. It works on both buffered and SSE-streamed responses.

origins:
  - name: customer-copilot
    action: ai_proxy
    pii:
      enabled: true
      rules:
        - name: email
          pattern: '\b[a-z0-9._%+-]{1,64}@[a-z0-9.-]{1,255}\.[a-z]{2,63}\b'
          reversible: true
          mask_template: "<placeholder:email:%d>"

Reversible masking and the semantic cache cannot share an origin: the cache keys on prompt shape, so two requests with different captured values could collide. When both are configured on one AI origin, the semantic cache is dropped from the compiled config and a warning is logged.

← PreviousMonitoringNext →Events