Documentation
Take control of your AI traffic.
One Apache-2.0 binary that routes to 200+ models across 72 providers, serves the weights on your own GPUs, and governs both. Bring your own keys; nothing sits between you and your providers' pricing.
Getting Started
Quick Start
Get up and running in minutes
→Run your first model in 60 seconds
One curl install, one sbproxy run, on a Linux GPU box, a Mac, or plain CPU
→Core Concepts
Origins, handler chains, and actions vs policies
→Request Flow
The full pipeline stage by stage, and where every hook and extension point attaches
→First Proxy
Working reverse proxy in under 5 minutes
→First AI Gateway
OpenAI-compatible AI gateway in under 5 minutes
→Use Case Guides
Stand up your own OpenRouter
API keys scattered across teams, no accounting
→Point Claude Code at your own GPU
You want your coding assistant on hardware you control
→Connect your coding agents
Use sbproxy connect with Codex, Claude Code, Cursor, Cline, and Copilot
→Serve Qwen, GLM, or Gemma on a cloud L4
GCP credits and an afternoon
→Local first, spill to cloud
A GPU that has to pay for itself
→Air-gapped and sovereign AI
Weights and prompts that must never leave the network
→Migrate off LiteLLM in an afternoon
A LiteLLM proxy you want off of
→Guardrails on every prompt, local or hosted
Shadow Ollama under someone's desk
→Meter and monetize the AI that calls you
AI crawlers eating your content for free
→Run it in production
It works on your laptop and on-call starts Monday
→Monetize content with Pay Per Crawl
Charge AI crawlers per request with HTTP 402 challenges and a token ledger
→Verify signed AI agents
Cryptographic identity for crawlers using RFC 9421 message signatures
→Serve content for agents
Negotiate Markdown for AI crawlers, project JSON, validate against OpenAPI
→Federate internal MCP tools
One catalog for every MCP server in your business
→Replace Your Reverse Proxy
Migrate from a traditional reverse proxy to sbproxy
→Secure Any API
Add auth, rate limiting, and WAF without code changes
→Inbound Governance
MCP and Agents
The map across MCP tool-calling and A2A agent-to-agent traffic
→Pay Per Crawl
HTTP 402 challenges, single-use token ledger, x402 and MPP settlement
→Payments
Get paid before serving a request, price AI crawler traffic, and prove what was consumed
→Web Bot Auth
RFC 9421 HTTP Message Signatures with Ed25519 and HMAC
→CAP Tokens
Crawler Authorization Protocol capability tokens: path globs, rate grants, agent binding, issuer JWKS
→Trust Tiers
The four-value trust tier every request gets, what earns each, and how policies consume it
→Content for agents
HTML to Markdown, JSON projection, OpenAPI emission, Content-Signal headers
→MCP Federation
One catalog for every internal MCP server, with OAuth2, DPoP, PKCE, DCR
→MCP Tool Composition
Config-declared tools on local MCP servers: static, HTTP, or a step DAG, with response shaping
→Tool Versioning
Publish several versions of one MCP tool, resolve per consumer, adapt, and sunset
→Traffic Management
API Gateway
The traditional reverse-proxy pillar: routing, auth, rate limiting, and WAF
→Routing
Proxy, load balancer, GraphQL, gRPC, WebSocket, and more
→Path Rules
Route requests to different origin configurations
→Protocols
HTTP/2, WebSocket, gRPC, SSE
→GraphQL
Transparent by default; fail-closed depth, introspection, and batch validation when enabled
→WebSocket
Upgrade semantics, what runs before the upgrade, message size limits, subprotocol negotiation
→Storage
Serve objects from S3, GCS, Azure, or local disk, plus where the gateway persists its own state
→MCP
Model Context Protocol server for LLM tool integration
→A2A
Agent-to-Agent protocol for inter-agent communication
→Fallback Origin
Automatic failover to backup origins on error or timeout
→AI Gateway
Overview
Multi-provider LLM gateway with OpenAI-compatible API
→Providers
All /v1/* endpoints: chat, completions, embeddings, images, audio
→Model Routing
Fallback, weighted, round-robin, cost-optimized routing
→Fallback Chains
Automatic failover and health monitoring across providers
→Guardrails
Content policies, PII redaction, and streaming evaluation
→Spend Tracking
Budget enforcement and cost tracking per consumer
→Predictive Budgets
Warn, then downgrade, then block as a scope approaches its cap
→Outcome-Aware Routing
Route by realized cost-per-success fed back from completed requests
→Guardrail Mesh
Collect every verdict, fuse on a quorum, redact-and-continue, verdict cache
→AI Policy Plane
One sandboxed CEL expression over guardrails, budgets, routing, and principal
→LLM-Aware Resilience
Classify upstream failures and set per-error retry counts
→Verifiable Usage Ledger
Hash-chained, optionally signed spend receipts you can re-derive and verify
→Caching
Response caching and semantic caching
→Local Inference & Semantic Cache
On-box embeddings and prompt-injection classify, no per-call cost, no prompt egress
→Security
Cedar Authorization
Authorize MCP tool calls, request confirmation, and replay policy decisions offline
→Authentication
The chooser across all fifteen providers: what each proves, when to use it, and composition
→Dynamic Key Management
Mint, rotate, and revoke virtual keys at runtime; hashed at rest, fail-closed, instant revoke
→Rate Limiting
Sliding window, fixed window, token bucket, leaky bucket
→WAF
OWASP Core Rule Set with paranoia levels
→DDoS Protection
Adaptive rate-based detection
→IP Filtering
Allowlist/blocklist with CIDR support
→Bot Detection
Per-origin bot detection with allow/deny lists
→Security Headers
HSTS, CSP, X-Frame-Options, Referrer-Policy
→CSRF
Token-based CSRF with cookie and header validation
→OWASP API Top 10
One policy pack synthesizing the ten 2023 items, with a report_only to enforce path
→AI gateway security coverage
OWASP LLM Top 10 (2026) mapping and the gateway-layer controls no published list covers
→Observability
Scripting and Extensions
Variables and Secrets
Environment variables, config variables, secrets providers
→Secrets Management
Secure management of sensitive values with external providers
→Template Engine
Dynamic variable substitution with {{variable}} syntax
→Scripting Reference
Hook contracts, context variables, sandbox limits, error behavior, and debugging
→CEL Expressions
Safe expression language for routing and matching logic
→Rego / OPA Policies
Run an existing OPA-style Rego policy set, evaluated in-process by Regorus
→Lua Scripting
Full scripting language for complex request logic
→JavaScript Scripting
QuickJS header modifiers, raw-body and JSON transforms, and sandbox limits
→WASM Development
Build response transforms in Rust or TinyGo against the WASI contract
→Extension Bundles
Package hooks with manifests, digests, capability grants, and atomic reload
→Extending SBproxy
Choose inline scripts, dynamic bundles, or linked Rust plugins
→Reference
Config Overview
Every component of an origin config, with working examples
→Config Reference
Field-by-field schema for sb.yml
→Request Rules
Control which requests an origin handles
→Modifiers
Conditional request and response modification
→Transforms
Response transformation types
→Webhooks
Fetch external data during request processing
→Session
Cookie-based tracking and client fingerprinting
→Error Pages
Custom error responses with template variables
→Examples
Common configuration patterns
→Architecture
How requests flow through the gateway
→CLI Reference
All CLI commands and flags
→OpenAPI Emission
Publish a live OpenAPI document derived from your config
→Glossary
Plain-English mapping of codes, acronyms, and SBproxy vocabulary
→Deployment
Overview
Deployment topologies and recommendations
→Mesh Clustering
Keep keys, budgets, and rate limits coherent across a replica fleet without external Redis
→Kubernetes
Helm charts, ConfigMaps, and HPA configuration
→Docker
Container deployment and compose files
→Bare Metal
Systemd service and binary deployment
→