Is SBproxy a reverse proxy, an AI gateway, or a model server?

All three, from one binary. It terminates TLS and handles HTTP/1.1, HTTP/2, WebSocket, and gRPC like nginx or Envoy. It routes LLM traffic to 200+ hosted models across 72 providers with guardrails, budgets, and MCP federation. And it serves open-weight models on your own GPUs by supervising vLLM or llama.cpp underneath. One config file drives all of it, and the AI features sit on a runtime that was already a production-grade proxy.

SBproxy is the open-source enterprise gateway for API, MCP, and AI model traffic. Outbound, it routes to 200+ hosted models across 72 providers through one OpenAI-compatible endpoint, picking models by prompt difficulty and rerouting on live cost-per-success, and serves open-weight models on your own GPUs. Inbound, it gates the crawlers, signed agents, and MCP clients that hit your APIs, with controls like Pay Per Crawl and Web Bot Auth. Both directions share one policy of virtual keys, budgets, and guardrails. Underneath sits a production-grade reverse proxy running auth, TLS, WAF, rate limiting, transforms, and caching. Single static binary, Apache License 2.0.

Can SBproxy serve models on my own GPUs?

Yes, on any GPU. A provider whose entire body is a serve: block boots to a first token: SBproxy resolves the weights, fits an engine and quantization to your card (bigger cards take FP8, smaller ones fall back to an int4 GGUF), runs vLLM or llama.cpp as a supervised subprocess, and routes to it. Run sbproxy doctor first and it reports what the host can serve. Every governance plane, meaning virtual keys, budgets, guardrails, and the usage ledger, applies to the local model unchanged, and a hosted provider placed after the local lane catches overflow under policy.

How is SBproxy different from vLLM or Ollama?

vLLM and Ollama serve a model. SBproxy stands up the environment around it: the governed OpenAI-compatible endpoint, virtual keys, budgets, guardrails on prompt and response, failover to hosted providers, and a verifiable usage ledger. It runs vLLM or llama.cpp underneath as supervised engines, so you keep the engine and gain the gateway.

How is SBproxy different from Cloudflare Pay Per Crawl?

Cloudflare runs at its own edge. SBproxy runs in your VPC, region, Kubernetes cluster, or air-gapped environment. Pricing is tier-aware, per route, per shape, and per agent rather than domain-wide. Inbound monetization, agent identity, content negotiation, and outbound AI governance live on one policy graph instead of separate Cloudflare products. Settlement supports both x402 and Stripe MPP, and you can act as your own Merchant of Record.

How is SBproxy different from other AI gateways like LiteLLM, Helicone, Portkey, or Kong AI Gateway?

Most AI gateways stop at routing calls to hosted providers. SBproxy also serves the weights: the same binary runs open models on your GPUs and applies the same keys, budgets, and guardrails to local and hosted traffic alike. It is also a real reverse proxy underneath, so auth, WAF, and rate limiting ride along instead of needing a second gateway. Helicone is excellent for LLM observability. LiteLLM is a Python library that grew a server, and sbproxy config import-litellm converts its config in an afternoon. Kong is a traditional API gateway with AI bolt-ons. None of them serve the model.

Where does SBproxy sit in my stack?

At the network hop between your apps or agents and the models they call, and between inbound AI traffic and your APIs. SDK gateways live inside one app's code, hosted edges govern from someone else's network, and agent platforms ask you to build on them; SBproxy is standalone infrastructure that deploys like nginx, works under any agent framework or platform, and fronts any hosted provider or your own GPUs. It terminates TLS with certificates it issues and renews from Let's Encrypt, so the governed endpoint can face the public internet behind virtual keys you mint and revoke at runtime.

How does SBproxy handle API keys and state across a fleet of instances?

SBproxy ships a dynamic key plane. Mint, rotate, and revoke virtual keys at runtime through the admin API with no reload, and a revoke takes effect on the next request. Keys are hashed at rest with HMAC-SHA256 and a server pepper, and upstream provider credentials are sealed with AES-256-GCM. Across a fleet of replicas, a built-in mesh keeps the keys, budgets, and per-key rate counters coherent using SWIM gossip and CRDT counters, so the cluster coordinates itself without an external Redis. LiteLLM and most AI gateways need a separate Redis to share that state across instances.

Where does SBproxy deploy?

Single binary on a server, Kubernetes operator on a fleet, or air-gapped install on a regulated network. Same policy graph in every shape. No phone-home. No SaaS edge. Bring your own Stripe, your own SSO, your own observability, and your own audit sink. Suitable for EU, India, Saudi Arabia, US public sector, defense, healthcare, and finance workloads with sovereignty requirements.

Is SBproxy open source?

SBproxy is open source under the Apache License 2.0. Free for any use, including production and commercial, with no field-of-use restriction. Read every line on GitHub. Verify every binary.

What is Pay Per Crawl and how does SBproxy support it?

Pay Per Crawl is the pattern of charging AI crawlers for access to content using HTTP 402 Payment Required. SBproxy ships the ai_crawl_control policy: AI crawlers that arrive without a valid payment token receive a 402 response with a JSON challenge. The crawler retries with the issued token and the proxy validates it through a pluggable ledger. Settlement supports x402 stablecoin and Stripe MPP. Tokens can be priced per route, per shape, and per agent.

What is Web Bot Auth and how does SBproxy verify AI agents?

Web Bot Auth is the IETF pattern for verifying signed AI crawlers and agents. SBproxy implements it through the bot_auth provider. Agents sign each request with an Ed25519 key under RFC 9421 HTTP Message Signatures and advertise their key id in the Signature-Input header. The gateway looks up the matching public key in its directory and verifies the signature. Agents that pass come through. Everything else gets 401.

Does SBproxy support MCP?

Yes. SBproxy ships an MCP gateway that federates internal tools under one identity model. Authentication supports OAuth2 with DPoP, PKCE, and DCR. Federated servers can be wrapped from any OpenAPI service, with per-server prefix and RBAC. The same policy graph that governs API and AI traffic also governs MCP tool invocations.

What does SBproxy replace?

A hosted model router (OpenRouter), an LLM proxy (LiteLLM, Portkey, Helicone), the hand-rolled shim in front of vLLM or Ollama, a traditional API gateway (Kong, nginx, Envoy), a guardrails service, a bot management product, and a payment rail for crawlers (TollBit, Skyfire). One runtime instead of six vendors stitched together, one policy plane instead of six control planes, one audit log instead of six disconnected feeds.

How many AI providers does SBproxy support?

SBproxy reaches 200+ models behind one OpenAI-compatible API through 72 native providers, including OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Azure AI Foundry, Groq, Mistral, Cohere, Together AI, Fireworks, xAI Grok, OpenRouter, Cerebras, NVIDIA NIM, Databricks, Snowflake Cortex, IBM watsonx, and Oracle GenAI, plus the models it serves itself on your GPUs via vLLM or llama.cpp and external self-hosted runtimes like Ollama and TGI. Routing supports fallback chains, weighted, round-robin, latency-based, cost-optimized, and outcome-aware strategies.

Is SBproxy safe to run in front of production traffic?

SBproxy ships as a single static Rust binary. Releases are cut from tagged commits in a public repository on GitHub. Dependencies are pinned and checksummed in Cargo.lock. The runtime does not fetch code from a public package registry when it starts. The deployment shape matches nginx, HAProxy, and Envoy. Hot reload is supported without dropping connections.

Glossary

Plain-English mapping of the codes, acronyms, and SBproxy vocabulary that appear in commits, ADRs, and runbooks.

Overview

A plain-English mapping of the codes and acronyms that appear in SBproxy commits, ADRs, the CHANGELOG, and the operator runbook. If you have ever seen a commit titled feat(modules): G3.4 multi-rail challenge emission and wondered which knob that turns, this is the page.

The terminology is split into four groups:

Wave codes (the seven roadmap waves that built v1.0)
Work-stream prefixes (G, A, R, B, S, Q, T, E) used inside each wave
Q-codes used for QA gates and quarterly markers
General SBproxy vocabulary (OLP, CAP, MCP, x402, MPP, DPoP, RFC 9421, RSL, and more)

Wave codes

Each wave delivered one cross-cutting capability. They are sequential and additive: Wave 2 builds on Wave 1, Wave 3 on Wave 2, and so on. Every commit subject in the repository carries the wave it belongs to so the audit trail stays legible.

Code	Theme	What it shipped
Wave 1	AI governance substrate	Tier-aware Pay Per Crawl pricing, HTTP ledger redemption, typed agent class on every request, dynamic Web Bot Auth directory, OpenTelemetry tracing, structured access and audit logs, outbound webhook framework, `/healthz` and `/readyz` endpoints, Grafana dashboards, Prometheus alerts.
Wave 2	Billing foundation	First production payment rail, wallet ledger, agent registry intake, signed-batch audit log v0, per-tenant rate-limit middleware with RFC 9239 headers, agent-aware tier resolution, hot-path vs async-path layering rule.
Wave 3	Multi-rail payments	x402 plus MPP negotiation in a single 402 challenge, per-shape pricing, signed quote-token JWS, end-to-end idempotency keys, EVM reorg-aware two-phase settlement, clock-skew monitor, JWKS endpoint at `/.well-known/sbproxy/quote-keys.json`.
Wave 4	Content for agents	Two-pass `Accept` content negotiation, per-shape pricing matrix, JSON envelope schema, citation-block and boilerplate-stripping transforms, well-known projection routes for `robots.txt`, `llms.txt`, RSL `licenses.xml`, TDMRep `tdmrep.json`, Content-Signal response header, `aipref` preference signal, `x-markdown-tokens` header.
Wave 5	Identity differentiator	RFC 8693 outbound credential resolver, vault-backed PATs, stored OAuth grants under one delegation-aware interface, JA3 / JA4 / JA4H TLS fingerprinting, headless-browser detection, Skyfire KYA token verification, ML agent classifier hook surface, header-mutating CEL transform, SIGHUP re-bootstrap.
Wave 6	Operational substrate	Hash-chained Merkle audit log v1, access log v1, CAP token verifier, revenue rollups, compliance scaffold, evidence-verify and audit-migrate CLIs, Open Licensing Protocol (OLP) issuer wiring, Crawler Authorization Protocol (CAP) issuer.
Wave 7	Bonus scope	Employee-bound agent identity (SCIM webhook receiver and chargeback), Agent-to-Agent (A2A) protocol envelope with depth cap and cycle detection, license-leak guardrail, Lightning rail (CLN production adapter, LND, spot-rate oracle).

Wave 6 closes the public commitment that all six waves shipped to main. Wave 7 is bonus scope.

Work-stream prefixes

Inside each wave, individual deliverables are tagged with a single-letter stream prefix and a sub-number. The streams describe who owns the work, not what it does.

Prefix	Stream	What it covers
G	Governance / feature	Customer-visible feature behaviour. New policy modules, tier resolvers, pricing matrices, request-context fields.
A	Architect / ADR	Architecture decision records. Each `A<wave>.<n>` corresponds to an `adr-*.md` file. ADRs merge before their G work.
R	Reliability / runtime	Cross-cutting runtime behaviour. Rate limits, idempotency, clock skew, observability wiring, OTel exporters.
B	Builder / packaging	Build, packaging, CI, deploy. Cargo features, Helm values, Grafana dashboards, Prometheus alerts, Dockerfiles.
S	Scribe / documentation	Operator-facing documentation. CHANGELOG entries, runbook playbooks, threat-model entries.
Q	Quality / QA gates	End-to-end and conformance tests. Fuzz harnesses. Bench fixtures. Each `Q<wave>.<n>` is a single test fixture.
T	Telemetry envelope	Wave 8 P0 work on the request envelope. Custom properties, sessions, users, RequestEvent emission.

A label such as G3.4 reads as Wave 3, governance / feature stream item 4, and corresponds to a single PR or merge.

Q codes

Q codes have two meanings depending on context:

Inside a wave (e.g. Q1.1, Q3.6): a quality / QA gate. Each Q item is one e2e or conformance test, a fuzz harness, a synthetic-monitor fixture, or a bench scenario.
In product-strategy notes (e.g. "Q15", "the Q5 narrative"): a question number on the YC application. These do not correspond to wave work and are unrelated to the engineering Q codes.

General vocabulary

Term	Stands for	What it means in SBproxy
OLP	Open Licensing Protocol	A four-step flow for publishers to advertise a licence catalogue, agents to discover it, and the gateway to issue licence tokens bound to a licence row. Verifier ships in OSS via the AI crawl control policy.
CAP	Crawler Authorization Protocol	A JWT-based capability-token format presented in `CAP-Token:` or `Authorization: CAP <jwt>`. The OSS verifier checks signature, claims, audience, glob-allowed paths, and (optionally) per-token rate limits.
MCP	Model Context Protocol	The Anthropic-originated tool-and-resource catalogue protocol. SBproxy ships an MCP federation action that aggregates tool catalogues across upstream MCP servers and routes `tools/call` per tool.
x402	x402 protocol	A stablecoin-on-chain payment rail riding HTTP 402. SBproxy emits x402 challenge entries in multi-rail 402 responses and verifies redemption tokens via the x402 facilitator. v2 is supported.
MPP	Merchant Payment Protocol	The card-and-stablecoin-on-Stripe payment rail. SBproxy emits MPP challenge entries that carry a Stripe `payment_intent` id; redemption confirms against Stripe.
DPoP	Demonstration of Proof-of-Possession (RFC 9449)	A JWS that proves the presenter holds the private key bound to an access token. Used on outbound credential resolution so a stolen access token alone is insufficient to call an upstream.
RFC 8693	OAuth 2.0 Token Exchange	The token-exchange grant that powers SBproxy's outbound credential resolver. Wave 5 uses RFC 8693 to swap an inbound identity for an upstream access token.
RFC 9421	HTTP Message Signatures	The IETF spec for signing HTTP messages. SBproxy implements per-origin message-signature configuration and the Web Bot Auth directory.
RFC 9239	RateLimit headers	The IETF spec for `RateLimit-Limit`, `RateLimit-Remaining`, `RateLimit-Reset`, and `Retry-After`. SBproxy emits these on every throttled response.
RSL	Really Simple Licensing	A licence-advertisement standard served as `licenses.xml`. Wave 4 added a projection route at `/licenses.xml`.
TDMRep	Text and Data Mining Reservation Protocol	A licence-reservation standard served as `tdmrep.json`. Wave 4 added a projection route at `/.well-known/tdmrep.json`.
aipref	AI Preferences Working Group draft	A request-side preference signal an agent can carry to declare training, search, or input intent. Parsed at request entry and exposed to CEL, Rego, Lua, JavaScript, and WASM.
Content-Signal	IAB Tech Lab Content-Signal header	A response header carrying one of `ai-train`, `search`, `ai-input`. Stamped on 200 responses per origin and reflected into licensing projections.
Pay Per Crawl	Cloudflare-coined term	Charging an AI crawler with HTTP 402 plus a `Crawler-Payment` token. Implemented by the `ai_crawl_control` policy.
Web Bot Auth	IETF draft	The signed-bot-traffic standard. SBproxy fetches `/.well-known/http-message-signatures-directory` from a vendor, caches the JWKS, and verifies signatures on inbound bot requests.
KYA	Know-Your-Agent (Skyfire)	A token format for verified agent identity. Wave 5 added a KYA verifier and exposed `request.kya` to scripting.
JA3 / JA4 / JA4H	TLS fingerprinting algorithms	ClientHello fingerprints captured at the TLS layer and stamped onto the request context for headless-detection signals.
Apache 2.0	Apache License, Version 2.0	The open source licence under which SBproxy is published. Free for any use, including production and commercial, with no field-of-use restriction.
Pingora	Cloudflare's Rust proxy framework	The async runtime SBproxy is built on. Plugs into `request_filter`, `response_filter`, and `response_body_filter` lifecycle hooks.
CEL	Common Expression Language	Google's expression language. Used for per-origin policy rules, request modifiers, and response transforms.
L2 cache	Layer-2 cache backend	A shared-state backend (Redis today) that turns rate-limit counters and response-cache entries into cluster-wide state.
Cache Reserve	Long-tail cold cache tier	A second cache tier sitting under the per-origin response cache. Sample-rate driven mirroring; admission gate by min TTL and size; promotion-on-hit.
Hot path / async path	Layering rule	The pin that keeps OSS billing latency-critical code on the synchronous request path and pushes per-rail accounting to async workers.