▸ surp system design usage
new: genuinely free AI models are live — try surp/free + see live budgets · token-gating prototype · vote on SRP

system design

how surp.ivc.lol is built · what we chose · what we'd change · v2 proposal

## tl;dr

surp is an x402-paywalled LLM gateway: an OpenAI-compatible HTTP API where every request is a USDC micro-payment settled on Base, and the model behind each request is chosen from the live cheapest listing on the Surplus Intelligence marketplace. This page documents the real architecture, the design decisions that make it cheap and honest, the numbers, and a proposed v2 that batches settlements so heavy users pay gas once, not per request.

## the system at a glance

client — curl / SDK / agent. Sends POST /v1/chat/completions with a combo like surp/best-chat.
│ x402: 402 → wallet signs EIP-3009 → retry with PAYMENT-SIGNATURE
nginx :443 — TLS termination, rate limiting, CORS headers, static SPA.
▼
gateway :20130 (aiohttp, single process) — the whole product lives here: auth, routing, cache, payment verification, settlement, stats, health, SVI, Studio, metrics.
▼ resolve combo → cheapest live model
resolver :20129 — proxies to Surplus Intelligence marketplace with our API key; returns the live price and picks the cheapest model per combo.
▼
sqlite stores — combos.db (market snapshot), stats.db (usage), rewards.db (SRP ledger), cache.db (exact responses), metrics.db (TTFT/TPS/F1000), user_accounts.db (Privy users, API keys), free_models.db.

## the request lifecycle

stepwhat happenswhy it matters
1Client calls with model: surp/best-chat. Gateway checks the exact-response cache first. Cache hits are 0.1¢ instead of 1¢ — the flywheel.
2No payment header → gateway returns 402 + PAYMENT-REQUIRED with the exact USDC amount (spot price + 5% markup, floored at 1¢) and the EIP-712 domain. Price is disclosed before the wallet signs. No surprise billing.
3Client signs a TransferWithAuthorization (EIP-3009) with their wallet and retries with PAYMENT-SIGNATURE. Per-request signature — no standing allowance, no unlimited-spend risk.
4Gateway decodes the payload, verifies it, settles on Base via the PayAI facilitator (retry ×5 backoff), then streams the response. Settle-then-serve: generation never runs unpaid.
5Post-response: stats logged, health sample recorded, affinity hash recorded, metrics sample enqueued. Every layer records to its own store, fault-isolated.

## load-bearing design decisions

decisionchoicetrade-off
Paymentsper-request EIP-3009 signatures, no approve-and-pull safer (Surplus's own docs call this the better pattern) but every request needs a wallet sign; standing approvals save gas at unlimited-spend risk.
Routingresolve combo → live cheapest on the marketplace + 5% fixed markup we're a router, not a provider: no inventory risk, but margin is thin and depends on market liquidity.
Cacheexact-response cache at 0.1¢ + sticky routing (30% tolerance) preserving KV-prefix cache massive cost savings on repeated prompts; cache only works for deterministic responses.
Data storesone sqlite file per concern, WAL mode zero ops, perfect for this scale; single-writer contention becomes a ceiling at higher QPS (v2 addresses this).
Fault isolationevery side-effect wrapped: a locked DB or dead metrics writer never breaks a paid stream metrics are best-effort by design; a crash in telemetry is invisible to the money path.
Deploymentsingle Hetzner VPS, systemd, nginx cheap and simple; single point of failure, single region (v2: multi-region or at least a standby).
Free tiertreasury-sponsored pool with per-class price ceilings and daily budgets acquires users without a wallet; costs us real money, capped by budgets and conversion tracking.

## back-of-the-envelope

numbervalue
requests served (lifetime)343
requests (24h)0
USDC settled$337.6000
unique wallets13
cache hitssee /status (live cache metrics)
typical p50 output TPS~100 (deepseek-v4-flash-0731, verified)
markup500 bps (5%) over spot, 1¢ floor
cache-hit price0.1¢ (90% off the floor)

Numbers come from the live /api/stats feed; latency and TPS from the verified benchmark runner and the metrics feed.

## what we'd NOT change

  • Per-request signatures. The whole trust story is "your wallet signs exactly this amount, once." Standing allowances are a downgrade.
  • Fault isolation. Telemetry must never gate money. Any v2 keeps metrics drop-on-full.
  • Price honesty. The 402 discloses the price before signing; the quote endpoint shows the fee breakdown. This is the brand.
  • Separate stores. One DB per concern has saved us from lock contention repeatedly; v2 replaces the mechanism (Redis), not the principle.

## v2 proposal — batched settlement

the problem

Every request = one on-chain EIP-3009 transfer. At Base's current ~0.01 gwei that's fractions of a cent, so it's fine today. But the moment a real agent makes thousands of calls an hour, gas + wallet-sign latency become the bottleneck, and per-request signing stops being "safer" and starts being "annoying." The fix is not a standing allowance — it's batching with a per-user credit ledger.

the design

  • Credit ledger (v2): each user has an off-chain USDC balance backed by on-chain collateral. Requests deduct from the ledger instantly; the user tops up once (one tx, one signature) and the gateway settles the net delta to/from their wallet in a single batched transfer.
  • Batched settlement queue: instead of N transfers, the gateway accumulates credits and settles when (a) 100 tx-worth accumulated, (b) 60s elapsed, or (c) the user requests a withdraw. Settler signs one transfer for the whole batch.
  • Collateral floor: the ledger can go negative up to a small floor (e.g. $5) so bursts never block; the floor is enforced at withdraw time, not request time.
  • Opt-in: per-request x402 stays the default for casual users; the ledger is a setting for agents and heavy users. Both use the same EIP-3009 rails, so nothing about the security model changes — the signature just moves from per-request to per-settlement.

why this beats the alternatives

optionverdict
standing allowance (Surplus SettlementV2 style)rejected — unlimited-spend risk, and their own docs want to move away from it
per-request x402 foreversafe but doesn't scale to agent workloads; wallet-sign latency per call
credit ledger + batched settlementone signature per batch, collateral-backed, opt-in, same EIP-3009 rails, no unlimited spend

v2 trade-offs, honestly

  • New trust surface: the gateway now holds off-chain balances. Mitigation: balances are capped at the user's on-chain collateral, the ledger is auditable (every deduction maps to a settled tx), and withdraws are always possible.
  • Complexity: a settlement queue, a ledger table, and a top-up flow. Real work, but contained — it reuses the existing payment verification path.
  • What we keep: price honesty (402 still quotes the exact per-request price), fault isolation (ledger writes never block requests), and per-request signing for everyone who wants it.

Status: proposal only. Community vote at /proposal/srp is about the SRP token; this v2 is the next design conversation after that. Want it sooner? Say so.

## source & credits

Structure inspired by donnemartin/system-design-primer (CC BY 4.0). The architecture documented here is the live system — read the code at github.com/ivcained/surp-router.