▸ verified llm performance usage
new: genuinely free AI models are live — try surp/free + see live budgets · token-gating prototype · vote on SRP

verified llm performance

real output TPS, TTFT, and throughput-per-dollar — measured, not claimed.

Every number on this page comes from a real streaming request sent through surp to the Surplus Intelligence marketplace. No vendor spec sheets, no extrapolated benchmarks — only observed generation throughput. This is the data the Surplus dashboard doesn't show.

why this matters: A model listing at $0.01/M might sound cheap, but if it generates at 20 tokens/second, a 1000-token response takes 50 seconds. A model at $0.06/M generating at 100 TPS delivers the same response in 10 seconds. Throughput-per-dollar is the metric that actually determines your cost-per-task. We measure it directly.

throughput leaderboard

Models ranked by throughput-value score (p50 output TPS × tokens-per-dollar). Green ≥ 80 TPS, amber 40-79, red < 40.

#modelp50 TPSp95 TPSrange p50 TTFTprice /1Mthroughput/$runs
1gpt-5.6-luna279.6400000.049.4-400000.05038ms$0.023012.18B196/216
2glm-5.2117.4276.815.8-1619.43670ms$0.01716.89B221/221
3deepseek-v4-flash71.7204.49.7-492.62280ms$0.01494.82B225/225
4deepseek-v4-flash-073184.5134.38.4-380.92789ms$0.03602.35B224/225
5deepseek-v4-pro69.8117.236.6-208.92428ms$0.03122.24B220/220
6llama-3.3-70b-instruct54.6516.412.3-551.71563ms$0.0611893.8M213/215

what "output TPS" means

Output TPS = generated completion tokens ÷ generation seconds (time from first token to last token). This is the metric LLM buyers care about: how fast the model produces text.

This is different from request RPS (requests per second), which measures how many separate requests the gateway handles. The health board tracks RPS; this page tracks output TPS.

methodology

  • Real requests: each run sends a streaming POST /v1/chat/completions through surp with max_tokens=400.
  • TTFT: time from request submission to first output token (time-to-first-token).
  • Generation time: wall time − TTFT (time spent generating, excluding queue/TTFT).
  • Output TPS: completion_tokens / generation_seconds.
  • Throughput-value score: p50_output_tps × (1,000,000 / price_per_1m) — higher is better.
  • p50/p95: percentile across all successful runs in the window.
  • Token counting: prefers usage.completion_tokens from the API; falls back to len(text)/4 estimate if usage is absent.
  • No cherry-picking: all runs (including slow ones) are recorded. Failed runs count against the failure rate.

recent observations: gpt-5.6-luna

Last 10 raw runs — verify the numbers yourself.

output TPSTTFTwalltokensprice/1Mstatus
1612.94668ms4916ms400$0.0187ok
1632.76217ms6462ms400$0.0187ok
207.42216ms3209ms206$0.0187ok
178.42160ms4077ms342$0.0187ok
400000.04595ms4596ms400$0.0187ok
0.06322ms6322ms400$0.0169ok
264.64091ms5603ms400$0.0169ok
201.21840ms2814ms196$0.0169ok
209.92713ms4595ms395$0.0169ok
1646.16149ms6392ms400$0.0169ok

try it yourself

The benchmark runner is open source. Clone the repo and run:

python3 benchmark_runner.py --model deepseek-v4-flash-0731 --runs 10 --api-key sk-sur-...

You'll get the same kind of numbers we publish here. Or just make a streaming request and time it:

curl -N -X POST https://surp.ivc.lol/v1/chat/completions \
  -H "Authorization: Bearer sk-sur-..." \
  -H "Content-Type: application/json" \
  -d '{"model":"surp/direct/deepseek-v4-flash-0731",
       "messages":[{"role":"user","content":"Write 200 words about HTTP/2"}],
       "max_tokens":400,
       "stream":true}'

related: provider health board (RPS, latency) · free models · cache-affinity auction · benchmark API