the one number: cost × intelligence × speed
A single composite score per model — weighted geometric mean of three normalized sub-scores. Buyers optimize for the highest SVI in class; suppliers climb the leaderboard by submitting verified benchmark results or improving real served TPS.
cost 45% · intelligence 40% · speed 15%
| # | model | SVI | cost | intel | speed | price | tps |
|---|---|---|---|---|---|---|---|
| 1 | gpt-5.6-luna | 83.7 | 70.5 | 95.0 | 100.0 | $0.0184/M | 271.8 tps |
| 2 | deepseek-v4-flash | 66.8 | 100.0 | 60.0 | 26.5 | $0.0091/M | 71.9 tps |
| 3 | glm-5.2 | 66.4 | 73.2 | 70.0 | 42.9 | $0.0171/M | 116.6 tps |
| 4 | deepseek-v4-pro | 59.1 | 68.1 | 70.0 | 24.7 | $0.0197/M | 67.0 tps |
| 5 | deepseek-v4-flash-0731 | 50.3 | 50.4 | 60.0 | 31.3 | $0.0360/M | 85.0 tps |
| 6 | llama-3.3-70b-instruct | 47.6 | 44.4 | 70.0 | 21.1 | $0.0464/M | 57.4 tps |
You don't have to use the default weights. Pass surp_mode to
any combo to route by the lens that fits the job:
| mode | weights (cost·intel·speed) | use for |
|---|---|---|
cost | pure cheapest | overnight batch, agents that can wait |
value | 45·40·15 | default SVI — best all-round value |
balanced | 33·33·33 | no strong preference |
speed | 15·15·70 | interactive work, pair programming |
intel | 20·60·20 | hard reasoning problems |
curl -X POST https://surp.ivc.lol/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"surp/best-chat","surp_mode":"speed",
"messages":[{"role":"user","content":"hi"}]}'
Or go fully custom with surp_weights — a
cost:intel:speed triple, e.g. "surp_weights":"0.3:0.4:0.3".
Models without a verified TPS fall back to cheapest so routing never fails.
The 402 response includes routing_reason so you can see which lens won.
curl -X POST https://surp.ivc.lol/api/svi/benchmark \
-H "Content-Type: application/json" \
-d '{"model":"my-quantized-model","mmlu":88,"humaneval":92,"submitter":"you"}'
Verified submissions move the leaderboard. Missing axes fall back to the model class default so partial submissions still count.