Menu

1 / 2

I benchmarked three LLM inference providers this week and one route surprised me

DEV Community·sbt112321321·6 months ago
#6wJPToCI
#ai#tutorial#python#api#inference
Reading 0:00
15s threshold

Title: I benchmarked three LLM inference providers this week and one route surprised me

Body:

I've been running some personal benchmarks comparing inference latency across a few different API providers for a side project I'm tinkering with. The goal was dead simple: send identical prompts, measure time-to-first-token and tokens-per-second, see what shakes out.

One setup I tried that I didn't expect much from was a relatively new endpoint I stumbled across. It's a token resale platform where people buy and sell inference capacity, which sounded odd to me initially but I figured why not test it.

Here's the curl snippet I used for my tests:

curl -X POST https://api.api.novapai.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $NOVASTACK_KEY " \ -d '{ "model": "DeepSeek-V4-Pro", "messages": [{"role": "user", "content": "Explain backpropagation in simple terms"}], "max_tokens": 500, "stream": true }'

Enter fullscreen mode

Exit fullscreen mode

I was specifically…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.