Menu

I ranked AI models using only public benchmarks — and stopped publishing scores
📰

I ranked AI models using only public benchmarks — and stopped publishing scores

DEV Community: machinelearning·XM Yang·about 15 hours ago
#drxM4tl3
Reading 0:00
15s threshold

Most "best AI" lists publish scores nobody can reproduce. Different agent harnesses, different prompts, vendor-reported numbers mixed with independent runs — the numbers look precise and mean almost nothing.

So on Tools Hub's rankings pages I made two rules. First: order only, no scores. Second: every board names its source on the page — SWE-bench Verified (independent runs compiled by Epoch AI) for coding, HealthBench for medical questions, LMArena's blind human votes plus EQ-Bench for writing, and Arena's text-to-image votes for image models.

The interesting part is where the boards disagree. EQ-Bench's judged board puts GPT first for creative writing; LMArena's human voters put Claude first. I follow the humans and say so on the page. That disagreement is more useful to a reader than a fake-precise 9.4/10.

Four boards are live (coding, medicine, writing, image).…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.