Comparing two language models fairly takes four numbers per request: time to first token, total time, output tokens and cost. This post gives a small script that collects them through one OpenAI-compatible client, plus what it showed for Qwen 4 Max and Claude Opus 5.5. Short answer: they tied on correctness for everyday tasks, Qwen 4 Max cost about a third as much, and public data favours Opus 5.5 once the work gets large.
Disclosure: I ran this on apimodels.app, which is the API I work on. apimodels.app is a multi-model API gateway: one key and one OpenAI-compatible endpoint for image, video, audio and language models, which is why one client can reach both models. I wrote this from my own test data, with AI assistance for drafting and editing.
The script
import os , time from openai import OpenAI
client = OpenAI ( api_key = os . environ [ "LLM_API_KEY" ] , base_url = os .…