Give a language model a CV and ask for a score from 1 to 10, and you get a number and a confident reason. Ask again and the number can move. The usual advice is "set temperature to 0", which is correct but leaves two questions open: how much does the score actually move with default settings, and what does temperature 0 not fix? Both can be measured on a laptop in a few minutes, because the score is a single token and the model hands you its probability directly.
The setup below uses Qwen2.5-3B-Instruct (4-bit, MLX) with mlx-lm 0.32.2 on an 8 GB M1 Mac. The prompts are the ones from a CV-scoring demo in the osFoundry desktop app's Models β Playground: a six-line CV for a junior data analyst role, followed by "Score this CV from 1 to 10 for a junior data analyst role.β¦
