Menu

#HumanEval

2 posts

Feed·
2 of 2 posts
HumanEval on a MacBook — 81.7% pass@1, Wi-Fi off
🖼️
0Read More

HumanEval on a MacBook — 81.7% pass@1, Wi-Fi off

DEV Community·Matt Macosko·7 months ago
#CJkiU5Jz
#why#ai#machinelearning#benchmark#humaneval

Qwen 3 Coder 30B (8-bit MLX) scored 81.7% pass@1 on HumanEval running on a single M5 Max MacBook with Wi-Fi off. Real run, all 164 problems, 14 minutes wall-clock. The first measured number for this variant on this hardware.

15s
What 500 curated failure pairs actually fix: a breakdown across 3 seeds
🖼️
0Read More

What 500 curated failure pairs actually fix: a breakdown across 3 seeds

DEV Community·namakoo [IDFU]·7 months ago
#SYq43zYl
#ai#python#model#seeds#idfu

From Dev.to - machinelearning: What 500 curated failure pairs actually fix: a breakdown across 3 seeds

15s