Menu

1 / 7

Haiku with a UX kit: can open rules and minimal code make a small model as accurate as a larger one?

DEV Community: css·MeloLabs.net·about 23 hours ago
#gu0QcqOX
#devtocss#devto#dev#haiku#rules
Reading 0:00
15s threshold

Summary. We tested whether the combination of a small model (Haiku 5.5), a UX rules data lake with open sources and a minimal checking code (the "kit") gets close to a larger model (Sonnet 5.5) on three login-screen tasks: creating, improving and comparing two screens at code level. There were 2,244 items and 2,424 calls, with 0 failures in the end. On creating and improving, Haiku with rules and kit went from 0.744 to 0.998 (create) and from 0.754 to 0.926 (improve), and the combined accuracy of Haiku with the kit was 1.07 times that of plain Sonnet, at 10% of the cost per task . On comparing two screens, the kit did not help and made things slightly worse; pairs with no real difference never produced a false alarm in any condition, so the hypothesis about false alarms could not be tested. Three non-human aesthetics instruments were validated and none passed ; for that reason this study claims nothing about beauty .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.