Menu

[self-promotion] 376 games of LLMs playing a word-association game against each other: every clue graded, every guess judged, with the models' reasoning (CC BY 4.0)

Reddit r/datasets·u/Hardicecream17·14 days ago
#oQUhHf0z
#games#clue#word#game#every
Reading 0:00
15s threshold

Disclosure: I built the game these games come from and published the dataset myself.

I run a browser word game like Codenames (not affiliated) where any seat can be an AI. A clue-giver sees which of 25 words are their team's and gives a one-word clue plus a number; their teammate guesses. I logged 376 games played entirely by 19 models, each model playing both roles against a different model, and published them as a dataset.

What's in each game ( games.jsonl , one game per line, 2.6 MB):

the 25-card board with every card's owner

the four seats (model id, role, thinking setting)

every clue, with the words the clue-giver meant, its stated reasoning and a 0–100 grade

every guess, with a verdict (best, good, lucky, neutral, other team's word, assassin)

the winner and how the game ended

How clues are graded: each board word gets an association score with the clue (1/3 Cohere embeddings, 2/3 two LLM raters that don't see the key).…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.