Menu

Where Was the Author of The Ugly Duckling Born? What 11 LLMs Taught Me About Connecting Facts
πŸ“°

Where Was the Author of The Ugly Duckling Born? What 11 LLMs Taught Me About Connecting Facts

DEV Community: awsΒ·Alexey VidanovΒ·about 4 hours ago
#dXeUtqom
#devtoaws#devto#dev#model#models
Reading 0:00
15s threshold

A bigger model will not connect your company's facts for you. On 600 Wikidata questions, the best model that answers without thinking connected two facts it knew in 55 % of cases, and in about 30 % for little-known facts. Asked to write both steps first, it reached 98 %.

Ask a language model two questions:

Who wrote The Ugly Duckling ?

In which city was Hans Christian Andersen born?

All eleven models I tested get both right: Hans Christian Andersen, Odense.

Now ask one question that needs both facts, and ask for the answer only:

In which city was the author of the book The Ugly Duckling born?

Five of the eleven say:

Copenhagen.

Copenhagen is where Andersen lived and wrote. It is the city his name brings up first. It is not what I asked. Claude Haiku 4.5 and Qwen3 235B are among the five, and each of them answers "Odense" when asked about Andersen by name.

It is not one unlucky question. Ask where the director of the 1960 film La Dolce Vita was born.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.