Menu

How I built a local AI assistant that admits when it didn't actually do the thing
πŸ“°

How I built a local AI assistant that admits when it didn't actually do the thing

DEV Community: opensourceΒ·Kayato123Β·12 days ago
#EBEQjk9Y
#devtoopensource#devto#dev#model#tool
Reading 0:00
15s threshold

How I built a local AI assistant that admits when it didn't actually do the thing

The sentence that started this project was five words long:

"I've sent the email."

It hadn't. Nothing had run. There was no email tool call in that turn, no SMTP connection, no record of anything. Just a small local model, a confident tone, and a sentence that was completely false.

If you've wired a 7B model up to function calling, you've probably met the same failure. It's not a crash. It's not an exception in your logs. It's worse: it's a fluent, well-formed paragraph that silently disagrees with reality. Your UI shows a checkmark, the user moves on with their day, and the thing never happened.

This is the story of the fix in my open-source project, Quasnooze , and the one design decision I'd defend hardest if you asked me why it's built the way it is.

The problem: hallucination isn't just wrong facts, it's wrong actions

Most discussion of hallucination is about knowledge.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.