(And how to use it anyway)
Jakub Zerdzicki For Unsplash
I asked an AI to find me sources on a topic I know well. It gave me five citations. Four were real. The fifth had a plausible author, a plausible journal, and a plausible title, and it didn’t exist.
I only caught it because I already knew the field. If I’d been new to the topic, I would have pasted that citation into my draft and moved on.
That’s the problem in miniature. AI doesn’t fail by looking wrong. It fails by looking right.
Let me be clear about where I stand. I use AI tools almost every day, and I’m not here to tell you to throw them out. But I’ve watched people treat them as a replacement for research instead of an aid to it, and the gap between those two things is where the damage happens. Here’s what I think goes wrong.
1. It’s built to sound confident, not to be correct
Language models work by predicting what a good answer looks like. That’s a different goal from finding out what’s true.…