Two Root Causes Behind LLM "Hallucinations"

Anyone who has worked with large language models has run into these two situations:

Situation 1: Knowledge Cutoff

You: What were our company's Q1 sales figures? GPT: I'm sorry, my training data only goes up to early 2024 and I have no access to your company's internal data.

Enter fullscreen mode

Exit fullscreen mode

Situation 2: Hallucination

You: How do I use LangChain's RunnablePassthrough? GPT: RunnablePassthrough can be enabled by calling .with_config(pass_through=True)... (This parameter doesn't exist.)

Enter fullscreen mode

Exit fullscreen mode

Both problems share the same root cause: an LLM's knowledge is frozen at training time.

The moment training completes, the model's "memory" is locked in place. It has no idea what happened today, knows nothing about your internal documents, and won't go look things up—it can only answer from memory.…