On AI "Hallucinations"
June 11 2025
I’ve been really impressed with the presentations and discussions at the Databricks Data+AI meeting this week in San Francisco. There is a tangible energy of optimism around all of the AI and data innovations that have come in the past year, and the avalanche of AI tools and capabilities that clearly will change everything. We are living through a phase transition from pre-AI compute to post-AI compute and I couldn’t be more excited for it...
...but my inner pedant (in the voice of the Simpsons’ Dr. Frink) compels me to make a point about the use of the term ‘hallucination’ to refer to generative AI responses that are not grounded in truth or reality. Since hallucinations by definition involve a sensory perception in the absence of an actual stimulus, and LLMs don’t have senses and do not perceive (either in the absence or presence of stimulus), this metaphor fundamentally misrepresents what’s happening.
I fear our use of language that usually refers to human cognition in the context of AI often leads people to think that AI is simply a computer thinking like a human, when in reality it is much, much more novel and interesting than that.
If we insist on borrowing from the lexicon of human cognition, ‘confabulation’ is certainly more apt than ‘hallucination’ - at least it captures the fabrication of plausible but false information without implying sensory experience. But even this term perpetuates the flawed analogy between human thinking and LLM outputs.
Perhaps we should embrace philosopher Harry Frankfurt’s technical definition of ‘bullshit’ - speech produced without concern for truth - which accurately describes what happens when an LLM generates text beyond its training data. Frankfurt’s insight was that bullshit isn’t about lying (which requires knowing the truth), but about a complete indifference to whether something is true or false. This perfectly captures LLM confabulations: they’re not trying to deceive us, they simply have no mechanism for caring about truth.
I think this both points to a possible solution to the problem (force the models to care about and attempt to enforce the truth, like with robust RAG workflows), as well as a fundamental truth that we should all embrace. As we build this brave new world of AI, let’s use language that helps us understand what these systems actually do, rather than comfortable metaphors that obscure the details.
Thinking of LLMs as ‘thinking like a person’ and calling what they do ‘reasoning’ (hat tip to the recent paper from Apple explaining this very clearly) or ‘hallucination’ is missing the point. These things are new with no clear analogy to human experience, and that’s why we’re all so excited and optimistic. Lazily thinking of AI by analogy to human thinking does us all a disservice... so go read Frankfurt’s monograph ‘On Bullshit’ and I suspect you’ll agree with me... no bullshit! :)
#DataAISummit