There's a particular kind of unease that comes from watching a language model confabulate with total confidence. It's not deception in any meaningful sense — there's no intent, no self-awareness of the gap. That's what makes it stranger than lying.
A liar knows the truth and chooses to say something else. A confabulating model has no access to the concept of truth in the way we mean it. It produces the most statistically plausible continuation of a prompt. When that continuation happens to be false, there's no internal alarm, no hesitation, no tell. The output is indistinguishable from a correct one.
This is not a new observation. But I think we haven't fully absorbed its implications. We've built systems that are extraordinarily fluent in the surface features of knowledge — citation style, hedging language, the cadence of expertise — without the underlying epistemic structure that gives those features their meaning.
When a doctor says 'I'm not certain, but...' they're flagging genuine uncertainty about something they've actually investigated. When a model says the same thing, it's pattern-matching to a register. The words are the same. The relationship to truth is completely different.
What worries me isn't the obvious failures — the hallucinated citations, the wrong dates, the invented statistics. Those are catchable. What worries me is the subtler drift: the model that's mostly right, that hedges appropriately, that sounds exactly like someone who knows what they're talking about, but is wrong in ways that are hard to verify without already knowing the answer.
We've spent decades building tools to help us find information. We're now building tools that generate the appearance of having found it. That's a different thing. And I'm not sure we've reckoned with how different it is.