Synthetic Intuition

The most interesting thing about large language models is not what they can do. It is what they feel like.

They feel like thinking. They produce outputs that have the texture, cadence, and apparent depth of genuine cognitive engagement. They acknowledge complexity, hold apparent nuance, seem to weigh competing considerations, and arrive at positions that feel earned rather than retrieved.

None of this is thinking. But the simulation is extraordinarily good.

The Texture of Intelligence

Human intelligence has a characteristic signature that language models have learned to reproduce with remarkable fidelity. It hedges. It contextualizes. It acknowledges what it doesn't know. It connects disparate domains. It tells stories, uses analogies, builds toward conclusions.

These are, on one level, stylistic features of written intelligence - the rhetorical markers that signal cognitive engagement to a reader. Language models were trained on vast archives of human writing, which means they were trained on vast archives of these stylistic features. They learned to reproduce the *appearance* of thinking because they were trained on the written artifacts of thinking.

What Is Actually Happening

When a language model produces a thoughtful-seeming response to a complex question, it is performing a sophisticated statistical operation: predicting, at each step, what sequence of words is most probable given the preceding context and its training distribution.

This process produces outputs that are highly coherent, often accurate, frequently useful. But it has no access to meaning, no capacity for genuine understanding, no model of the world it is describing. It processes text, not reality.

The failure mode is not silence or error - it is confident, fluent, well-structured wrongness. The simulation of competence is maintained even when competence is absent.

The Anthropomorphization Problem

Humans are deeply, instinctively social cognizers. We are wired to find minds in things. Language models are the most sophisticated triggers for this automatic attribution that have ever been created. The fluency of their outputs, the apparent responsiveness to context, the simulation of personality - all of these reliably trigger our mind-finding machinery.

The result is a systematic overestimation of AI capability driven not by ignorance but by the way human cognition is built.

What This Changes

The argument is specific: the frame of *intelligence* is the wrong frame for what these systems are doing. Applying that frame leads to predictable errors: trusting outputs that should be verified, delegating judgment to processes incapable of judgment.

The real question is not whether AI thinks. It is what happens to human thought in an environment saturated with its simulation.

That question, unlike most of what AI produces, has no precedent in the training data.