04 · Philosophy of Mind

Part I's fourth hearing: metaphysics asked what a program is; this chapter asks whether that same kind of thing could be what you are.

Is the mind what the brain computes?

In 1980, John Searle asked readers of Behavioral and Brain Sciences to imagine him locked in a room with a rulebook written in English and a slot through which cards covered in Chinese characters arrive. Searle does not read Chinese. But the rulebook is good enough — for any incoming string of characters, it tells him, by shape alone, which characters to pass back out. To a Chinese speaker outside, the answers look fluent, even witty; the room, treated as a black box, could pass a Turing test conducted in Chinese. Searle's question was simple and became one of the most argued-over pages in twentieth-century philosophy: does anyone, or anything, in that room understand Chinese? His answer was no. A person manipulating symbols by their shape, however successfully, is not thereby extracting their meaning — and if that is true of a man with a rulebook, it is true of a computer running the equivalent program, however large. Searle called the paper "Minds, Brains, and Programs" (Behavioral and Brain Sciences 3, 1980: 417–457); the journal ran it in its open-peer-commentary format, its objectors printed alongside, and the arguing has not really stopped since.

The Chinese Room was aimed at a specific, then-dominant target, which is this chapter's first principle: the computational theory of mind. Hilary Putnam's papers from 1960 onward, culminating in "The Nature of Mental States" (1967), proposed that a mental state is not defined by what it is made of — neurons, silicon, anything — but by its functional role: what causes it, what it causes, how it relates to other mental states. His guiding picture was the Turing machine, whose abstract "machine table" of states is exactly the same whether it runs on relays, vacuum tubes, or a napkin and a pencil, so long as the transitions are honored. Jerry Fodor pushed the theory from an analogy into an architecture in The Language of Thought (1975): thinking, he argued, is computation over an internal symbolic code — "Mentalese" — with a combinatorial syntax and a compositional semantics, exactly the kind of structure a digital computer manipulates. On this view, a mind literally is a program running on the wet hardware of a brain, and Searle's room is the theory's most inconvenient test case: a system that, on Putnam and Fodor's own terms, is running the right program, and yet — Searle insists — plainly understands nothing.

The second principle sits one level below that metaphysical claim, and it is easy to confuse with it: Alan Turing's 1950 proposal, in "Computing Machinery and Intelligence" (Mind 59, no. 236: 433–460), that the question "can machines think" is too laden with undefined terms to argue productively, and should be replaced by an operational one — can a machine's conversational behavior be mistaken for a human's, in a structured game of questions and answers? Turing was not claiming that passing the game proves the presence of a mind; he was proposing to stop asking the metaphysical question and start measuring the behavioral one. The distance between Turing's move and Putnam and Fodor's is exactly the distance the Chinese Room exploits: Turing offers a test for behavior; the computational theory of mind claims that the right behavior-producing process, whatever it turns out to be, simply is thinking; Searle's room passes the first while, he argues, refuting the second.

The third principle is the argument's afterlife: the replies, and what they reveal. The "systems reply" holds that Searle is right that the man alone understands nothing, but wrong to conclude that no one does — understanding, on this view, belongs to the whole system: man, rulebook, and paper together. Searle's counter was to have the man memorize the entire rulebook and work the transformations in his head, leaving no separate "paper" for understanding to hide in; by his lights, still no understanding, only a faster clerk. The "robot reply" tries a different repair: wire the room to cameras and motors, so the symbols are caused by, and cause changes in, a real world instead of arriving from and returning to another room. Searle's response — the symbols are still just more symbols, now merely triggered by transducers rather than slips of paper — names, without quite solving, what later literature calls the symbol grounding problem: rules connecting symbols to other symbols, however elaborate, may never on their own reach the world the symbols are supposed to be about.

The fourth principle is an older civil war that current AI reopened rather than settled. Fodor, together with Zenon Pylyshyn, turned his defense of the language of thought into a direct attack on the rival architecture then rising in cognitive science: connectionism, the parallel distributed processing (PDP) networks of David Rumelhart and James McClelland's 1986 volumes. "Connectionism and Cognitive Architecture: A Critical Analysis" (Cognition 28, 1988: 3–71) argued that thought is systematic in a way pure connectionist networks cannot, in principle, explain: anyone who can think John loves Mary can think Mary loves John — the two thoughts share combinatorial parts that recombine, which requires, they claimed, an underlying symbolic syntax, not a pattern-matcher trained on enough examples of both shapes. For thirty years the argument looked mostly settled toward hybrid views. Large language models reopened it at a scale nobody in 1988 anticipated: systems built from nothing but weighted connections now produce output systematic enough to fool most casual tests, with no hand-built symbolic layer in sight — and the field is still arguing — empirically now, not only philosophically — whether that refutes Fodor and Pylyshyn or whether large networks quietly learn something symbol-like after all, a question chapter 13 picks up.

The fifth principle is the one no reply above touches: David Chalmers's distinction, first presented at the 1994 Tucson conference on consciousness and published as "Facing Up to the Problem of Consciousness" (Journal of Consciousness Studies 2, no. 3, 1995: 200–219), between the easy problems of mind — explaining discrimination, integration, reportability, attention, all of them tractable, in principle, by computational or neural mechanism — and the hard problem: why any of that processing is accompanied by subjective experience at all, why there is something it is like to be the system doing it. A computational theory of mind, however complete, explains structure and function; Chalmers argues it can stay silent on why function comes with feeling, illustrated by his "zombie" duplicate — functionally identical to a conscious being, with no inner experience at all. Whatever one makes of zombies, the hard problem marks where this chapter's apparatus — programs, functions, computation — may run out of the right kind of thing to explain. What current AI revives is all four earlier principles at once, unresolved, at a scale nobody argued at before: the field keeps measuring behavior, because behavior is what it knows how to measure, while the Chinese Room's question has not gone anywhere.

The people, briefly. Putnam abandoned computational functionalism before many of his own students did, uneasy with a theory that could not survive his own triviality argument from chapter 02. Fodor defended the language of thought against successive rivals for forty years and outlived most of them. Searle has never claimed computers cannot in principle think — his target is the sufficiency of syntax, not the possibility of machine minds by other means. Chalmers named the hard problem in his twenties and has spent every year since being asked to solve it.

Connections

Chapter 02's implementation problem is this chapter's silent premise: if implementing a program does not settle what a system is computing, functionalism inherits that gap directly. Chapter 08 returns to embodiment from Dreyfus's angle, against which the robot reply is a partial, contested answer. Chapter 09 owns the Lucas–Penrose argument, which aims Gödel's limits at this chapter's thesis from outside. Chapter 13 owns the full empirical run of connectionism versus symbols, grounding, and what interpretability finds inside the systems this chapter can only interrogate from outside.

Open questions

If Turing's operational test and Searle's understanding-requirement are both individually reasonable, can they be held at once — is passing the imitation game evidence of thought, weak evidence, or simply orthogonal to it? And if the hard problem is genuinely intractable to computational explanation, what follows for how we ought to treat systems whose behavior increasingly resembles a mind's, when behavior is the only thing any test — Turing's included — was ever built to measure?

Go deeper

• Margaret A. Boden, Mind as Machine: A History of Cognitive Science, 2 vols., Oxford University Press, 2006 — the field's own history, told by one of its central participants.
• John R. Searle, "Minds, Brains, and Programs," Behavioral and Brain Sciences 3 (1980): 417–457; Jerry A. Fodor and Zenon W. Pylyshyn, "Connectionism and Cognitive Architecture: A Critical Analysis," Cognition 28 (1988): 3–71.
• The Computational Theory of Mind, Stanford Encyclopedia of Philosophy.