Herbert A. Simon's "Literary Criticism: A Cognitive Approach" radiates good sense and generosity. Many schools of criticism are in good order as they stand. Let us all join hands, he suggests, to investigate the different kinds of information-fact-stating, emotional, imagistic, sacred, or whatever-that may be carried by different texts and recovered by us, the beings who process them. We might hope to heal the rift between C. P. Snow's two cultures. Science can explain the structures and procedures of information processing; various humanists can help us to attend to the information that gets processed.
It is all very generous and sensible. But nonetheless I think there is something badly wrong. I can focus my worries on one small parenthesis. Simon writes:
The physical system hypothesis asserts that possessing these processes [viz. suitably associating symbols with one another, recalling them appropriately to produce what we accept as fluent performance] is the necessary and sufficient condition for a system to be capable of thinking (8).
In short, passing some version of the Turing test is necessary and sufficient for thinking. Simon then adds this: "To the extent that the hypothesis is true (an empirical question), computers can be used to simulate human thinking. . . ."(8). Is the sentence hidden in that parenthesis true? That is, is it, as Simon says, an empirical question whether a machine that passes the Turing test thinks?
Sometimes I find myself tempted to say that it is impossible that a machine pass what I would regard as a suitable Turing test, responding in ways that are indistinguishable from the responses of a human being. The machine would have to be, pace Turing, not just connected to a terminal, but connected to a body. That body would have to shudder with pleasure (with what feels to me like its pleasure) when touched in certain ways, recoil in pain when touched in others. It would have to laugh at some of my jokes, and be puzzled by others. It would have make promises and ask for forgiveness, even issuing mute glances that hint at wishes for reconciliation or recognition. (Notice that Simon as a functionalist says that emotions, images, and feelings of wonder are nothing but bits of information that lie beneath and cause apt response. On his account they will have to show themselves in the apt behavior of the machine.) So it performs, acts, as a human being. But then we (you and I) push the release mechanism that unlocks its (lifelike, humanlike) chest, and we see the mechanical workings inside. Maybe they even resemble human organs. Despite all its indiscernability from the appearances and performances of a human being, you assure me that you have built it out of chemicals and wires. What do we say?1
Sometimes, I said, I feel like saying that all this could not happen. Surely machines could not do what we now do: joke, forgive, flirt, and avoid or seek one another. But such a response, while tempting, has the ring of stipulation. It sounds like an armchair discovery of a necessary truth, an intellectualist philosopher's empty game. And so Simon says: but haven't we discovered-you conceded it was possible-that a machine can do this, can think?
But it is not really a matter of what could physically happen. The point is that we do not know what to say when we are asked: does such a physical creature, or creation, think? Simon's view is that whether passing an enriched Turing test counts as thinking is itself an empirical matter. We discover, he imagines, that it does count, just as we might discover how many moons Neptune has. It is one fact among others, the kind of fact that can be discovered. (If it is not that kind of fact, I do not see what it means to call it an empirical matter.)
And is that right? No. What happens is that words fail me. The machine does what it does: performs in a way that is indistinguishable to me from the performances of a human being. And now I am asked: "Does it think? Don't you see that it thinks? Haven't we discovered that it thinks?"-And I don't know what to say. It does all those things human beings do. What information, what fact, is now added, discovered, stated, by saying that it thinks? It seeks reconciliation with me. Once I have conceded that, am I then supposed to see that as a result, hitherto unnoticed by me, it thinks? That claim is nonsense. Or I can put no sense in it. And the negation of nonsense is still nonsense, rather than a necessary truth stating a super fact about the essence of thinking.
There is a deep temptation, evident in Simon's article, to turn every significant question into an empirical question. What could be more open-minded than to wait for the facts to show themselves? We are familiar with the language-game of science, which has found out so much about physical systems in the world. But that language-game also has its costs. It can blind us to seeing how issues about the nature of the human arise and are settled. Blinded by our wish for a hidden fact, we miss where thinking is. Simon's view-that there is something about thinking still to be found out-represents humanity or thought as somehow hidden, locked up, mutely beneath our behavior (perhaps some of us, but not others) and causally effective. That view teaches us to look for our humanity in the wrong place, casting it as a potential object of control. I can put no sense in it.
1. My imagining this scenario in this way is based on Stanley Cavell's pages on "perfecting an automaton" (Cavell, 1979:pp. 403-8).