CROSSPOST: SCOTT AARONSON (2013): Against John Searle's Analogy of an Apparently Chinese

CROSSPOST: SCOTT AARONSON (2013): Against John Searle's Analogy of an Apparently Chinese-Understanding Room Scott Aaronson’s complexity rebuttal to Searle's "machines can mimic but cannot understand"; Searle's intuition pump hides the scale of the computations... John Searle’s “room that speaks but does not understand Chinese” feels powerful and attractive only because it hides the ball. Make the “room” the size it would actually have to be, and the intuition reverses. This is what Scott Aaronson says, and I believe he is right” Thinking about “AI”, I find my mind once again going—in almost a reflex, like a low-order Markov process iterating on a fixed function—to Scott Aaronson’s argument that our instincts betray us when you go full reductionist and start saying that things are “just…” His argument comes in Quantum Computing since Democritus and elsewhere. It is a rebuttal to John Searle’s image: an apparently Chinese-speaking room that, in fact, does not understand Chinese. Aaronson’s is a “complexity” response: Searle’s image gets its force from a misleading picture of a small rule book, or a few shelves of books that together constitute a lookup table. But consider not a bookshelf but a datastore Earth-sized, and searched by tens of thousands of near-light-speed robots. The sheer scale of the computation makes it much more than plausible that you might see such a 电脑, a diànnǎo, as plausibly understanding Chinese. Thus after a certain point complexity gains metaphysical significance: it is no longer “just…”, but rather something for which, while the reductionist perspective is still true, the emergent properties of the complex are so weighty that the reductionist perspective misses what any sensible observer would see as the real point. Here, for reference, is the argument as Scott makes it: CROSSPOST: SCOTT AARONSON (2013): Against John Searle’s Analogy of an Apparently Chinese-Understanding Room . In the last 60 years, have there been any new insights about the Turing Test itself? In my opinion, not many. There has, on the other hand, been a famous “attempted” insight, which is called Searle’s Chinese Room. This was put forward around 1980, as an argument that even a computer that did pass the Turing Test wouldn’t be intelligent. The way it goes is, let’s say you don’t speak Chinese. You sit in a room, and someone passes you paper slips through a hole in the wall with questions written in Chinese, and you’re able to answer the questions (again in Chinese) just by consulting a rule book. In this case, you might be carrying out an intelligent Chinese conversation, yet by assumption, you don’t understand a word of Chinese! Therefore, symbol-manipulation can’t produce understanding. So, how might a strong AI proponent respond to this argument? Well, she might say: you might not understand Chinese, but the rule book does! Or if you like, understanding Chinese is an emergent property of the system consisting of you and the rule book, in the same sense that understanding English is an emergent property of the neurons in your brain. Searle’s response to that is, fine, just memorize the rule book! Then there’s no “system” other than your own brain, but you still don’t “understand” Chinese. To which the AI proponent shoots back: there is too another “system” in this case! Supposing you memorized the rule book, we’d need to distinguish between the “original” you and the new, simulated being brought into existence by your following of the memorized rules – a being whose only relation to you might be that it happens to inhabit the same skull. That response might sound crazy, but only to someone who’s never studied computer science. To a computer scientist, it seems perfectly reasonable to say that one computation (say, a LISP interpreter) can conjure into existence a different, unrelated computation (say, a spaceship game) just by dutifully executing rules. Look, as I’ll discuss later, I don’t know whether the conclusion of the Chinese Room argument is true or false. I don’t know what conditions are necessary or sufficient for a physical system to “understand” Chinese – and neither, I think, does Searle, or anyone else. But considered as an argument, there are several aspects of the Chinese Room that have always annoyed me: One of them is the unselfconscious appeal to intuition – “it’s just a rule book, for crying out loud!” – on precisely the sort of question where we should expect our intuitions to be least reliable. A second is the double standard: the idea that a bundle of nerve cells can understand Chinese is taken as, not merely obvious, but so unproblematic that it doesn’t even raise thequestion of why a rule book couldn’t understand Chinese as well. The third thing that annoys me about the Chinese Room argument is the way it gets so much mileage from a possibly misleading choice of imagery, or, one might say, by trying to sidestep the entire issue of computational complexity purely through clever framing. We’re invited to imagine someone pushing around slips of paper with zero understanding or insight – much like the doofus freshmen who write (a + b)2 = a2 + b2 on their math tests. But how many slips of paper are we talking about? How big would the rule book have to be, and how quickly would you have to consult it, to carry out an intelligent Chinese conversation in anything resembling real time? If each page of the rule book corresponded to one neuron of a native speaker’s brain, then probably we’d be talking about a “rule book” at least the size of the Earth, its pages searchable by a swarm of robots traveling at close to the speed of light. When you put it that way, maybe it’s not so hard to imagine that this enormous Chinese-speaking entity that we’ve brought into being might have something we’d be prepared to call understanding or insight. Of course, everyone who talks about this stuff is really tiptoeing around the question of consciousness. See, consciousness has this weird dual property that, on the one hand, it’s arguably the most mysterious thing we know about, and on the other hand, not only are we directly aware of it, but in some sense it’s the only thing we’re directly aware of. You know, cogito ergo sum and all that. So, to give an example, I might be mistaken about my shirt being blue – I might be hallucinating or whatever – but I really can’t be mistaken about my perceiving it as blue. (Or if I can, then we get an infinite regress.)…. Many people’s “antirobot animus” is probably a combination of two ingredients: (1) the directly experienced certainty that they’re conscious – that they perceive colors, sounds, positive integers, etc., regardless of whether anyone else does; and (2) the belief that, if they were just a computation, then they could not be conscious in this way…. For people who think this way (as even I do, in certain moods), granting consciousness to a robot seems strangely equivalent to denying that one is conscious oneself. Is there any respectable way out of this dilemma – or in other words, any way out that doesn’t rely on a meatist double standard, with one rule for ourselves and a different rule for robots? My own favorite… is one… advocated by… David Chalmers… a reduction of one mystery to another…. If computers someday become able to emulate humans in every observable respect, then we’ll be compelled to regard them as conscious, for exactly the same reasons we regard other people as conscious. And… we’ll understand… [how] just as well or as poorly as we understand how a bundle of neurons could be conscious. Yes, it’s mysterious, but the one mystery doesn’t seem so different from the other… Aaronson, Scott. 2013. Quantum Computing Since Democritus. Cambridge: Cambridge University Press. .