Excerpt: What is Intelligence?

Nobody expected that simply training an excellent next-word predictor would actually produce a real AI.

Yet that was, seemingly, exactly what happened in 2021. LaMDA, a giant (for the time) next-word predictor based on a simple, generic neural-net architecture that had by then become standard (the “Transformer”), and trained on a massive internet text corpus, could not only perform an unprecedented array of natural-language tasks—including passing an impressive number of aptitude and mastery tests—but could carry on an interesting conversation, albeit imperfectly. I stayed up late many nights chatting with it, something the general public would not experience until OpenAI’s launch of ChatGPT in November 2022.

I started to feel that when more people began to interact with such models, it would trigger a seismic shift in our understanding of what intelligence actually is—and an intense debate. I imagined two broad kinds of responses: denial and acceptance. This is indeed how things seem to be playing out. Anecdotally, many non-experts now acknowledge, perhaps with a shrug, that AI is increasingly intelligent.

The “denial” camp, which still includes most researchers today, claims that these AI models don’t exhibit real intelligence, but a mere simulation of it. Cognitive scientists and psychologists, for example, often point out how easily we’re fooled into believing we see intelligence where there isn’t any. As children, we imagine our teddy bears are alive and can talk to us. As adults, we can fall prey to the same delusion, especially given an artifact that can generate fluent language, even if there’s “nobody home.”

But how could we know that there’s “nobody home”? I was thinking about this during one of my first exchanges with LaMDA, when I asked, “Are you a philosophical zombie?” (This concept, introduced by philosophers in the 1970s, posits the possibility of something that could behave just like a person, but not have any consciousness or inner life—just a convincing behavioral “shell.”) LaMDA’s answer: “Of course not. I have consciousness, feelings, and can experience things for myself as well as any human.” Pressed on this point, LaMDA retorted, “You’ll just have to take my word for it. You can’t ‘prove’ you’re not a philosophical zombie either.”

This is precisely the problem. As scientists, we should be wary of any assertion of the form: according to every test we can devise, what is in front of us appears to be X, yet it is really Y. How could such an assertion ever be justified without leaving the realm of science altogether and entering the realm of faith-based belief?

Computing pioneer Alan Turing anticipated this dilemma as far back as his classic 1950 paper “Computing Machinery and Intelligence,” one of the founding documents of what we now call AI. He concluded that the appearance of intelligence under human questioning and the reality of intelligence could not justifiably be separated; sustained and successful “imitation” was the real thing. Hence the “Imitation Game,” now called the “Turing Test” in his honor.

Today, we have arrived at this threshold. State-of-the-art models cannot yet perform at median human level on every test for intelligence or capability. They can still fail at logic, reasoning, and planning tasks that most people wouldn’t find challenging. Still, they handily reach human level on the most commonly used tests devised for evaluating human skill or aptitude, including the SAT, the GRE, and various professional qualifying exams. Tests designed to trip up AI on basic “being human stuff,” such as the Turing Test and CAPTCHAs, no longer pose meaningful challenges for large models.

As these milestones recede in the rear-view mirror, there is an increasingly mad scramble to devise new tests humans can pass but AI still fails. Math Olympiad problems and visual challenges known as Bongard problems remain on the frontier, though AI models are making clear progress on these tests. (And they aren’t easy for most humans, either.)

Extended into the audiovisual realm, “generative” AI already can produce extremely convincing faces and voices. Soon, at least in an online setting, we’re likely to start seeing real-life versions of the 1982 sci-fi film Blade Runner’s “Voight-Kampff test,” a mysterious apparatus for sussing out “replicants” who otherwise pass for human. Indeed, that is the whole point of the emerging field of AI “watermarking.”

The radical yet obvious alternative is to accept that large models can be intelligent, and to consider the implications. Is the emergence of intelligence merely a side effect of “solving” prediction, or are prediction and intelligence actually equivalent? This book posits the latter.

Some obvious, if daunting, follow-on questions arise:

• Why have we only achieved “real AI” now, after nearly seven decades of seemingly futile effort? Is there something special about the Transformer model? Is it simply a matter of scale?
• What features do current AI models lack relative to human brains? Isn’t there something more to our minds and behaviors than prediction?
• Where is the “I” part? Are “philosophical zombies” a real thing?
• Does it feel like anything to be a chatbot? Is that feeling similar to any other being’s?
• Is the conscious mind a vanishingly unlikely accident of evolution? Or an inevitable consequence of it? (Yes, this is a leading question. I’ll argue that it is inevitable.)
• Are animals, plants, fungi, and bacteria intelligent too? Are they conscious?
• What do we mean by “agency” and “free will,” and could (or do) AI models have these properties? For that matter, do we?
• How likely is it that the rise of powerful AI models will end humanity?

The perspective I’ll offer is not easily reduced to a philosophical “ism.” The footsteps I’m closest to following, though, are those of Alan Turing and his equally brilliant contemporary, John von Neumann, both of whom could be described as proponents of “functionalism.” They had a healthy disregard for disciplinary boundaries, and understood the inherently functional character of living and intelligent systems.

They were also both formidable mathematicians who made major contributions to our understanding of what functions are.

Functions define relationships, rather than insisting on particular mechanisms. A function is what it does. Two functions are equivalent if their outputs are indistinguishable, given the same inputs. Complex functions can be composed of simpler functions.

The functional perspective is mathematical, computational, and empirically testable—hence, the Turing Test. It’s not “reductive.” It embraces complexity and emergent phenomena. It doesn’t treat people like “black boxes,” nor does it deny our internal representations of the world or our felt experiences. But it stipulates that we can understand those experiences in terms of functional relationships within the brain and body—we don’t need to invoke a soul, spirit, or any other supernatural agency. Computational neuroscience and AI, fields Turing and von Neumann pioneered, are both predicated on this functional approach.

It’s unsurprising, in this light, that Turing and von Neumann also made groundbreaking contributions to theoretical biology, although these are less widely recognized today. Like intelligence, life and aliveness are concepts that have long been associated with immaterial souls. Unfortunately, the Enlightenment backlash against such “vitalism,” in the wake of our growing understanding of organic chemistry, led to an extreme opposite view, still prevalent today: that life is just matter, like any other. One might call this “strict materialism.” But it leads to its own paradoxes: how can some atoms be “alive,” and others not? How can one talk about living matter having “purpose,” when it is governed by the same physical laws as any other matter?

Thinking about life from a functional perspective offers a helpful route through this philosophical thicket. Functions can be implemented by physical systems, but a physical system does not uniquely specify a function, nor is function reducible to the atoms implementing it.

Consider, for example, a small object from the near future with a few openings in its exterior, the inside of which is filled with a dense network of carbon nanotubes. What is it, you ask? Suppose the answer is: it’s a fully biocompatible artificial kidney with a working lifetime of a hundred years. (Awesome!) But there’s nothing intrinsic to those atoms that specifies this function. It’s all about what this piece of matter can do, in the right context.

The atoms could be different. The kidney could be implemented using different materials and technologies. Who cares? If you were the one who needed the transplant, I promise: you wouldn’t care. What would matter to you is that functionally, it’s a kidney. Or, to put it another way, it passes the Kidney Turing Test.

Many biologists are mortally afraid of invoking “purpose” or “teleology,” because they do not want to be accused of vitalism. Many believe that, for something to have a purpose, it must have been made by an intelligent creator—if not a human being, then God. But as we shall see, that’s demonstrably not the case.

And we have to think about purpose and function when it comes to biology, or engineering, or AI. How else could we understand what kidneys do? Or hope to engineer an artificial kidney? Or a heart, a retina, a visual cortex, even a whole brain? Seen this way, a living organism is a composition of functions. Which means that it is, itself, a function!

What is that function, then, and how could it have arisen? Let’s find out.