The following excerpt comes from Being You: A New Science of Consciousness by Anil Seth.

You can watch our conversation where Seth discusses his book and more here.

Chapter 4: Perceiving from the Inside Out

I open my eyes and a world appears. I’m sitting on the deck of a tumbledown wooden house, high in a cypress forest a few miles north of Santa Cruz, California. It’s early morning. Looking straight out, I can see tall trees still wreathed in the cool ocean fog that rolls in every night, sending the temperature plummeting. I can’t see the ground, so the deck and the trees all seem to be floating together with me in the mist. There are some old plastic chairs— I’m sitting on one—a table, and a tray arranged with coffee and bread. I can hear birdsong, some rustling around in the back—the people I’m staying with—and a distant murmur from something I can’t identify. Not every morning is like this; this is a good morning. I try to persuade myself, not for the first time, that this extraordinary world is a construction of my brain, a kind of “controlled hallucination.”

Whenever we are conscious, we are conscious of something, or of many things. These are the contents of consciousness. To understand how they come about, and what I mean by controlled hallucination, let’s change our perspective. Imagine, for a moment, that you are a brain.

Really try to think about what it’s like up there, sealed inside the bony vault of the skull, trying to figure out what’s out there in the world. There’s no light, no sound, no anything—it’s completely dark and utterly silent. When trying to form perceptions, all the brain has to go on is a constant barrage of electrical signals which are only indirectly related to things out there in the world, whatever they may be. These sensory inputs don’t come with labels attached (“I’m from a cup of coffee”; “I’m from a tree”). They don’t even arrive with labels announcing their modality—whether they are visual, auditory, sensations of touch, or from less familiar modalities such as thermoception (sense of temperature) or proprioception (sense of body position).*

* The familiar but completely wrong idea that humans have only five senses can be traced back to Aristotle’s De Anima—“On the Soul”—written around 350 BC.

How does the brain transform these inherently ambiguous sensory signals into a coherent perceptual world full of objects, people, and places? In part II of Being You, we explore the idea that the brain is a “prediction machine,” and that what we see, hear, and feel is nothing more than the brain’s “best guess” of the causes of its sensory inputs. Following this idea all the way through, we will see that the contents of consciousness are a kind of waking dream—a controlled hallucination—that is both more than and less than whatever the real world really is.

Here’s a commonsense view of perception. Let’s call it the “how things seem” view.

There’s a mind‑independent reality out there, full of objects and people and places that actually have properties like color, shape, texture, and so on. Our senses act as transparent windows onto this world, detecting these objects and their features and conveying this information to the brain, whereupon complex neuronal processes read it out to form perceptions. A coffee cup out there in the world leads to a perception of a coffee cup generated within the brain. As to who or what is doing the perceiving—well, that’s the “self,” isn’t it, the “I behind the eyes,” one might say, the recipient of wave upon wave of sensory data, which uses its perceptual readouts to guide behavior, to decide what to do next. There’s a cup of coffee over there. I perceive it and I pick it up. I sense, I think, and then I act.

This is an appealing picture. Patterns of thinking established over decades, maybe centuries, have accustomed us to the idea that the brain is some kind of computer perched inside the skull, processing sensory information to build an inner picture of the outside world for the benefit of the self. This picture is so familiar that it can be difficult to conceive of any reasonable alternative.

***

LUDWIG WITTGENSTEIN: “Why do people say that it was natural to think that the sun went round the Earth rather than that the Earth turned on its axis?”

ELIZABETH A NSCOMBE: “I suppose, because it looked as if the sun went round the Earth.”

LUDWIG WITTGENSTEIN: “Well, what would it have looked like if it had looked as if the Earth turned on its axis?”

In this delightful exchange between Wittgenstein and his fellow philosopher (and biographer) Elizabeth Anscombe, the legendary Austrian thinker uses the Copernican revolution to illustrate the point that how things seem is not necessarily how they are. Although it seems as though the sun goes around the Earth, it is of course the Earth rotating around its own axis that gives us night and day, and it is the sun, not the Earth, that sits at the center of the solar system. Nothing new here, you might think, and you’d be right. But Wittgenstein was driving at something deeper. His real message for Anscombe was that even with a greater understanding of how things actually are, at some level things still appear the same way they always did. The sun rises in the east and sets in the west, same as always.

As with the solar system, so with perception. I open my eyes and it seems as though there’s a real world out there. Today, I’m at home in Brighton. There are no cypress trees like there were in Santa Cruz, just the usual scatter of objects on my desk, a red chair in the corner, and beyond the window a totter of chimney pots. These objects seem to have specific shapes and colors, and for the ones closer at hand, smells and textures too. This is how things seem.

Although it may seem as though my senses provide transparent windows onto a mind‑independent reality, and that perception is a process of “reading out” sensory data, what’s really going on is—I believe—quite different. Perceptions do not come from the bottom up or the outside in, they come primarily from the top down, or the inside out. What we experience is built from the brain’s predictions, or “best guesses,” about the causes of sensory signals. As with the Copernican revolution, this top‑down view of perception remains consistent with much of the existing evidence, leaving unchanged many aspects of how things seem, while at the same time changing everything.

***

In the late nineteenth century, among a string of influential contributions, German physicist and physiologist Hermann von Helmholtz proposed the idea of perception as a process of “unconscious inference.” The contents of perception, he argued, are not given by sensory signals themselves but have to be inferred by combining these signals with the brain’s expectations or beliefs about their causes. In calling this process “unconscious,” Helmholtz understood that we are not aware of the mechanisms by which perceptual inferences happen, only of the results. Perceptual judgments—his “unconscious inferences”—keep track of their causes in the world by continually and actively updating perceptual best guesses as new sensory data arrive. Helmholtz saw himself as providing a scientific version of Kant’s insight that perception cannot allow us to know things in the world directly—that we can infer only that things are there, behind the sensory veil.

***

My own take on Helmholtz’s enduring idea, and on its contemporary incarnations, is best captured by the notion of perception as controlled hallucination, a phrase I first heard from the British psychologist Chris Frith many years ago. The essential ingredients of the controlled hallucination view, as I think of it, are as follows.

First, the brain is constantly making predictions about the causes of its sensory signals, predictions which cascade in a top‑down direction through the brain’s perceptual hierarchies (the gray arrows in the image below). If you happen to be looking at a coffee cup, your visual cortex will be formulating predictions about the causes of the sensory signals that originate from this coffee cup.

Second, sensory signals—which stream into the brain from the bottom up, or outside in—keep these perceptual predictions tied in useful ways to their causes: in this case, a coffee cup. These signals serve as prediction errors registering the difference between what the brain expects and what it gets at every level of processing. By adjusting top‑down predictions so as to suppress bottom‑up prediction errors, the brain’s perceptual best guesses maintain their grip on their causes in the world. In this view, perception happens through a continual process of prediction error minimization.

The third and most important ingredient in the controlled hallucination view is the claim that perceptual experience—in this case the subjective experience of “seeing a coffee cup”—is determined by the content of the (top‑down) predictions, and not by the (bottom‑up) sensory signals. We never experience sensory signals themselves, we only ever experience interpretations of them.

Mix these ingredients together and we’ve cooked up a Copernican inversion for how to think about perception. It seems as though the world is revealed directly to our conscious minds through our sensory organs. With this mindset, it is natural to think of perception as a process of bottom‑up feature detection—a “reading” of the world around us. But what we actually perceive is a top‑down, inside‑out neuronal fantasy that is reined in by reality, not a transparent window onto whatever that reality may be.

And—to channel Wittgenstein once more—what would it seem like, if it seemed as if perception was a top‑down best guess? Well, just as the sun still rises in the east and sets in the west, if it seemed as if perception was a controlled hallucination, the coffee cup on the table—the entirety of anyone’s perceptual experience—would still seem the same way it always did, and always will.

BUY THE BOOK

Excerpted from Being You by Anil Seth, published by Dutton, an imprint of the Penguin Publishing Group, a division of Penguin Random House, LLC. Copyright © 2021 by Anil Seth.