In his new book, Something Deeply Hidden: Quantum Worlds and the Emergence of Spacetime, theoretical physicist and long-time Closer to Truth contributor Sean Carroll shows for the first time that facing up to the essential puzzle of quantum mechanics utterly transforms how we think about space and time. In this exclusive excerpt provided to Closer to Truth, Carroll explains that today’s physicists have largely stopped trying to understand quantum theory… and that reprioritizing this pursuit is of crucial importance.
Prologue: Don’t Be Afraid
You don’t need a PhD in theoretical physics to be afraid of quantum mechanics. But it doesn’t hurt.
That might seem strange. Quantum mechanics is our best theory of the microscopic world. It describes how atoms and particles interact through the forces of nature, makes incredibly precise experimental predictions. To be sure, quantum mechanics has something of a reputation for being difficult, mysterious, just this side of magic. But professional physicists, of all people, should be relatively comfortable with a theory like that. They are constantly doing intricate calculations involving quantum phenomena, and building giant machines dedicated to testing the resulting predictions. Surely we’re not suggesting that physicists have been faking it all this time?
They haven’t been faking, but they haven’t exactly been honest with themselves either. On the one hand, quantum mechanics is the heart and soul of modern physics. Astrophysicists, particle physicists, atomic physicists, laser physicists—everyone uses quantum mechanics all the time, and they’re very good at it. It’s not just a matter of esoteric research. Quantum mechanics is ubiquitous in modern technology. Semiconductors, transistors, microchips, lasers, and computer memory all rely on quantum mechanics to function. For that matter, quantum mechanics is necessary to make sense of the most basic features of the world around us. Essentially all of chemistry is a matter of applied quantum mechanics. To understand how the sun shines, or why tables are solid, you need quantum mechanics.
Imagine closing your eyes. Hopefully things look pretty dark. You might think that makes sense, because no light is coming in. But that’s not quite right; infrared light, with a slightly longer wavelength than visible light, is being emitted all the time by any warm object, and that includes your own body. If our eyes were as sensitive to infrared light as they are to visible light, we would be blinded even when our lids were closed, from all the light emitted by our eyeballs themselves. But the rods and cones that act as light receptors in our eyes are cleverly sensitive to visible light, not infrared. How do they manage that? Ultimately, the answer comes down to quantum mechanics.
Quantum mechanics isn’t magic. It is the deepest, most comprehensive view of reality we have. As far as we currently know, quantum mechanics isn’t just an approximation of the truth; it is the truth. That’s subject to change in the face of unexpected experimental results, but we’ve seen no hints of any such surprises thus far. The development of quantum mechanics in the early years of the twentieth century, involving names like Planck, Einstein, Bohr, Heisenberg, Schrodinger, and Dirac, left us by 1927 with a mature understanding that is surely one of the greatest intellectual accomplishments in human history. We have every reason to be proud.
On the other hand, in the memorable words of Richard Feynman, “I think I can safely say that nobody understands quantum mechanics.” We use quantum mechanics to design new technologies and predict the outcomes of experiments. But honest physicists admit that we don’t truly understand quantum mechanics. We have a recipe that we can safely apply in certain prescribed situations, and which returns mind-bogglingly precise predictions that have been triumphantly vindicated by the data. But if you want to dig deeper and ask what is really going on, we simply don’t know. Physicists tend to treat quantum mechanics like a mindless robot they rely on to perform certain tasks, not as a beloved friend they care about on a personal level.
This attitude among the professionals seeps into how quantum mechanics gets explained to the wider world. What we would like to do is to present a fully formed picture of Nature, but we can’t quite do that, since physicists don’t agree about what quantum mechanics actually says. Instead, popular treatments tend to emphasize that quantum mechanics is mysterious, baffling, impossible to understand. That message goes against the basic principles that science stands for, which include the idea that the world is fundamentally intelligible. We have something of a mental block when it comes to quantum mechanics, and we need a bit of quantum therapy to help get past it.
Sean Carroll – Events in quantum mechanics | Watch more interviews
° ° °
When we teach quantum mechanics to students, they are taught a list of rules. Some of the rules are of a familiar type: there’s a mathematical description of quantum systems, plus an explanation of how such systems evolve over time. But then there are a bunch of extra rules that have no analogue in any other theory of physics. These extra rules tell us what happens when we observe a quantum system, and that behavior is completely different from how the system behaves when we’re not observing it. What in the world is going on with that?
There are basically two options. One is that the story we’ve been telling our students is woefully incomplete, and in order for quantum mechanics to qualify as a sensible theory we need to understand what a “measurement” or “observation” is, and why it seems so different from what the system does otherwise. The other option is that quantum mechanics represents a violent break from the way we have always thought about physics before, shifting from a view where the world exists objectively and independently of how we perceive it, to one where the act of observation is somehow fundamental to the nature of reality.
In either case, the textbooks should by all rights spend time exploring these options, and admit that even though quantum mechanics is extremely successful, we can’t claim to be finished developing it just yet. They don’t. For the most part, they pass over this issue in silence, preferring to stay in the physicist’s comfort zone of writing down equations and challenging students to solve them.
That’s embarrassing. And it gets worse.
You might think, given this situation, that the quest to understand quantum mechanics would be the single biggest goal in all of physics. Millions of dollars of grant money would flow to researchers in quantum foundations, the brightest minds would flock to the problem, and the most important insights would be rewarded with prizes and prestige. Universities would compete to hire the leading figures in the area, dangling superstar salaries to lure them away from rival institutions.
Sadly, no. Not only is the quest to make sense of quantum mechanics not considered a high- status specialty within modern physics; in many quarters it’s considered barely respectable at all, if not actively disparaged. Most physics departments have nobody working on the problem, and those who choose to do so are looked upon with suspicion. (Recently while writing a grant proposal, I was advised to concentrate on describing my work in gravitation and cosmology, which is considered legitimate, and remain silent about my work on the foundations of quantum mechanics, as that would make me appear less serious.) There have been important steps forward over the last ninety years, but they have typically been made by headstrong individuals who thought the problems were important despite what all of their colleagues told them, or by young students who didn’t know any better and later left the field entirely.
In one of Aesop’s fables, a fox sees a juicy bunch of grapes and leaps to reach it, but can’t quite jump high enough. In frustration he declares that the grapes were probably sour, and he never really wanted them anyway. The fox represents “physicists,” and the grapes are “understanding quantum mechanics.” Many researchers have decided that understanding how nature really works was never really important; all that matters is the ability to make particular predictions.
Scientists are trained to value tangible results, whether they are exciting experimental findings or quantitative theoretical models. The idea of working to understand a theory we already have, even if that effort might not lead to any specific new technologies or predictions, can be a tough sell. The underlying tension was illustrated in the TV show The Wire, where a group of hardworking detectives labored for months to meticulously gather evidence that would build a case against a powerful drug ring. Their bosses, meanwhile, had no patience for such incremental frivolity. They just wanted to see drugs on the table for their next press conference, and encouraged the police to bang heads and make splashy arrests. Funding agencies and hiring committees are like those bosses. In a world where all the incentives push us toward concrete, quantifiable outcomes, less pressing big-picture concerns can be pushed aside as we race toward the next immediate goal.
Sean Carroll – Do multiple universes exist? | Watch more interviews
° ° °
This book has three main messages. The first is that quantum mechanics should be understandable—even if we’re not there yet—and achieving such understanding should be a high-priority goal of modern science. Quantum mechanics is unique among physical theories in drawing an apparent distinction between what we see and what really is. That poses a special challenge to the minds of scientists (and everyone else), who are used to thinking about what we see as unproblematically “real,” and working to explain things accordingly. But this challenge isn’t insuperable, and if we free our minds from certain old-fashioned and intuitive ways of thinking, we find that quantum mechanics isn’t hopelessly mystical or inexplicable. It’s just physics.
The second message is that we have made real progress toward understanding. I will focus on the approach I feel is clearly the most promising route, the Everett or Many-Worlds formulation of quantum mechanics. Many-Worlds has been enthusiastically embraced by many physicists, but it has a sketchy reputation among people who are put off by a proliferation of other realities containing copies of themselves. If you are one of those people, I want to at least convince you that Many-Worlds is the purest way of making sense of quantum mechanics— it’s where we end up if we just follow the path of least resistance in taking quantum phenomena seriously. In particular, the multiple worlds are predictions of the formalism that is already in place, not something added in by hand. But Many-Worlds isn’t the only respectable approach, and we will mention some of its main competitors. (I will endeavor to be fair, if not necessarily balanced.) The important thing is that the various approaches are all well-constructed scientific theories, with potentially different experimental ramifications, not just woolly-headed “interpretations” to be debated over cognac and cigars after we’re finished doing real work.
The third message is that all this matters, and not just for the integrity of science. The success to date of the existing adequate-but-not-perfectly-coherent framework of quantum mechanics shouldn’t blind us to the fact that there are circumstances under which such an approach simply isn’t up to the task. In particular, when we turn to understanding the nature of spacetime itself, and the origin and ultimate fate of the entire universe, the foundations of quantum mechanics are absolutely crucial.
I’ll introduce some new, exciting, and admittedly tentative proposals that draw provocative connections between quantum entanglement and how spacetime bends and curves— the phenomenon you and I know as “gravity.” For many years now, the search for a complete and compelling quantum theory of gravity has been recognized as an important scientific goal (prestige, prizes, stealing away faculty, and all that). It may be that the secret is not to start with gravity and “quantize” it, but to dig deeply into quantum mechanics itself, and find that gravity was lurking there all along.
We don’t know for sure. That’s the excitement and anxiety of cutting-edge research. But the time has come to take the fundamental nature of reality seriously, and that means confronting quantum mechanics head-on.
Something Deeply Hidden: Quantum Worlds and the Emergence of Spacetime
By Sean Carroll