The World According to Quantum Mechanics: Why the Laws of Physics Make Perfect Sense After All

We shall not cease from exploration
And the end of all our exploring
Will be to arrive where we started
And know the place for the first time.
[T.S. Eliot, Little Gidding]

While still in high school, I learned that the tides act as a brake on the Earth’s rotation, gradually slowing it down, and that the angular momentum lost by the rotating Earth is transferred to the Moon, causing it to slowly spiral outwards, away from Earth. I still vividly remember my puzzlement. How, by what mechanism or process, did angular momentum get transferred from Earth to the Moon? Just so Newton’s contemporaries must have wondered at his theory of gravity. Newton’s response is well known:

I have not been able to discover the cause of those properties of gravity from phaenomena, and I frame no hypotheses…. to us it is enough, that gravity does really exist, and act according to the laws which we have explained, and abundantly serves to account for all the motions of the celestial bodies, and of our sea.

In Newton’s theory, gravitational effects were simultaneous with their causes. The time-delay between causes and effects in classical electrodynamics and in Einstein’s theory of gravity made it seem possible for a while to explain “how Nature does it.” One only had to transmogrify the algorithms that served to calculate the effects of given causes into physical processes by which causes produce their effects. This is how the electromagnetic field—a calculational tool—came to be thought of as a physical entity in its own right, which is locally acted upon by charges, which locally acts on charges, and which mediates the action of charges on charges by locally acting on itself.

Today this sleight of hand no longer works. While classical states are algorithms that assign trivial probabilities—either 0 or 1—to measurement outcomes (which is why they can be reinterpreted as collections of possessed properties and described without reference to “measurement”), quantum states are algorithms that assign probabilities between 0 and 1 (which is why they cannot be so described). And while the classical laws correlate measurement outcomes deterministically (which is why they can be interpreted in causal terms and thus as descriptive of physical processes), the quantum-mechanical laws correlate measurement outcomes probabilistically (which is why they cannot be so interpreted). In at least one respect, therefore, physics is back to where it was in Newton’s time—and this with a vengeance. According to Dennis Dieks, Professor of the Foundations and Philosophy of the Natural Sciences at Utrecht University and Editor of Studies in History and Philosophy of Modern Physics,

the outcome of foundational work in the last couple of decades has been that interpretations which try to accommodate classical intuitions are impossible, on the grounds that theories that incorporate such intuitions necessarily lead to empirical predictions which are at variance with the quantum mechanical predictions.

But, seriously, how could anyone have hoped to get away for good with passing off computational tools—mathematical symbols or equations—as physical entities or processes? Was it the hubristic desire to feel “potentially omniscient”—capable in principle of knowing the furniture of the universe and the laws by which this is governed?

The question that will be centrally pursued in this book is: what does it take to have stable objects that “occupy space” while being composed of objects that do not “occupy space”? (The latter are commonly referred to as “pointlike.”) And part of the answer at which we shall arrive is: quantum mechanics.

As said, quantum states are algorithms that assign probabilities between 0 and 1. Think of them as computing machines: you enter (i) the actual outcome(s) and time(s) of one or several measurements, as well as (ii) the possible outcomes and the time of a subsequent measurement—and out pop the probabilities of these outcomes. Even though the time dependence of a quantum state is thus clearly a dependence on the times of measurements, it is generally interpreted—even in textbooks that strive to remain metaphysically uncommitted—as a dependence on “time itself,” and thus as the time dependence of something that exists at every moment of time and evolves from earlier to later times. Hence the mother of all quantum-theoretical pseudo-questions: why does a quantum state have (or appear to have) two modes of evolution—continuous and predictable between measurements, discontinuous and unpredictable whenever a measurement is made?

An approach that rejects the very notion of quantum state evolution runs the risk of being dismissed as an ontologically sterile instrumentalism. Yet it is this notion, more than any other, that blocks our view of the ontological implications of quantum mechanics. One of these implications is that the spatiotemporal differentiation of the physical world is incomplete; it does not “go all the way down.” The notion that quantum states evolve, on the other hand, implies that it does “go all the way down.” This is not simply a case of one word against another, for the incomplete spatiotemporal differentiation of the physical world follows from the manner in which quantum mechanics assigns probabilities, which is testable, whereas the complete spatiotemporal differentiation of the physical world follows from an assumption about what is the case between measurements, and such an assumption is “not even wrong” in Wolfgang Pauli’s famous phrase, inasmuch as it is neither verifiable nor falsifiable.

For at least twenty-five centuries, theorists—from metaphysicians to natural philosophers to physicists and philosophers of science—have tried to model reality from the bottom up, starting with an ultimate multiplicity and using concepts of composition and interaction as their basic explanatory tools. If the spatiotemporal differentiation of the physical world is incomplete, then the attempt to understand the world from the bottom up—whether on the basis of an intrinsically and completely differentiated space or spacetime, out of locally instantiated physical properties, or by aggregation, out of a multitude of individual substances—is doomed to failure. What quantum mechanics is trying to tell us is that reality is structured from the top down.

This textbook is based on a philosophically oriented course of contemporary physics I have been teaching for the last ten years at the Sri Aurobindo International Centre of Education (SAICE) in Puducherry (formerly Pondicherry), India. This non-compulsory course is open to higher secondary (standards 10–12) and undergraduate students, including students with negligible prior exposure to classical physics.1 I wish to thank the SAICE for the opportunity to teach this experimental course in “quantum philosophy” and my students—the “guinea pigs”—for their valuable feedback.2

— August 15, 2010

Quantum mechanics has been compared to a wolf in sheep’s clothing. While the theory’s formalism can be written down on a napkin, attempts to interpret it fill entire libraries. In this book we attempt to make sense of quantum mechanics in a way that steers clear of two common errors, which jointly account for most of the stacks in those libraries. The vastly more pervasive of the two errors, Ψ-ontology, has its roots in “the bizarre view that we, at this point in history, are in possession of the basic forms of understanding needed to comprehend absolutely anything”, a view that appears to be particularly de rigueur in the philosophy of science. It leads at once to “the great scandal of physics”, “the disaster of objectification”, which consists in the insolubility of the “BIG” measurement problem —the problem of explaining how measurement outcomes arise dynamically. The other, less common, error is that made by the so-called anti-realists, who content themselves with looking upon the theory as a tool for making predictions. What appears to have escaped everyone’s notice is the possibility of a coherent conception of reality that does not fall prey to “our habit of inappropriately reifying our successful abstractions”, a conception that explains why the formal apparatus of quantum mechanics is a probability calculus, and why the events to which (and on the basis of which) it serves to assign probabilities, are possible measurement outcomes.

This book has been written with three kinds of readers in mind. Students may find it to be an invaluable supplement to standard textbooks. While quantum physics makes use of many of the concepts that students are familiar with from classical physics, the manner in which these concepts enter the quantum theory is rarely clarified sufficiently. How, for instance, did momentum become a self-adjoint operator acting on vectors in a Hilbert space? Such fertile sources of perplexity are at once disposed of by the insight that the formal apparatus of the theory is a probability calculus. As one reviewer of the first edition put it:

The way this book covers the two slit experiment everything falls into place and makes perfect sense. There is no wave particle dualism, just the naked necessity of a probabilistic regime. It is so simple. Painfully obvious. Easy to grasp with just a minimum of mathematical rigor. It boggles the mind that QM has not been understood this way from the get go. This feels like 20/20 hindsight writ large…. If you’ve been trying to make sense of QM you will hate this book. It’ll make you feel stupid for not having been able to see this all along.

Teachers may appreciate the resulting disentanglement of the theory’s formalism from its metaphysical issues. My co-author Manu Jaiswal is a case in point. Encouraged by the first edition, he began teaching, with remarkable success,3 what had previously appeared to him an abstruse subject.
And finally, the metaphysically interested general reader may welcome this book as the missing link between the proliferating popular literature on quantum mechanics and the equally proliferating academic literature. For them, the requisite mathematical tools are introduced, partly along the way and partly in an Appendix, to the point that all theoretical concepts can be adequately grasped. In doing so, we (Manu and I) tried to adhere to a principle known as “Einstein’s razor,” according to which everything should be made as simple as possible, but no simpler.

The book is divided into three parts. After a short introduction to probability, Part 1 (“Overview”) follows two routes to the Schrödinger equation—the historical route and Feynman’s path–integral approach. On the second route we stop once for a concise introduction to the special theory of relativity. Two sections have been added, one on tunneling and one discussing a quantum bouncing ball.

Part 2 (“A Closer Look”) begins by deriving the theory’s formal apparatus from the obvious existence of “ordinary” objects—stable objects that “occupy space” while being composed of objects that do not “occupy space” (which are commonly thought of as pointlike). We come to understand the need to upgrade from the trivial probability calculus known as classical mechanics to the nontrivial probability calculus known as quantum mechanics, and how to do so. The next two chapters are concerned with what happens if the fuzziness that “fluffs out” matter is ignored. (What happens is that the quantum-mechanical correlation laws degenerate into the dynamical laws of classical physics.) In Chapter 10 the discussion of quantum mechanics resumes along more traditional lines, with new sections on Ehrenfest’s relations, conservation of probability, and the uncertainty relation for non-commuting operators. Chapter 11, on spin 1/2 systems, has a new section on the Stern-Gerlach experiment as an example of an unsharp observable, in which POVMs are introduced. This is followed by a newly added chapter on angular momentum and the hydrogen atom. The chapter on composite systems has been split into two, with new sections on EPR, Kochen and Specker, the respective inequalities of Klyachko and CHSH, and the apparent conflict between quantum mechanics and relativity. The two remaining chapters of Part 2 have survived largely unchanged.

The most significant changes, accounting for the bulk of the nearly 200 pages added, occur in Part 3 (“Making Sense”). Chapter 17 concerns how the founders—in particular, de Broglie, Schrödinger, Heisenberg, and Bohr—sought to make sense of the new theory. The key concept there, introduced by Schrödinger, is that of objectivation, which is both counterpoint and answer to the “disaster of objectification.” Whereas objectification would (if it did) occur in a pre-existent external world, the term “objectivation” refers to the representation of a mentally constructed internal world as a shared objective world. This concept goes back to Kant —easily the most important philosopher of the modern era—who insisted that “we cannot understand anything except that which has something corresponding to our words in intuition”. Schrödinger, Heisenberg, and Bohr would all have agreed with von Weizsäcker—a student of Bohr and Heisenberg—that “those who really want to understand contemporary physics—i.e., not only to apply physics in practice but also to make it transparent—will find it useful, even indispensable at a certain stage, to think through Kant’s theory of science”. As we are doing in this chapter.

Chapter 18 discusses attempts—by von Neumann, London and Bauer, Wigner, and Schrödinger—to come to terms with the role that consciousness plays in our accounts of the physical world. A derivation of quantum mechanics by the transcendental method introduced by Kant is outlined, and the notion that quantum-mechanical indeterminism provides the physical basis of free will is briefly discussed.

Chapter 19 is devoted to QBism, the “new kid on the block” of interpretations of quantum mechanics, which Mermin thinks “is as big a break with 20th century ways of thinking about science as Cubism was with 19th century ways of thinking about art.” The importance of this interpretation is that it roots the definiteness of measurement outcomes as firmly as none other in the personal experiences of each user (of quantum mechanics) or agent (in the quantum world).

The subject of Chap. 20 is Ψ-ontology in its two dominant forms, Everettian quantum mechanics and the de Broglie/Bohm theory, and Chap. 21 deals with environmental decoherence. This makes up for a deficiency of older textbooks (including our first edition) that was pointed out by Tegmark: “If you are considering a quantum textbook that does not mention `Everett’ and `decoherence’ in the index, I recommend buying a more modern one.”

The presentation of our own interpretation begins in Chap. 22, with a statement of the interpretive principle that replaces the eigenvalue-eigenstate link, which is regarded by many as an essential ingredient of the standard formulation of quantum mechanics. Our interpretive principle implies that what is incomplete is not quantum mechanics, as EPR had argued, but the spatiotemporal differentiation of the physical world. This allows us to establish the theory’s semantic consistency (or the should-be unsurprising fact that the quantum-mechanical correlation laws are consistent with the existence of their correlata). Also implied by our interpretive principle is the numerical identity of all fundamental particles in existence, which is the subject of Chap. 23.

In Chap. 24 we come to the heart of our interpretation, the manifestation of the world. Put in the proverbial nutshell: by entering into reflexive spatial relations, Being—that which all existing fundamental particles identically are, which in the first edition was called Ultimate Reality (UR)—creates matter, space, and form, for space is the totality of existing spatial relations, forms resolve themselves into particular sets of spatial relations, and matter is the apparent multitude of the corresponding relata—”apparent” because the relations are reflexive. We come to understand the rationale for the all-important distinction, made by the founders and all but criminally neglected by modern interpreters, between a classical or macroscopic domain and a quantum or microscopic domain. This distinction amounts to a recognition of the difference between the manifested world and its manifestation. Because the latter consists in the gradual realization of distinguishable objects and distinguishable regions of space, the question arises as to how the intermediate stages are to be described, and the answer is that whatever is not completely distinguishable can only be described by assigning probabilities to what is completely distinguishable. This explains why the general theoretical framework of contemporary physics is a calculus of correlations between measurement outcomes. Particles, atoms, and molecules, rather than playing the roles of constituent parts, are instrumental in the process of manifestation, and what is instrumental in the manifestation of the world can only be described in terms of correlations between events that happen (or could happen) in the manifested world.

Chapter 25 summarizes our derivation of the mathematical formalism of quantum mechanics from the existence of “ordinary” objects, and goes on to argue that even the classical (long-range) forces, the nuclear (short-range) forces, and general relativity are preconditions for the possibility of a world that conforms to the classical narrative mode—a world whose properties allow themselves to be sorted into causally evolving bundles (i.e., re-identifiable substances).

In Chap. 26 we turn to the second great theoretical challenge of our time, besides making sense of quantum mechanics, namely the challenge of making sense of the fact that the world appears to exist twice—once for us, in human consciousness, and once again by itself, independently of us. The conclusion that forces itself on us is that there is no such thing as a self-existent external world, and that the “hard problem of consciousness” is as insoluble a pseudo-problem as the “BIG” measurement problem, both of which presuppose such a world. The world is not simply manifested; it is manifested to us. Or else, Being does not simply manifest the world; it manifests the world to itself. It is not only a single substance by which the world exists, but also a single consciousness for which the world exists. How we, at this evolutionary juncture, are related to that consciousness, is the subject of the final chapter, in which we also come to understand why “ordinary” objects (having spatial extent) are “composed” of finite numbers of objects lacking spatial extent, and how Being enters into reflexive relations (and thereby manifests both matter and space).

— February 21, 2018