It was July 5th, 1687. Three days earlier, fed up with Parliament’s attempts to enforce Anglican Protestantism on the Nation via the restrictive Test Act, Roman Catholic King James II of England had disbanded it. A year later William of Orange would invade England, overthrow James, and force him into exile in the “Glorious Revolution” of 1688.
While Europe fought over doctrinal issues of religion, a new kind of science was being born, a science free from the ambiguities of human language, for on that summer’s day Sir Isaac Newton published his long awaited Philosophiæ Naturalis Principia Mathematica or in English simply the Mathematical Principles of Natural Philosophy. The world would never be the same again.
One of the main features of Newton’s new kind of science was the idea that, given the state of a thing at some point in time, one could, in principle, compute both where that thing is going and where it had been. People had long known this to be true of the planets in the heavens of course, but there was no corresponding assumption that this should be true of anything on the Earth. After all, the regularity and predictability of the celestial sphere was a sign of its perfection, which, for both Greeks and the Roman Catholic church, did not apply to earthly things. Such notions had already been disrupted by the discoveries of Kepler and Galileo. Much like the battles going on between the Catholics and protestants, the old science of Aristotle and Plato was battling with the new science of Galileo, Kepler, and Newton. But whereas we still have Catholics and Protestants today who still disagree on the same issues, science has long since moved on from Aristotle. Isaac Newton, despite being deeply religious himself, was largely responsible for the last shrugging off of the old way of thinking about the world in terms of contrasts of earthly and divine. The new way was, in a sense, a merger between the heavenly and the earthly, a realization that all things move according to rules no less perfect that those that govern the planets. And this perfection had, at its core, the idea of predictability.
Even subsequent discoveries such as the apparent randomness of Brownian motion, the jiggling of grains of pollen suspended in fluid, have at their core this Newtonian predictability. For, even though we cannot, in practical terms, predict the random motions, they are, in a sense, not truly random but pseudo-random. Each molecule obeys a precise, Newtonian law with some Maxwellian electromagnetism thrown in. Although the influences of molecules on one another and on a grain of pollen are so numerous and complex to defy anything but a statistical understanding, we know that their motion is exact. The universe at its heart beats according to a predictable and, in a sense, divine drum.
We can feel confident in this even in unpredictable events like the weather, which is subject to the butterfly effect. It is true that complex systems like the Earth’s atmosphere are sensitive to initial conditions meaning that small changes can grow and become large. Thus a butterfly flapping its wings in China can make it rain in New York and there is no way we can follow the chain of events to know that. Yet, we can feel confident that there is a single, precise chain of events and some Divine Being could in principle understand and know. Nothing that happens, by this principle, is random.
That is, nothing is random until we talk about the smallest members of this world, atoms and subatomic particles. It is there that our comfort in the predictability that Newton gave us breaks down. While the laws that govern quantum theory are not at all random (they have a precise Newtonian predictability to them), our measurements are random. Our predictions can only tell us about probabilities not actualities and we do not know what those probabilities actually mean since they come in the form of a mysterious “wavefunction” that moves and evolves as if it were a real thing.
Erwin Schrödinger, whose famous equation is a precise, Newtonian-style law governing the wavefunction, gives this example of the baffling nature of quantum randomness when describing a radioactive decay experiment:
the emerging particle is described … as a spherical wave … that impinges continuously on a surrounding luminescent screen over its full expanse. The screen however does not show a more or less constant uniform surface glow, but rather lights up at one instant at one spot ….
With this simple observation, all of Newton’s philosophy comes crashing down. Imagine if Newton’s laws described not the precise motion of a planet in the heavens but only some expanding probability field and that the only way to know where a planet is is to look up and measure where it actually showed up. In such a world, there would be no rocket science, no trips to the moon and the planets, for we could only know on average where to go. This would, in a sense, be a world ruled by some Divine Trickster, intent on robbing us of any ability to learn the nature of the cosmos we inhabit. Yet, at the quantum level, this is exact what we encounter.
It is easy to understand why Albert Einstein, who not only explained Brownian motion but gave us the first new theory of gravity since Newton, fought so hard against the idea that quantum theory must be random, despite the evidence. He insisted that God does not play dice. It is not that Einstein doubted the evidence of our eyes but instead felt that Schrödinger had simply gotten his equation, with its expanding spherical wave, wrong. It was incomplete for it did not model the hidden motion of the particle, he felt. The particle had to be somewhere, not nowhere, before it hit the screen. We just needed a better theory. It was this conviction that led him to collaborate to publish what has become one of the most famous paradoxes in modern physics: the Einstein-Podolsky-Rosen paradox that told us, once and for all, that something spooky was going on in quantum theory,
Resolving that spookiness, however, turned out to be no mean task. As the evidence of experiments continued to mount, it was impossible to do away with something like Schrödinger’s equation. You could generalize it or you could add to it. You could reformulate it. But the motion of quantum particles was simply not predictable in the way that the motion of a rocket or a planet was. Particles become entangled. They influence one another at a distance requiring some form of instantaneous communication between them. Attempts to remove the randomness that followed Einstein’s intuition that Schrödinger’s equation was incomplete, like Bohmian mechanics, eventually had to add it back in to explain particle creation and annihilation that occurs in particle accelerators. It’s one thing to explain a measurement of a particle you know is there. It’s another to explain measurements of particles when you don’t know they are there.
There are a handful of other ways to remove randomness that don’t go down Einstein’s path. (The “standard” Copenhagen interpretation includes randomness of course, so that doesn’t count.) These are reinterpretations rather than additions to Schrödinger’s. The most famous one is the Many Worlds Interpretation that assumes that the world splits into multiple worlds so that Schrödinger’s equation, far from being incomplete, simply models more worlds than we can see. Many Worlds is easy to understand and so tends to be the only one the general public is familiar with. There are several others that I have written about extensively elsewhere that are more conservative but have their own issues.
Another interpretation that is virtually unknown is the chaotic interpretation. In this interpretation, quantum randomness derives from chaotic fluctuations in quantum fields. Early versions of this theory relied on chaotic fluctuations in quantum particles themselves, but not all quantum particles have chaotic behavior. Therefore, the particles must be in contact with something else that is inherently chaotic. My hypothesis is that this is space and time. Like the pollen suspended in liquid, particles are constantly bombarded with fluctuations in space and time itself and thus appear to behave randomly when in fact everything is as precise and exact as Newton laws. In order to agree with quantum field theory, including particle creation and annihilation, such a theory must introduce a fifth dimension through which the chaotic dynamics occurs. This fifth dimension is much like time in that quantum particles flow through it, but their past and future histories flow through it all at once so that their entire lives fluctuate all at once. This means that the past can change, at least for subatomic particles.
It turns out that, besides eliminating randomness from quantum theory, this interpretation has a major cosmological implication. A dilemma sometimes called the fine tuning problem or “the worst prediction ever made” occurs in particle physics. A naive calculation of quantum vacuum energy, the energy of quantum fluctuations, should have an enormous effect on the universe and this vacuum energy looks exactly like a “cosmological constant” in Einstein’s equations for gravity. Our best theories say this constant is non-zero because we have measured acceleration in the expansion of the universe, which would not occur without some kind of vacuum energy term (constant or otherwise). Particle physics, however, predicts a constant about 120 orders of magnitude (1 with 120 zeros after it) times more than our measurement. There are many ways to resolve this issue (the holographic principle is one of the most compelling) but it turns out that chaotic theory may resolve it on its own.
In the chaotic theory quantum vacuum fluctuations are simply responses to fluctuations in space and time. Thus, every fluctuation in a quantum particle has an equal and opposite reaction in space and time at the tiniest level of the Planck scale. Likewise, every fluctuation in the gravitational field generates a reaction equal and opposite in matter fields. All of this back and forth fluctuating occurs mainly at the smallest scales, however, and as one goes to larger scales you find that the fluctuations average out almost completely to zero. In other words, the gravity of vacuum energy is “screened out” by gravity itself.
Thus, the chaotic interpretation returns quantum theory back to predictability without ad hoc additions like Bohmian mechanics and without strange, inexplicable theories like like Many Worlds. All it requires is a fifth dimension.