General Relativity: How Einstein’s wrong ideas led to his greatest success
Einstein spent his life misunderstanding his own theory.
In 1915 Albert Einstein presented his magnum opus, his theory of “General Relativity” to the world. The theory would go on to be one of the pillars of modern physics along with the standard model of quantum field theory.
That the theory was so successful is surprising because Einstein never really understood what principles it was based on and changed his mind about them during his life.
Principles like general covariance, Mach’s principle, and the equivalence principle that he relied on were all imprecise axioms that later physicists would reject completely or redefine as something other than he intended. Some remain as teaching tools or ways of explaining to the general public a more precise axiom, but you won’t find these in any derivation of the theory.
It is as if Euclid came up with his geometry and later mathematicians found that his principles were not quite true or so imprecisely defined that they could be rejected. Or if Newton’s three laws were found to be simply wrong but classical mechanics perfectly sound.
The fact is that articulating the principles he discovered was not Einstein’s strong suit.
This article is about how Einstein came to understand his theory, why that understanding was wrong, and what the correct understanding appears to be (although there is some debate even now about it). First we need to start in 1905, with Einstein’s discovery of his theory of special relativity.
Special Relativity
Einstein’s first foray into relativity occurred with his 1905 paper on special relativity. Physicists had spent years looking for some kind of background “aether” to explain the medium through with light propagated.
Maxwell’s equations that described not only the propagation of light but also all of electromagnetism had been the greatest achievement of 19th century physics. One problem with it was that it defined an exact speed for the propagation of the electromagnetic force as well as light in vacuum: the speed of light. Galileo had taught us that all velocities are relative to some frame of rest. Therefore, to what frame of rest was this speed relative to? It must be the aether, they surmised. The trouble was that no experiment could discover any change in the speed of light no matter what direction of motion light was measured, either with the orbit of the Earth or perpendicular to it.
From the equations, however, Alfred Lorentz showed that you could transform the coordinate variables of Maxwell’s equations: x, y, z, and t (for time) in such a way that you get new coordinates: x’, y’, z’, and t’ that give you the same equation with the same speed of light. What this meant, however, was baffling. Did it mean that distances change depending on your relationship to the aether?
Einstein said no. There was, in fact, no aether, no rest frame. The speed of light was constant for all observers. The Lorentz transformation (and electrodynamics’ invariance to it) meant that depending on your state of motion, two people or “observers” in different states of motion relative to one another could not agree on measurements they made of space and time distances of one of the observers. For example, one would measure one second per second of their own clock, but the other observer would measure slightly more than a second per second.
Einstein’s theory had many bizarre and difficult to test consequences and was something of a curiosity at the time as far as physics went, but it did put the mathematical concept of covariance front and center.
Covariance is just the idea that you can transform coordinate systems of an equation and get the same form equation making the same predictions back out. While the idea that you could change coordinate systems of an equation had existed for a long time, the physical meaning was not taken seriously until Einstein’s special relativity.
The principle on which Einstein’s theory of special relativity is called Lorentz covariance and all natural laws obey it.
It was only with the mathematician Minkowski’s subsequent work in 1908–9 that physical laws obtained Lorentz invariance as well, for Minkowski reduced Einstein’s complex algebraic manipulations to geometry on spacetime and showed how, using four dimensional vectors and tensors, quantities did not change under Lorentz transformations.
General Relativity
Unlike special relativity, which emerged quickly in its final state, general relativity followed a messy and tangled road of false turns and failed anticipation.
At its core, Einstein placed the one idea he had learned from special relativity, covariance, and turned his intellect towards how to expand the idea to non-inertial (accelerated) frames of motion.
Principle of Equivalence
The tool Einstein came up with in 1907 to do this was called the hypothesis or principle of equivalence. In that paper, he proposed a thought experiment of a uniformly accelerating box and a person inside who would be unable to distinguish the acceleration from a gravitational field. The reason is because inertial mass, how much objects resist changes in motion, and gravitational mass, the mass of gravitation pull, are the same.
The equivalence of the gravitational field and acceleration became one of the founding principles of his theory.
Mach’s Principle
Another principle that Einstein adopted was Mach’s principle. Ernst Mach’s 1893 principle of inertial motion was a refutation of Isaac Newton’s demonstration of an absolute frame of accelerated motion.
In his Principia (1687), Newton pointed out that a bucket of water, spun around on its axis, would develop a curvature in the surface of the water. Thus, it seemed that accelerated motion was distinguishable from non-accelerated, unlike inertial motion of Einstein’s special theory. If true, this was a serious problem to developing a theory of relativity for accelerated motion.
Mach, however, pointed out that the bucket’s motion was still relative to all the rest of the matter in the universe. As the bucket rotated, any frame of reference at rest with respect to the bucket, such as an ant sitting on the rim, would see the stars and planets whizzing around it.
Mach’s proposal indicated that, in an empty universe with no stars or planets, the bucket would not demonstrate any curvature of its surface. Likewise, it would be meaningless to say that the universe as a whole is rotating because there is nothing for it to rotate relative to.
In 1912, Einstein claimed he had developed a theory that demonstrated Mach’s principle with a force interaction that would cause the bucket’s curvature. Einstein proposed in 1913 that
. . . the entire inertia of a point mass is an interaction with the presence of all the remaining masses and based on a kind of interaction with them.
He called this idea the hypothesis of relativity of inertia. In other words, the universe as a whole defines, by its collective, average state of motion of billions of galaxies, what an inertial state of motion is.
General covariance
It was also in 1912 that Einstein finally gave up his high school mathematics and adopted, with the help of his friend Marcel Grossmann, the rich formalism of Ricci, Levi-Civita, Riemann, and Gauss. This formalism enabled mathematicians to develop physical theories in arbitrary and even unspecified coordinate systems. Taking the geometry of Minkowski as a starting point, Einstein and Grossmann began to develop a theory that was immune to changes in coordinate systems. In other words, it was generally covariant.
Using general covariance, they published together a theory on the relativity of accelerated motion that combined Minkowski’s coordinate systems and then altered the coordinate system to an accelerating one. Einstein showed how the metric, that matrix of quantities that defined distances in Minkowski space, reflected the parameters of the accelerating motion.
Einstein had most of his theory of relativity by now but all his geometries were flat, meaning that they had no inherent curvature. Einstein and Grossmann’s theory needed non-flat spacetimes in order to account for gravity. The problem was that their theory didn’t agree with Newton’s, so they ended up presenting a non-generally covariant theory of gravity instead.
Einstein spent the next 3 years wrestling with his theory in an intense struggle to reconcile gravity with general covariance. Deeply conflicted, he published multiple papers attempting to prove that physical theories could not be generally covariant, effectively refuting his earlier work with Grossmann.
Yet, by 1915 these problems had evaporated, and in 1916 he published his now famous review article on his general theory of relativity, still a classic today. Why did they evaporate? A lot has to do with his abandonment of certain principles that he thought had to underlie his theory but did not.
Einstein’s Fundamental Principles
In 1918, Einstein attempted to codify the principles of his general relativity theory as follows (adapted from his March 1918 paper):
(a) “Principle of relativity. The laws of nature are only assertions of timespace coincidences; therefore they find their unique, natural expression in generally covariant equations.”
(b) “Principle of equivalence. Inertia and weight are identical in essence.” The metric of the coordinate system determines all the metric properties of space including both inertial and gravitational effects.
(c) “Mach’s principle.” The mass and energy of all bodies determines the metric field which accounts for both accelerated and gravitational effects on any individual body.
Einstein Refutes Mach’s Principle
While popular accounts of relativity held to Einstein’s early viewpoint, Einstein’s own thoughts about his 1918 principles began to evolve almost immediately.
The first to go was Mach’s principle which Einstein began to distance himself from as early as 1919. By 1924 he equated Mach’s original, Newtonian version of the relativity of inertia with “action at a distance”. His own modified form had its own problem in that the definition of inertial motion was given to a “makeshift” quantity, the stress energy tensor representing matter and energy.
By 1954, a year before his death, he said: “In my opinion we ought not to speak about Mach’s principle any more.”
It turned out, however, that the introduction of Mach’s principle in the first place was really intended to solve problems in his earlier incarnations of his theory and abandoning it did no real harm. His final theory of relativity accounted for Mach’s principle by being a field theory that described the interaction of spacetime curvature and matter. Thus, matter affects space and time and space and time affect matter. There is no need for the metric field to be fully determined by the distribution of matter (which may not be true given the need for a cosmological constant) because the distribution of matter is determined by the metric and vice versa.
Thus, while Newton ascribed an absolute reference frame to accelerated motion of his bucket and Mach suggested the reference frame was given by the distribution of bodies in the universe by some action at a distance, Einstein’s more mature understanding was that the distribution of bodies contribute to the shape of space and time itself. The bucket experiences accelerated motion not because of the bodies in the universe directly but because it exists in a spacetime that has been formed by all the matter and energy fields in the observable universe.
General Covariance Refuted
While Einstein dropped Mach’s principle and it was never incorporated into technical expositions of general relativity anyway, he saw general covariance as an essential determining feature of the theory.
As early as 1917, however, Kretschmann pointed out that general covariance was physically vacuous. In his words,
any physical theory can be brought into agreement with any, arbitrary relativity postulate, even the most general one, and this without modifying any of its content that can be tested by observation.
Thus, Einstein’s statement that general covariance is a fundamental principle of his theory is physically meaningless.
Einstein in 1918 grudgingly accepted Kretschmann’s observation:
I believe Herr Kretschmann’s argument to be correct, but the innovation proposed by him not to be commendable.
By the later statement he appeals to Ockham’s razor suggesting that, while, yes, you can make any theory generally covariant, most of them, such as Newton’s, would be overly complicated to the point of being impossible. Thus, he appeals to the “heuristic force” of the principle for adopting his theory.
This is quite a retreat from a fundamental principle to one that is just a good heuristic. Again, can you imagine Newton’s laws only having heuristic force? They only point to a way of figuring out the correct physical laws rather than representing them?
It only took 5 years for Cartan (1923) to develop a generally covariant mathematical formalism for Newton’s laws, making Einstein’s Ockham’s razor argument even weaker.
These days most theories are presented in a generally covariant form pioneered in the 1960’s and 70’s that is entirely free of coordinates. The only difference between Einstein’s theory and those is that the metric and manifold (geometry) of spacetime is not predetermined.
Equivalence Principle Redefined
These days you never hear about Mach’s principle or general covariance being tested in any experiments. Yet you frequently hear of tests of the equivalence principle passing. “Einstein is right again!” The headlines say, but was he? Yes and no.
The equivalence principle we use today is different from the one Einstein claimed in 1918 and clung to throughout his life.
For all the mathematical sophistication he acquired, Einstein never abandoned his notion of the accelerating elevator thought experiment as underlying the principle of equivalence. Presenting in 1922 the idea of one coordinate system, K, that is not uniformly accelerating and another, K’, that is
. . . there is nothing to prevent our conceiving this gravitational field as real, that is, the conception that K is ‘at rest’ and a gravitational field is present we can consider as equivalent to the conception that only K is an ‘allowable’ system of co-ordinates and no gravitational field is present.
Physicists almost universally, however, accepted the principle of equivalence to be something different, given here in the words of the contemporary physicist Wolfgang Pauli:
For every infinitely small world region (i.e. a world region which is so small that the space- and time-variation of gravity can be neglected in it) there always exists a coordinate system, K, in which gravitation has no influence either on the motion of particles or any other physical process.
If you ask any physicist what the principle of equivalence is, they may explain it in terms of Einstein’s example, but ask them to define it mathematically and they will give you Pauli’s version. The problem is that they are not the same. Pauli’s version is sometimes called “local Lorentz covariance” and is the one that is tested in experiments.
Einstein objected to this version and wanted a weaker equivalence principle that applied to uniform acceleration, not arbitrary gravitational fields, somewhere in between the Lorentz covariance of special relativity and the general covariance of general. In his own words in 1916,
The requirement of general covariance of equations embraces that of the principle of equivalence as a quite special case.
In other words, Einstein insisted that general covariance superseded the equivalence principle. Unfortunately, general covariance is far too general to define the theory.
Does General Relativity generalize relativity? (Hint: no)
Another problem with general covariance is that, besides being physically vacuous, it also failed to generalize the principle of relativity that Einstein thought it did.
By the 1960’s many scientists insisted that “general relativity” was no such thing but rather just Einstein’s Theory of Gravity. Far from generalizing relativity, Einstein’s theory refuted it by positing spacetime as having an absolute geometric shape. Therefore, it constrained relativity to flat geometries only
In 1959 Fock, a critic of the idea of general relativity, summed up Einstein’s failure to understand his own theory,
The fact that the theory of gravitation, a theory of such amazing depth, beauty and cogency, was not correctly understood by its author, should not surprise us. We should also not be surprised at the gaps in logic, and even errors. which the author permitted himself when he derived the basic equations of the theory.
Years later, in 1974, Fock wrote about general relativity,
[G]eneral relativity can not be physical, and physical relativity cannot be general.
This echoed the sentiments that had grown from the 1960s on that Einstein’s Theory of Gravity was in no way a generalization of relativity but a refutation of it in the same way that special relativity was a refutation of Newton’s laws.
This realization became prominent with the development in the 1950s of group theory as the standard for defining all physical theories. Group theory is just a way of defining symmetries, ways of transforming mathematical objects contained within a theory without changing their reality. Lorentz symmetry was one of the first of these discovered.
So what is the symmetry group of general relativity?
It is not general covariance, which is a mere symmetry of coordinate systems. Rather, it is the identity group — a trivial group that contains one transformation from a thing to itself up to a constant factor.
Kretschmann was the first to point this out in 1917 (in the same paper as mentioned above). Through a long and convoluted mathematical analysis, he concluded that
Einstein’s theory satisfies no relativity principle at all …it is a completely absolute theory.
In other words, in Einstein’s theory once you fix the physical parameters of the system, the metric representing the gravitational field is fixed. The only freedom you have is to change coordinates. Since coordinate changes are not physical, you have no freedom at all.
Thus, what Einstein mistook as general relativity was no relativity, only a freedom to change coordinates.
Part of the reason for this is that in going from special to “general” relativity, the measurements of time by clocks and distances by rods that are natural in the coordinate frames of special relativity lose their physical relevance. In Einstein’s theory of gravity, coordinate systems have no relationship at all to measurements. Rather measurements can only be obtained from collapsing coordinate system-dependent vectors and tensors into scalar (numerical) values that are coordinate invariant. Thus, the Lorentz covariance of special relativity is a byproduct of giving physical significance to coordinate frames which is impossible in the general theory.
Just what are the principles of Einstein’s theory?
While general covariance and generalized relativity seem like a bust in terms of defining relativity, and the identity group is hardly a symmetry group to hang your hat on, many expositions rely on the one principle that Einstein rejected: infinitesimal Lorentz covariance (the equivalence principle).
Even this principle can be attacked however since coordinate changes should not be able to eliminate the presence of a gravitational field if there is one (which is defined by a non-vanishing curvature in the spacetime geometry if you want to get technical). This may be precisely why Einstein disliked the idea.
These can be observed because all objects, no matter how small, experience non-zero tidal forces in the presence of a gravitational field. Even the smallest rain drop will encounter them. Tidal forces are simply differentials in the gravitational pull on an object. One cannot eliminate these entirely from the theory. Thus, the equivalence principle that we successfully test so often is only valid because tidal forces are small enough to be discounted.
For example, astronauts and their space craft in free fall are never in a truly zero-g state. Rather the tidal forces the Earth exerts on them are negligible. While you can choose a coordinate system that pretends local Lorentz covariance, this is unphysical.
Another option for an underlying principle is simply the freedom of the manifold itself, that there is nothing absolute within spacetime since the manifold is dynamical. This would eliminate generally covariant Newtonian theory since it contains an absolute space and time. This may have been what Einstein was really driving out: the elimination of any absolutes.
There is something compelling about this idea but it does not mean no rules, for Riemann certainly had assumptions about curved geometries as Euclid did about flat ones.
The third option is that the theory simply contains no non-trivial symmetries (i.e., it only contains the identity group) and, therefore, the basic principles are that the theory is the simple assertion that spacetime is geometric, Einstein’s field equations govern its relationship to matter, and gravity is represented by curvature. This way of stating things is empirically useful but lacks some explanatory value.
Combining the two ideas above, Einstein’s equations are easily derived from a basic assertion of a differential form on a spacetime geometry and so, in reality, they embody a complete lack of freedom, meaning, physically, that theory is perfectly constrained by itself. Thus spacetime is only constrained by the rules of geometric manifolds.
Where do we go from here?
Einstein’s theory of gravity has so far proven enormously successful, but most physicists believe that it is only an effective theory, a lowest order approximation to the true theory of gravity and possibly everything else. Many approaches to creating a quantum theory of gravity have focused on one or another symmetry group but given the lack of non-trivial groups in Einstein’s own theory, is that the right direction? Or does it make more sense to remove the symmetries from other forces? Time will tell.
Norton, John D. “General covariance and the foundations of general relativity: eight decades of dispute.” Reports on progress in physics 56.7 (1993): 791.