Throughout most of this book we have been considering the cutting edge of experimental gravitational physics research, ranging from tests on the scale of millimetres to those on the size of the entire observable Universe. In this chapter we will consider some of the issues involved in the theoretical description of gravity.

Since 1915, it has been Einstein’s theory that has shaped our understanding of the gravitational interaction. This theory treats space and time as a single object, and lets the properties of that object be determined by the matter that exists within it. I hope that the reader has, by now, been convinced of the phenomenal success of Einstein’s theory. It is truly extraordinary that a single theory should be able to explain such a wide array of physical efects. Einstein’s theory, however, is unlikely to be the final word in our understanding of gravity. A lot has happened in the world of theoretical physics since 1915, and much of it suggests that we should expect there to be an even more fundamental theory.

Quantum mechanics and gravity

Not long after Einstein published his theory of gravity, at the beginning of the 20th century, the world of theoretical physics was forever changed (once again) by scientists such as Bohr,Heisenberg, Schr’dinger, and others. Since Newton, and up until this point, physics had been thought to be deterministic. That is, if you know enough information about the position and motions of all objects in the Universe, then you should be able to predict the future with arbitrarily high precision. Physical theories that work in this way are now referred to as classical theories. Einstein’s relativity theory is an example of a classical theory. The revolution led by Bohr, Heisenberg, and Schr’dinger created another type of theory, quantum mechanics. The new quantum theory was based on probability, and resulted in a description of nature in which it was only ever possible to calculate the odds that certain events will happen in the future, and within which it is never certain what the future holds.

Quantum mechanics was an astonishing success. It described the nature of light, and the building blocks of all known matter, to extraordinary precision. Later, Paul Dirac, a professor at the University of Cambridge, showed the world how these new ideas could be used to create quantum theories of electricity and magnetism. This led to what is now called the Standard Model of particle physics: a quantum mechanical description of all of the known particles in nature, and the forces between them. Winding forward to the present day there can be no serious doubt that nature is quantum mechanical. The predictions of the Standard Model have all been verified, with the crowning jewel being the experimental verification in 2012 of the existence of the Higgs boson (a theorized particle that is an important component of the modern Standard Model, and which gives the other particles their mass). It is quantum mechanics that underpins much of modern chemistry and material science, giving us the semi-conductors that our computers are made from; and the lasers and LEDs that are used to construct our DVD players and televisions. Quantum mechanics is a fact of life. It is how nature works, and it becomes increasingly important when we seek to describe the microscopic world.

Yet despite its ubiquity in all other areas of physics, the application of this approach to gravity is still not understood. While the electromagnetic force can be quantized in a relatively straightforward way, and while the matter in the Universe has long been described using quantum mechanics, the quantum theory of gravity remains elusive. This is probably the single biggest unsolved problem in physics, and has been for the past fifty years. It is extraordinarily diicult to apply the logic that has been so successful in other areas of physics to gravity, and so our current best description of gravity is still Einstein’s classical theory.

Part of the problem with this state of afairs is that there are situations in which one would expect both quantum mechanics and gravity to be required. One example of this is the centre of a black hole. As we’ve discussed already, black holes form when large stars undergo catastrophic collapse. The matter that was originally part of the star gets compressed by gravity to ever higher densities. In fact, according to Einstein’s theory, the collapse should continue until all the matter is squashed to a single point. Now, according to quantum theory we should expect new quantum efects to become apparent when we consider very small distance scales and very high energies. What I’ve described is therefore a regime where both quantum mechanics and gravity are expected to be required in order to get a description of the physics at play. But there is no agreed-upon theory of quantum gravity, so it is (at present) impossible to know what should happen at the centre of a star after it has collapsed. This is obviously an unsatisfactory state of afairs. If we want to be able to describe everything that exists in nature, then we need a quantum theory of gravity.

The reasons for the apparent incompatibility of quantum theory and gravity are many, and can be somewhat complicated. First there is a conceptual diference between Einstein’s approach to gravity, and how forces are treated in quantum mechanics. In Einstein’s theory, gravity is a result of the curvature of space-time. There is no external force that pulls things together. The apparent way in which massive bodies move towards each other is simply a result of the curvature of space-time. The Earth, for example, is not pulled towards the Sun, it is simply in free fall, following the shortest path available to it in a curved space-time. This is not the case for other forces. The electric force, for example, is a result of the electric field that gets generated by charged particles. The electric field exists within space and time, but is not the same as space or time, in any sense. Space and time are simply the arena within which the electric force plays out. This idea of space and time having their own independent existence, and being passive parts of the problem, is built into most approaches to quantum mechanics. To try to use these approaches to describe gravity therefore goes against what we were taught by Einstein.

There are also very good mathematical reasons why gravity and quantum mechanics are incompatible. The foremost of these is a property of gravity called non-renormalizability. When quantum mechanics is used to describe a force, the result of the calculations that we perform can often result in answers that contain infinities. An example of this occurs when we consider two charged particles. To work out the force between them, quantum mechanically, we have to add the contributions from all of the possible positions of the two particles. Some of these configurations are when the particles become very, very close,in which case the force becomes very, very large. Summing over all possible positions then gives a result that is infinite. This unreasonable answer can be corrected for, in the case of the electric force, by using a process called renormalization. This process removes the parts of the equations that contributed the infinities. That is, the infinities are essentially subtracted from the original equation. The result is then a sensible answer, which can be tested with experiments. Renormalization, however, doesn’t work with Einstein’s gravity. The infinities cannot be subtracted, as there is nothing in the original equations that looks anything like the terms that become infinite. Quantum gravity calculations therefore seem to give infinite answers. Something is obviously wrong.

There have been many attempts to fix these problems. They range from changing Einstein’s equations (so that they appear to be more renormalizable); to changing quantum mechanics (so that it is no longer based on particles); through to changing the nature of what we think of as space and time (so that they are not continuous). It is not possible to give a fair representation of all of these approaches here, or to go into any one of them in any detail. They are very complicated ideas and they are all works-in-progress. There are a couple of theories that I feel I should mention though’these are String Theory and Loop Quantum Gravity. These are both extremely bold and ambitious attempts to construct quantum theories of gravity. If correct, the hope is that physicists could use these theories to describe what happens at the centre of black holes. They are, however, quite diferent from each other. They prioritize diferent aspects of the problem, and approach the technical and conceptual diculties just described in quite diferent ways.

String Theory was born out of particle physics. It is based on the idea that the basic constituents of matter are not point-like particles, but are instead tiny one-dimensional strings. This is a radical idea, and it has led to a lot of interesting maths and physics. Indeed, many physicists consider it to be our best hope of finding a theory of quantum gravity. The hypothesized strings are very small, so that for the most part they appear to us efectively as point-like particles. When we try and quantize them, however, their stringy-nature leads to diferent results. The equations that govern the strings also contain aspects that look a lot like the equations that govern Einstein’s theory of gravity. It therefore appears that gravity is, to some extent, built into String Theory. There are drawbacks, however, as the consistency of the equations that govern the strings require us to add between six and twenty-two extra dimensions of space to our description of the Universe.

These extra dimensions are thought to be wound up and compact, so that we don’t see them in our everyday lives. Nevertheless, they have to be there for the theory to be self-consistent. Interestingly, the existence of these small extra dimensions leads to the possibility that gravity could work diferently on very small scales.

Loop Quantum Gravity is often seen as the main competitor to String Theory. The starting point for this theory is the idea that on very small scales space-time has a granular structure. That is, space and time are not the smooth continuous variables that we usually consider them to be. Instead, space-time is atomized. Quantum theory is then applied to the loops that make up this new structure. This is also a radical idea, and it tends to be favoured by aficionados of General Relativity because of its emphasis on space-time as the fundamental object of interest’instead of as a background. Loop Quantum Gravity is, however, very much a work in progress. It remains to be seen whether the approach it employs is the correct one or whether String Theory, or some other as yet undiscovered theory, is a preferable description of nature. Further work is needed before most of us would be prepared to make any bets on the outcome of this debate.

Particles in gravitational fields

Quantizing gravity is fraught with diculties. So let’s turn to another question: how does quantum mechanics work in a gravitational field？ Here we will treat space-time in the same way that Einstein did in his classical theory of gravity. Once we have a classical space-time at our disposal, however, we will consider what happens when we try to treat the matter content as being governed by quantum theory. This mixed approach, with matter being treated quantum mechanically, and gravity being treated classically, is usually referred to as semi-classical physics. It is a less ambitious project than full quantum gravity but nevertheless gives us interesting insights into how quantum systems work in the presence of gravity.

One of the pioneers of this subject was Stephen Hawking, who in 1974 showed that quantum mechanics should lead to black holes emitting radiation. This discovery shocked the scientific community, as according to Einstein’s theory nothing can ever escape from a black hole. Hawking’s calculation was a semi-classical one. He took the classical description for the space-time around a black hole and allowed quantum mechanical particles to exist within it. He showed, using a fairly simple quantum mechanical calculation, that if there was no radiation in the distant past, then there must exist radiation in the future. The only possible explanation for this was that the radiation was produced by the black hole. Of course, radiation carries energy, and in this situation the only source of energy is the mass within the black hole (remember that mass is a type of energy in Einstein’s theory). So Hawking had shown that black holes naturally shrink, by radiating away their mass, and that they must eventually cease to be.

Hawking’s result was novel, and sparked several new fields of research in gravitational physics. Almost immediately after Hawking’s discovery it was shown, by Bill Unruh, that the very existence of particles can be questioned when we consider them within relativity theory. Particle physics is very much the domain of quantum mechanics, and Unruh showed that if observers are in relative motion, with one accelerating with respect to the other, then it is entirely possible for one of them to detect the existence of quantum particles while the other detects nothing at all. That is, whether or not particles exist depends on the motion of the person who is trying to measure them.

To get across the strangeness of this result, let me illustrate it with an example. Consider that you’re an astronaut, ’oating around freely in outer space somewhere. You see nothing anywhere near you. If you then start accelerating, by holding on to a passing spaceship, say, then suddenly what you thought was empty space bursts into a sea of particles. I’m exaggerating a little here, of course. You would need to accelerate very quickly indeed to see a large number of particles. Nevertheless, the principle is sound. When you accelerate you detect particles where previously there were none. Now introduce gravity and the situation becomes even more complicated. Gravity is caused by acceleration, so by sitting at my desk, in the gravitational field of the Earth, I am being exposed to a small number of particles that wouldn’t be there if I were falling freely. The number is too small to measure, but if I moved my desk so that it was near a black hole (where gravity is much stronger) then it would be an entirely diferent story. I would be bombarded with high-energy particles and radiation.

All of this has interesting consequences for black holes, which can now have a temperature associated with them based on the temperature of the particles and radiation that they emit. But it also has consequences for other areas of gravitational physics, including cosmology. In some ways the gravitational field of a cosmological model of the Universe is similar to the gravitational field of a black hole, and indeed Gary Gibbons and Stephen Hawking showed that the radiation that black holes produce should also be produced by the expansion of the Universe. The faster the expansion, the more radiation there should be, and the higher its temperature. This radiation isn’t emitted from anything within the Universe, but is a by-product of the expansion itself. It is a direct consequence of considering quantum particles existing in a gravitational field.

Cosmic inflation

To date, the most successful application of quantum theory to gravity has probably been in the very early stages of the Universe’s history. The name that physicists use to describe what happened during this period is cosmic in’ation. In Chapter 5, we considered the Big Bang model of the Universe, and its various successes. As well as explaining a lot of astronomical data, however, the Big Bang model presents us with a few problems. One of the gravest of these is that some of the ripples we see in the CMB appear to be so large that light could not have made it from one of their edges to the other within the lifetime of the Universe. This is a very serious problem, because nothing can travel faster than light. So what could possibly have caused these ripples？

The answer is not obvious, but one possible explanation for their existence is that the Universe might have expanded very quickly in its very early history. If this happened, then very small ripples would have been forced to grow into large ones, and the problem would be solved. The hypothesized period of rapid expansion is what is called cosmic in’ation. Now, as scientists, the way to test a hypothesis of this type is to try and predict other consequences that it might have, and to get out our telescopes to see if we can verify those predictions. This is dicult when considering in’ation, as we do not know exactly what caused it. It also happened a very long time ago. Nevertheless, there are a number of generic predictions one can make, and that can be verified by observing the night sky. One of these is that the geometry of space should be close to ’at. This matches observations, as we have seen. Probably the most impressive prediction, however, involves the application of the semi-classical physics just described.

You will recall that Gibbons and Hawking demonstrated that an expanding space creates a sea of radiation. It turns out that the radiation produced is not perfectly uniform at every point in space. The statistical nature of quantum mechanics means that random ’uctuations are introduced, so that at some points there is a little more radiation, and at other points there is a little less. It’s impossible to predict where any one of these quantum mechanical ’uctuations will occur, but the theory does let us predict how often we should expect a randomly selected point to be over-dense or under-dense. It also tells us how we should expect the over-dense and under-dense regions to be distributed, on average. These are all predictions of semi-classical physics, and we can test the theory by looking for their consequences. In particular, we can test the idea of cosmic in’ation by looking for the consequences of the quantum ’uctuations that it would produce.

Now recall that in Chapter 5 we discussed the ripples that exist in the CMB. These ripples are a very important source of information for cosmologists, but so far we haven’t spelt out where they came from. That is, we haven’t said what caused the small seeds from which they grew. These seeds need to have a very special form in order to explain the statistical properties of the ripples that astronomers measure in the CMB, and, before in’ation was introduced, there was no clear idea about where they should have come from. If in’ation really did happen in the very early Universe, then one way of sowing these seeds could have been through the small quantum mechanical ’uctuations that Gibbons and Hawking predicted. It turns out the ripples that were measured by COBE, WMAP, and the Planck Surveyor are just what we should expect from such a scenario.

This was a remarkable discovery. Not only has the most generic prediction of the in’ationary epoch been verified, but it also appears that we have verified the peculiar calculations that result from considering quantum mechanical processes in a gravitational field. The type of radiation that Hawking had predicted in 1974 still hasn’t been seen directly, but its consequences appear quite plainly in the CMB. The evidence is all there, in the maps of the CMB that have been recorded by astronomers. But, again, this is not the end of the story; there is very likely more evidence out there, waiting to be collected. The same quantum mechanical processes that generate ripples in the CMB should also be expected to generate gravitational waves. These are exactly the gravitational waves that the observers running the BICEP2 experiment mistakenly thought that they had detected in March 2014 (see Chapter 5). At the time of writing, the gravitational waves from cosmic in’ation have still not yet been confirmed observationally, but if they can be found by future experiments then they will open up a whole new window on the early Universe.

The cosmological constant

We noted earlier that the acceleration of cosmic expansion is said to be caused by dark energy, but we didn’t go into any detail about what dark energy might be. The truth is that we don’t yet know what dark energy is, but we do have a favourite candidate: the cosmological constant. In this section we will consider the cosmological constant in more detail.

The cosmological constant was first introduced by Albert Einstein in 1917. At that time, it was not known that the Universe was expanding, and Einstein introduced his cosmological constant in order to produce a cosmological model that was static (neither expanding nor contracting). Of course, we now know that there is very good evidence to support the idea that the Universe is expanding. When Einstein became aware of this he withdrew his cosmological constant, which was promptly brushed under the carpet as something of a scientific embarrassment. However, the cosmological constant remained a perfectly consistent modification that one could make to the field equations of his theory of gravity. It’s just that there was no need for it. Not, that is, until it was noticed that the expansion of the Universe was accelerating.

A cosmological constant can be thought of as a universal gravitational force, pulling or pushing all particles in the Universe together at the same rate. This is exactly the sort of thing that’s required to make the Universe accelerate. All we have to do is make sure the cosmological constant is set up to push things apart, give it the right magnitude, and it will make the expansion of the Universe accelerate. In fact, it’s by far the simplest way to make the Universe accelerate.

The cosmological constant, tuned to have the correct magnitude, fits all of the current observations. Of course, we expect the quality of this data to improve considerably over the coming decades. When this happens we will be able to see whether the cosmological constant remains a good fit’or not. If it is, then this will be good evidence for its existence. If it’s not, then we will have to be more imaginative. For now, we can speculate on what it would mean if there really was a cosmological constant in our Universe. This is interesting because the cosmological constant, although simple, brings with it a number of problems.

The first and foremost problem associated with the cosmological constant is that, if it is to cause the expansion of the Universe to accelerate at the present time, then it must have been very finely tuned in the early Universe. Fine-tuning is one of the bugbears of theoretical physics. It’s one thing to come up with an explanation for a physical efect, but if your explanation requires things to be arranged in an extremely special way then it starts to look less and less compelling. The fine-tuning associated with the cosmological constant comes from the fact that its value doesn’t change in time (it’s a constant). This means that if we want it to have the correct magnitude today, then in the very early Universe we have to pick a value that is very, very, very small compared to the energy scale of matter at the time, but not quite zero. If the cosmological constant were too large, it would have caused the expansion of the Universe to accelerate at much earlier times. If this happened, then stars and galaxies would never have formed, and there would have been no life anywhere. If it were any smaller, it would not cause the required amount of acceleration, and we would never have noticed it. To fit into this sweet spot we need to pick the magnitude of the cosmological constant to have a very particular value, with very high precision. This precision is widely thought to be at the level of about one part in 10120 (that’s one followed by one hundred and twenty zeroes).

The cosmological constant problem, just described, is exacerbated by the fact that quantum mechanical efects also contribute to its magnitude. Given our current understanding of quantum mechanics, we would expect these contributions to throw the value way of from the very special value that is needed observationally. One might counter this argument by saying that we do not yet fully understand the quantum processes that would cause these efects, and how they work in the presence of gravity. One could speculate that there might be some reason why the various quantum contributions should cancel each other out, and that we just don’t know about it yet. This might be possible, but there’s a further problem. The particular quantum contributions that we expect the cosmological constant to receive are not always the same’they change during the diferent epochs of the Universe’s expansion history. To consider the possibility that a set of quantum corrections might all cancel each other out is one thing. To assume this should happen over and over again is quite another. The fact that the cosmological constant is so very finely tuned therefore looks even more surprising when we take quantum mechanical efects into account. This is why the cosmological constant problem has been called, by some, the worst fine-tuning problem that has ever occurred in physics.

The multiverse

The problem of the cosmological constant is considered so great, and so pressing, that many physicists have started to entertain some quite drastic proposals in order to try and explain it. One of the most fantastical, and most widely considered, ideas is the possibility that there is more than one universe. If this were the case, and if the cosmological constant were somehow to take a diferent value in each universe, then it might be possible for us to find ourselves in a universe with any given value for the cosmological constant. Even if the value we observe looks fine-tuned, this might just mean that we are in a relatively rare universe, and that there are many other universes in which the cosmological constant takes a more natural-looking value.

This idea of many universes, or a multiverse, doesn’t by itself alleviate the problems associated with the improbability of measuring the cosmological constant to have the value that we observe. Instead of fine-tuning the value of this constant, we are instead forced to carefully select an unlikely universe for ourselves to live in. However, if we couple the idea of a multiverse with what is known as the anthropic principle, then things become very diferent. The anthropic principle, roughly stated, says that we(as life forms) can only ever observe a universe that is capable of supporting life. This sounds obvious, but it provides a mechanism to select which of the possible universes we might be able to find ourselves within. If a particular universe contains a cosmological constant that is so large that stars and planets can never form, then it is unlikely that we would find ourselves living there. This automatically de-selects a large part of the multiverse, and makes our universe look a lot more likely.

This idea raises a lot of questions. Where are these other universes？ How are they connected to ours？ How does the value of the cosmological constant change between them？ And how likely are we to find ourselves in any one of them？ These are very fundamental questions, and although there are mechanisms for generating many universes from some theories of cosmic in’ation, it is pushing the boundaries of science to say that we can investigate them as if they were physical realities. For some, the idea of a multiverse is a glorious one, motivated by observations of how gravity works on astronomical scales, and ’eshed out by our theories of what happened near the Big Bang. For others it is worse than the problem it was intended to solve. Many in this latter group consider it wrong to invoke unobservable regions of space and time in order to solve a problem in our own observable Universe. While it might be self-consistent, and even well motivated, to do so, the existence of these other universes cannot be tested directly. Some people within this group therefore argue that such an approach is essentially unscientific, and belongs to the realm of metaphysics.

Whether or not the limits of science can be stretched to include a multiverse is a topic of lively debate, with diferent groups passionately arguing their various cases. Future astronomical missions will advance this argument by measuring the properties of whatever it is that’s causing the Universe to accelerate in its expansion. The future development of theoretical physics may also shed light on the naturalness of the cosmological constant we appear to measure. For now, however, we must wait.