The cosmological constant problem is one of the biggest crises in physics

Supernova remnant SNR 0454-67.2 is likely the result of a Type Ia supernova explosion. In the late 1990s, scientists studied the light from such supernovae to find that the universe’s expansion is accelerating.
| Photo Credit: ESA/Hubble, NASA

Can you get rid of all the energy in the room in which you are reading this? Move the room far from the earth’s gravity, then toss out everything made of matter so you can cross off mass, kinetic, and potential energies. Also pump out the air, the cosmic rays constantly streaming in, and the fog of neutrinos that were made during the Big Bang. Next, kill the energy in photons: darken the room completely, and clear away the microwave radiation left over from the infant universe.

Now all that’s left in your room is empty space, and no energy – right? No! Your room still has “dark energy”. In a patch of space the size of your room, dark energy is so scant that it is nearly impossible to detect. But across the cosmos, which is full of space, it contributes a titanic 70% of all energy. Matter, in the form of stars, gas, and the mysterious dark matter, supplies most of the other 30%, while radiation in the form of photons and neutrinos chips in 0.01%.

The hand of dark energy

The basic truth about space that Albert Einstein taught us is that it is not a state of ‘nothingness’. Instead, it is a bendable, stretchable medium that we occupy, much like water is for fish. Add energy uniformly across a patch of space and that patch will expand (or shrink, if the energy is negative). Each form of energy tells the universe how to expand in its own way. This is much like how you can inflate a balloon with air, water or sand, and in each case it will have a characteristic look and feel.

Since dark energy dominates the energy budget of the universe, it also dictates the rate at which space expands. We can reverse-engineer this fact to estimate how much dark energy is present in any volume of space, by considering the size and age of the universe. Add too much positive energy and the cosmos would expand too fast: galaxies would fly away from us faster than light, so that only the regions of the universe nearest to us will be visible. Effectively, this “observable universe” would appear to shrink. Add too much negative energy and the universe would actually shrink to a tiny point. The greater this negative energy, the sooner this event.

Everyone can agree that the universe is larger than India and older than the Indus Valley Civilisation. These facts alone restrict the density of dark energy to the caloric content of a pinch of sugar in a cubic metre. In reality, the universe is wider than billions of lightyears and older than 10 billion years, so the dark energy is actually as dilute as one sugar crystal in a cubic kilometre.

The problem arises

And here is the crisis: the calculated dark energy content of the universe, based on theory – a bread-and-butter particle physics calculation – is bizarrely off the mark. In the simplest estimate, there should be enough energy in a cube with sides of length 10^-21 cm to unbind the entire Milky Way, yet in reality there appears to be much, much less. Nobody knows a convincing way to get around this. This is to say that while the universe is observed to be incredibly big, physicists calculate that it must be tinier than a proton. This, in a nutshell, is the cosmological constant problem, and it has come to be called rightly as “the worst theoretical prediction in the history of physics”.

How is the amount of dark energy predicted from just theory? To begin with, particle theorists have a pretty sharp notion of what dark energy is composed of. (This situation is different from that of dark matter, whose identity is a total mystery.) There are three unavoidable quantities that behave exactly like dark energy.

1. The weight of the vacuum – Einstein realised that space supplied its own energy and that it was spread uniformly, i.e. an energy that was a “cosmological constant”. Back then, physicists believed that the universe, instead of expanding, stayed still. So in his equations, Einstein cancelled the cosmological constant against the energy of matter. But when he soon learnt from astronomer Edwin Hubble that the universe is actually expanding, he rued the missed opportunity to forecast this observation, calling it his “biggest blunder”.

2. Zero-point energy – Thanks to Heisenberg’s uncertainty principle of quantum mechanics, any physical system has a minimum positive energy. This is also true of quantum fields that source elementary particles such as electrons and photons (like sugarcane sources sugar cubes). These fields fill space, thus furnishing energy at every point in the universe.

3. Field potentials – All fields have kinetic energy, but certain fields that carry no quantum spin, such as the Higgs field (which sources the Higgs boson), also have potential energies. They also contribute energy to every point in the universe.

The fine-tuning

Contributions 2 and 3 are calculable in theory, and end up supplying an enormous amount of energy that should make the universe smaller than the proton. But contribution 1 is unknown. Imagine trying to buy a ship of unknown cost using all your stocks and real estate, and getting a paisa back as balance. Wouldn’t you suspect the seller of tuning the price over seven decimal places?

The cosmological constant appears to be fine-tuned over a breathtaking 122 decimal places. That really is the heart of the problem: what is the mathematical principle that can explain away this apparent fine-tuning? The possible answers – posited by Stephen Hawking and Steven Weinberg, among others – are equally dizzying, but that is for another day.

Nirmal Raj is an assistant professor of theoretical physics at the Centre for High Energy Physics in the Indian Institute of Science, Bengaluru, and tweets at @PhysicsNirmal.