The Worst Prediction in the Whole of Physics

One of the most amazing consequences of quantization of the electromagnetic field is the appearance of the so called zero-point energy or the quantum vacuum. This is a direct consequence (mathematically speaking) of the fact that – in quantum mechanics – the electric and the magnetic fields become q-numbers and, unlike in classical physics, they can no longer be specified simultaneously (they do not commute). The energy of the field equals the sum of the squares of the electric and the magnetic fields, but – because they are now q-numbers – even in the lowest energy state we cannot make this sum vanish (if we could, we would be able to specify the values of both electric and the magnetic field, which would violate the fact that they are q-numbers; in other words, if the energy of the quantum vacuum was zero, this fact alone would violate the Heisenberg Uncertainty Relations since it would mean that both the electric and the magnetic field are simultaneously zero!).

Photo by Christopher Burns on Unsplash

I’ve written about the reality of the quantum vacuum before (for which we have indirect evidence in the spontaneous emission of radiation by matter, in the Lamb shift, the Casimir effect and so on), however, I would now like to talk about one of its paradoxical consequences. We don’t understand this one, and it could signal some kind of trouble in the foundations of quantum physics (or field theory or both).

To a good approximation, we can think of our universe as a box filled with the quantum vacuum. Each frequency that can occur within the universe (the lowest frequency being inversely proportional to the size of the universe and the highest being inversely proportional to the smallest possible length in the universe – let’s assume this is the so-called Planck length) gets one half of Planck’s constant times the frequency worth of energy in the vacuum.

Now in quantum electrodynamics, this enormous amount of energy is not a problem because only the energy differences are relevant. In fact, in many applications we can ignore the vacuum energy and set it equal to zero without any consequences (though the vacuum state will still play a role).

But there is one force of nature that unfortunately ought to see this vacuum energy in all its totality, absolutely speaking, and not just the differences between energies. Enter gravity.

According to General Relativity, our best theory of gravity, the total energy of the universe does actually affect gravity. Even an otherwise empty universe would, according to quantum mechanics, produce an enormous amount of energy that would gravitate strongly. In the equations of general relativity, this vacuum energy sometimes goes by the name of the cosmological constant.

Our best astronomical observations (to do with the rate of the expansion of the universe) are telling us that the cosmological constant is tiny. It is the energy equivalent of having a single Hydrogen atom on average in every cubic meter of the universe! However, if we calculate the quantum vacuum energy of the universe it turns out to be 120 orders of magnitude bigger. Hence: the worst prediction in the whole of physics.

Now, of course, there are a number of ways out of this problem.

One is simply to say that our quantum calculation is not correct. We only added up the electromagnetic contribution (i.e. due to the bosonic fields), however, when we add up all fields, various contribution might just cancel out (or almost cancel out). The logic here could be that the fermionic fields (e.g., electrons) produce the same vacuum energy but of the opposite sign to the bosonic fields.

Another way out is to deny the connection between the cosmological constant and the vacuum energy. Perhaps the cosmological constant is due to something else, say the total mass in the universe.

Yet another way of resolving the apparent paradox is to deny the reality of this quantum vacuum energy. Maybe this energy does not exist, or, maybe, even if it did, it has no gravitational influence whatsoever. Some physics heavyweights like Pauli and Schwinger thought this way.

The most interesting possibility, I think, would be if this discrepancy really signalled that there was a fundamental problem with quantum mechanics itself. Perhaps just as classical physics predicted an infinite amount of energy radiated by a black body (clearly not true in practice), the huge amount of energy that quantum physics predicts in the vacuum (which, incidentally, would also be infinite if there was no smallest length, a question that we have no answer to at present) could also mean the breakdown of quantum physics.

So, could the “worst” prediction in physics give us a clue about how to “fix” quantum physics? Perhaps, but given the success that quantum physics has enjoyed in the microscopic world of atoms and photons and subatomic particles and molecules, it is difficult to see how one ought to modify it while preserving all the good explanations it has provided us with so far.

The key to fixing classical physics was the introduction of q-numbers, namely the fact that quantities such as position, momentum, energy and so on, are no longer just represented by the ordinary (real) c-number, but ought to be represented by more complex objects (matrices). Briefly, this resolved the black body issues since first emitting light and then absorbing it, now, in quantum physics, has a different probability to first absorbing light and then emitting it (in the technical jargon, the two processes no longer commute, just as the product of two matrices generally doesn’t). Writing a detailed balance between matter and light then gives us the famous quantum Planck distribution (which is the correct formula for the black body).

So, should we be upgrading q-numbers to some other entities, say the w(eird)-numbers? My personal feeling is that it might not be that quantum physics is at fault here. It could be that we should be looking into modifying General Relativity instead. But I don’t mean quantizing it (yes, we have to do that too, however, I don’t think this will solve the problem of the cosmological constant). I mean that perhaps gravity is not what we understand it at present to be (curvature of spacetime).

One can only speculate on how gravity ought to be modified. Wheeler succinctly described general relativity by saying that “matter tells spacetime how to curve, and spacetime tells matter how to move”. Maybe there is not such thing as spacetime (it is all about relative positions of masses). Maybe there is no such thing as matter either…I mean at the fundamental level, these are not the right “elements of reality”.

In this spirit, there is an interesting idea, called induced gravity, which is the idea that gravity is not really a fundamental force, but is the result of all the quantum vacuum fluctuations. So, it’s not that the vacuum contributes on top of gravity, but it is, in fact, gravity in toto. This is very interesting; however, it has not really been developed properly. For instance, suppose we have a mass in a superposition of two different places at the same time, what kind of an (induced) gravitational field does this produce?

Also, and this is related to something I’ve written about before (the BMV effect), what kind of dynamics do we have if two masses are each superposed and then coupled by induced gravity. Can such gravity entangle the two masses? I hope the answer is yes, since – otherwise, the whole idea of gravity induced by quantum mechanics might be suspicious. But luckily, our experiments are getting closer and closer to being able to test this and, as I frequently say in my blogs, now is certainly a great time to be a physicist!