Quantum mechanics is both the most powerful theory ever conceived by physicists and the most puzzling. On the one hand, countless experiments have confirmed his predictions; the theory underpins modern technology and enables the electronic devices we use every day. On the other hand, quantum mechanics describes an underlying reality that is completely at odds with the world we perceive. In the quantum realm, a single particle exists in multiple places at once, at least when no one is looking at it. The theory also allows for inexplicable connectivity: a pair of atoms, no matter how far apart, can be “entangled,” so that whatever happens to one atom instantly affects the other. Albert Einstein called this phenomenon “spooky action at a distance.”
These paradoxes have defined – or tormented – the theory since its inception more than a century ago. To this day, physicists still disagree on what quantum mechanics tells us about the nature of reality. Are there several universes? Do things only exist when they are observed? Is consciousness somehow at the heart of the laws of physics? What if all these mysteries could have been solved since the birth of quantum mechanics?
. This is the case of physicist Antony Valentini, a physicist at Imperial College London, in his new book. Beyond quantum: a quest for the origin and hidden meaning of quantum mechanics (Oxford University Press, 2026).
On supporting science journalism
If you enjoy this article, please consider supporting our award-winning journalism by subscribe. By purchasing a subscription, you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Valentini says that Louis de Broglie, a French physicist and Nobel laureate, developed a framework for quantum mechanics that eliminated its paradoxes about 100 years ago. In pilot wave theory, as de Broglie’s original idea is called, particles are guided by the waves that accompany them. The particles themselves are always in one position and one position only; it is the spatially extended pilot wave which creates the impression that a particle is immediately here And there. It is not necessary for an observer to give birth to this particle. Although de Broglie’s 1924 conjecture about the wave nature of matter was quickly confirmed by experiment and became part of quantum theory, the physics community dismissed or misrepresented the broader ideas from which he derived his key ideas.
Valentini spent his entire career defending and extending de Broglie’s views. He recently spoke to Scientific American on his lonely path and why de Broglie might have been onto something.
[[An edited transcript of the interview follows.]
In the history of science, has there ever been another situation like this, where there were such divergent views on the meaning of a theory?
I’m not sure that’s the case. If you go back to the days of [Isaac] Newton believed that space was empty and that there was direct gravitational action at a distance. And on the continent, there were the Cartesians [followers of mathematician and philosopher René Descartes]who thought: “Oh no, space is filled with this material medium, and that explains the gravitational attraction. ” But [the debate] didn’t last very long. In the quantum case, the wide variety of interpretations that say such different things about the world — I think it’s a safe bet that there is no analogue in the history of science.
One of the most striking things about modern physics is the deep divide between the macroscopic and quantum worlds, each seeming to be governed by entirely different physical laws. You compare this to the way medieval astronomers divided the cosmos into terrestrial and celestial regions.
I think it’s a useful and valid parallel, this idea that there was a celestial kingdom that we couldn’t understand; everything above the moon and beyond was eternal and unchanging, completely different from the sublunary world, which consisted of ordinary, imperfect, ever-changing matter. This is a distinction that dates back to Aristotle. The parallel with quantum mechanics is extraordinary: the quantum system is something that our minds cannot understand. We can only understand the macroscopic.
The Austrian physicist Erwin Schrödinger developed quantum theory wave equationwhich describes quantum systems as waves that evolve over time. What role did this equation play in the so-called measurement problem: if a particle exists in different places at once, why do measurements find a given particle in only one place?
Schrödinger created the measurement problem by removing particles from de Broglie’s theory. Mathematically, a [quantum wave] is a superposition of many different positions: a particle can be here and here and here; it could be anywhere. You may have a superposition of a living cat and a dead cat, or a superposition of different energies. They are just different variations on the same theme. The wave equation contains all possible positions. How can you then explain that we see this small point object if the only reality is an extended wave?
And this enigma has been recognized since the development of quantum theory.
Here is Wolfgang Pauli writing to Niels Bohr in 1927: “In the last issue of Journal of Physics, an article by De Broglie has appeared… It is very rich in ideas and very precise, and at a much higher level than the childish articles of Schrödinger, who, even today, thinks he can… abolish material points. “And because Schrödinger removed [particles from his equation]we found ourselves in decades of confusion.
Why do you think De Broglie’s theory has been sidelined and neglected?
I’m not sure there is a simple answer. Maybe it’s a mix of reasons.
In 1923, de Broglie developed a new theory of movement. It was a total break, very different from Newtonian or even Einsteinian physics. And yet, this has completely escaped people’s attention. The only thing that entered the collective consciousness of physicists was that de Broglie had shown that a particle can behave like a wave.
De Broglie’s thesis became widespread, even though almost no one read it. Einstein did it. It was Einstein who really alerted people to the fact that de Broglie had done something very important. He encouraged Schrödinger to read it – and he read it. It seems that most people never read de Broglie’s thesis.
And then there is the sociological point: de Broglie was quite isolated in Paris. De Broglie was a bit of a loner; he worked essentially alone. At that time, in the 1920s, France was truly a country lost in theoretical physics. He was strong in experimental physics, strong in mathematics but not in theoretical physics.
Has your search for pilot wave theory been lonely? Reward? Frustrating?
The short answer is all of this and more. Was it lonely? This is a special situation. I really tried to get the key points across to the physicists. And it seems to fall on deaf ears. It’s as if people are stuck in a loop: the same flawed arguments, the same historical misconceptions keep circulating.
When I first learned about pilot wave theory, it seemed obvious to me. Oh my, pilot wave theory is in principle broader physics; quantum theory is a special case of something bigger. The pilot wave theory presents exciting new physics, and perhaps we can find evidence for it.
In your book you describe how the predictions of pilot wave theory about the physics of matter differ in some cases from the predictions of accepted quantum mechanics. In particular, you mention how the cosmic microwave background (CMB) – the radiation created during the big bang that now permeates the universe – could confirm some of the predictions of pilot wave theory.
The CMB is an excellent and promising avenue, and I have worked a lot on it with different collaborators. Anomalies reported in the CMB correspond qualitatively to the type of anomalies predicted by pilot wave theory. There are some tantalizing clues, but the data is simply too noisy to draw any firm conclusions. This question will probably not be resolved for around ten years.
Is the pilot wave theory true? Is this an accurate theory of the world? If I knew it was true, I wouldn’t do research. There is always, in the back of my mind, the idea that this could all be completely wrong! Or it might be partly true. At the end of the 19th century, Ludwig Boltzmann modeled [gas molecules] like little billiard balls, little hard spheres that bounce. It turns out that molecules are much more complicated than that. But his model nevertheless contained a lot of truth. It could be that pilot wave theory is a bit like that, a rough model.