After 80 years of fruitless struggle by human mathematicians, a major geometry conjecture has finally been solved with a simple query to a chatbot.
The company OpenAI, manufacturer of ChatGPT, announcement the result yesterday, with comments from a number of experts, who described the artificial intelligence method as “clever” and “elegant”. This feat follows months of noise, but less impressive Mathematical advances based on AI mark a real milestone. Unlike all of these previous feats, this result would be worthy of publication in a leading mathematical journal, as well as attracting media attention, even if achieved by humans alone.
“No previous AI-generated proof has come this close” to meeting these high standards, wrote Timothy Gowers, a mathematician at the University of Cambridge, in a comment requested by OpenAI.
On supporting science journalism
If you enjoy this article, please consider supporting our award-winning journalism by subscribe. By purchasing a subscription, you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
“It is THE interesting and unique result produced autonomously by AI so far,” says Daniel Litt, a mathematician at the University of Toronto, who was consulted by OpenAI to verify the proof but is not involved in the company.
The “unit distance” problem is simple to explain but daunting to solve: a mathematician’s favorite quality.
Draw nine dots on a piece of paper. The goal is to get as many pairs of dots as possible spaced one inch apart. You can line them all up so that you have eight pairs separated by an inch. Or you can draw a three-by-three grid and count 12 pairs. For any number of points, even billions or trillions, the problem is: what is the largest number of pairs you can get?
In 1946, mathematician Paul Erdős guessed the best strategy. This was the grid approach, but with much smaller spacing between points, so that pairs could be established across multiple points on the grid. Erdős showed that by using sophisticated mathematics to choose this spacing extremely with attention, you might do a little better than a simple grid, but only slightly.
In fact, Erdős claimed that no one could do better. And despite valiant efforts, for eight decades, no one succeeded. But no one was able to prove him right either, even though most experts agreed with his intuition.
That changed two weeks ago, when OpenAI mathematicians Mehtaab Sawhney and Mark Sellke, who recently made headlines for using AI to solve a number of problems, less prestigious “forest problems”– passed the conjecture to a large internal language model (LLM) trained in general reasoning. They asked him if Erdős was right. After producing hundreds of pages of careful logic and calculations, he broke his long-standing record.
“It’s like magic,” Sawhney says. “It’s an incredible experience to see a machine give me something that really resembles the way I work. »
“What the model did is totally different from the ‘square grid’ construction,” Sellke explains.
Instead, he constructed a more elaborate grid, living in a sort of higher dimension. This higher-dimensional array of points had special mathematical symmetries that made it easier to separate even more pairs by the same distance. The AI model then developed a way to map this otherworldly grid down to the two-dimensional page, producing a flattened digital “shadow.” The result is far from a grid, and Sawhney says it’s too difficult to draw on paper for even a small number of dots.
AI, however, has not proven that its approach is the best that can be done. In fact, mathematician Will Sawin already improved the AI grid.
OpenAI privately contacted Litt, Sawin, Gowers and a number of other mathematicians to verify the proof of the LLM. Together (and without direct involvement from the company), they wrote their individual conclusions. (No external experts have seen the AI’s original output, however – just an edited version of its thought process.)
What stood out, they said, was the AI’s preternatural patience and focus. Human experts, largely agreeing with Erdős’ thinking, have devoted more effort over the years to trying to prove rather than disprove the conjecture. And even the few who were looking for a counterexample would probably not follow such a difficult and tedious path – building this large shape – without any signs of success. But an LLM experiences the costs and benefits of trial and error differently.
“AIs have an advantage: they can’t just try every known method,” says Jacob Tsimerman, a mathematician at the University of Toronto, who was not involved in the work but participated in the companion article requested by OpenAI. “They can play longer and in more dangerous waters than mathematicians without getting overwhelmed.”
Several experts consulted by OpenAI noted that even if the unit distance problem was well known, a proof that Erdős was right would have been much richer mathematically than a counterexample. Such proofs typically require completely new knowledge that can then be applied to a wider range of problems. The mathematical tools used here are not new, although their application in this area appears to be. “The model did not invent something fundamentally new that no one saw coming,” explains Sébastien Bubeck, a mathematician at the head of OpenAI’s mathematical explorations. “He performed like a amazing mathematician.”
Experts were also quick to add that without human intervention to “clean up” the AI’s work, the result would not be as convincing. “Humans still play a critical role in discussing, digesting, and refining this evidence, as well as exploring its consequences,” mathematician Thomas Bloom wrote in the paper “Reflections.”
Harvard University mathematician Melanie Matchett Wood says humans’ progress was likely limited by their belief that the conjecture was true. If all the experts gathered afterwards to analyze the LLM response had spent the same amount of time looking for a counterexample, she says, they would have found one. “Maybe people should spend more time, you know, playing devil’s advocate,” says Wood, who also commented for OpenAI.
This is plausible because the AI solution was, in hindsight, a simple approach that no human had ever attempted despite the fact that the tools already existed. Such circumstances are believed to be rare for major unsolved mathematical problems. “I guess he was lucky to have found one of the cases where the experts tried and missed something,” Litt says. Truly new and revolutionary ideas remain beyond the reach of today’s LLMs, instead leaving machines to mine literature for rare gems where humans missed a relatively simple approach. Even so, Litt adds, “I guess we’re about to find out that they’re actually not that rare.”
Wood also warns against AI less desirable traits as a mathematicianlike his tendency to present every idea as his own. “We recognized that there were some very similar ideas in the literature that weren’t being credited,” Wood says. “If a human being had been familiar with these results and had not credited them, then that would be malpractice.” She believes the community needs to urgently decide how to deal with AI’s failure to meet academic standards, as things are changing rapidly.
“Any mathematician who hasn’t used the latest models should be surprised,” says Wood. “It’s quite a different world than it was in December of last year.”