OpenAI claims AI models solved complex math and physics problems

Claims of major breakthroughs

OpenAI recently announced a series of results suggesting its internal models have solved several high-level problems in mathematics and physics. As reported by Gizmodo, these claims include the supposed disproof of the Exponential Time Hypothesis, which is a fundamental idea in theoretical computer science regarding how long certain problems take to solve.

The company also claims to have resolved the Kakeya conjecture. This is a problem involving the study of waves and harmonics, specifically by finding a novel proof for all dimensions in what is known as the local smoothing conjecture. These results represent a significant leap in the types of complex reasoning the company claims its models can perform.

In the realm of physics, OpenAI asserts that its models have provided proofs for certain interacting quantum field theories. These theories contain gauge fields and fermions in four dimensions, which includes Quantum Chromodynamics and a unified completion of the Standard Model of particle physics. These are foundational concepts used to describe the basic building blocks of matter and forces.

The role of Lean verification

To support these claims, OpenAI states that the math results were formalized or verified using Lean. Lean is a proof assistant tool designed to help reduce the need for humans to manually check and recheck every step of a complex mathematical proof. By using a tool like Lean, the company aims to provide a higher level of certainty for the outputs generated by its models.

However, the use of Lean has already faced scrutiny from the research community. When OpenAI previously released a solution for the Navier-Stokes Millennium Problem, human readers expressed concern that the Lean verification actually confirmed a weaker statement than what was presented in the public blog post. This discrepancy highlights the difficulty of ensuring that automated tools correctly interpret the intended meaning of a scientific claim.

Because of these past issues, the community remains cautious about how much weight to give to automated proofs. The transition from a machine-verified string of logic to a human-understood scientific truth is a process that still requires significant oversight. OpenAI is now navigating these growing pains as it attempts to position its models as tools for high-level scientific discovery.

A new plan for scientific credit

OpenAI has acknowledged that using private internal models to achieve these results has faced pushback from advisory groups. To address this, the company stated it does not seek academic credit for the discoveries produced by its internal models. They are instead proposing a plan to disperse the credit and the work across the broader math and physics communities.

Under this new plan, OpenAI will release the model-generated texts as a form of raw research data. The company suggests that the task for the scientific community will be to convert this raw data into actual scientific understanding. This approach positions the AI as a generator of raw material rather than a finished product that is ready for immediate publication.

To facilitate this, OpenAI intends to reach out to external experts to act as shepherds. These individuals would be responsible for putting together groups that will take the AI outputs and turn them into new human knowledge. OpenAI admits that choosing these initial shepherds will necessarily shape who gets involved first and may create an asymmetry of information in the field.

Current status of the data

While some of the raw information from the 377 math results has been placed on GitHub, the release of the physics data is moving at a different pace. The group working on the physics results has decided to hold off on releasing the AI-produced artifacts for the time being. Instead, they plan to work on writing up the findings themselves before making them public.

This distinction between the math and physics results shows that the company is still determining the best way to handle different types of scientific data. Some results may be ready for immediate distribution as raw data, while others require more human intervention to become coherent. The goal remains to empower scientists with state-of-the-art capabilities while maintaining a responsible release of the models.

The scientific community will now have to evaluate these results as they trickle out. Because the AI-generated proofs are not yet peer-reviewed, the validity of the claims regarding quantum field theories and the Kakeya conjecture remains unverified. The next phase involves human experts determining if the 'raw data' provided by OpenAI actually holds up to the rigors of formal science.