OpenAI releases BioBasicsBench to test AI on messy wet-lab data

Testing the messy reality of science

Scientific research in a physical laboratory, often called a wet-lab, is rarely as clean as a textbook. Researchers frequently encounter failed experiments, ambiguous images, and conflicting observations that require human intuition to navigate. OpenAI recently released BioBasicsBench to see if artificial intelligence can handle these specific types of hurdles.

The benchmark was developed in partnership with Ligo Biosciences, an enzyme-design startup. It consists of 100 routine lab tasks designed to test whether models can interpret raw data, catch flawed controls, or recognize when evidence is too thin to support a conclusion. This differs from standard tests that use polished charts or images selected for publication.

To ensure the benchmark is rigorous, the team turned real experimental data into test questions. Each question underwent three separate reviews: one by a Ligo scientist, one by a PhD-level scientist from an outside lab, and a final internal review. This process ensures the scoring criteria reflect the actual standards of professional science.

How the benchmark evaluates models

OpenAI tested seven of its own models across two generations, ranging from the GPT-5 line to the more recent GPT-6 series. The testing involved 37 different model-and-setting combinations, which controlled how much effort the model spent thinking before providing an answer. In total, the company graded 18,500 answers across the various tasks.

The scoring system rewards models for several specific elements. Points are awarded for providing accurate measurements, demonstrating sound reasoning, and correctly identifying uncertainty. This helps researchers see not just if a model gets the right answer, but if it follows a logical path to get there or if it is simply guessing based on patterns.

The results highlight the strengths of the newer Astra model. At the 'xhigh' reasoning setting, Astra earned 66.7% of the available credit. Notably, it achieved this while using about half the output tokens of the GPT-5.6 Sol model, suggesting that stronger native multimodal capabilities might be improving its efficiency.

Successes and failures in the lab

Despite the progress, the benchmark revealed that AI models still exhibit uneven performance in wet-lab settings. A common issue identified by OpenAI is that models often pair a correct observation with a wrong final call. For example, a model might correctly identify a pattern in a gel but then reach the wrong conclusion about what that pattern means for the experiment.

One specific example involved a DNA cloning task where a model correctly sorted colony lanes into two groups. However, it then assumed a larger product meant successful cloning and recommended colonies that still carried the original plasmid. It also overcounted the colonies on one plate by about 12%, which directly inflated the estimated efficiency of the DNA entry.

On the other hand, Astra showed impressive attention to detail in other scenarios. In one cloning task, it correctly rejected a plate as a valid control. It did this by reading a handwritten label on the rim and noticing that the plate contained the wrong antibiotic for the specific experiment being conducted.

The path toward scientific discovery

The creation of BioBasicsBench reflects a growing focus on science at OpenAI. CEO Sam Altman recently noted that the company's progress in complex math problems suggests they are closer to creating a machine for scientific discovery than previously expected. This move toward specialized scientific reasoning is part of a broader trend in AI development.

Other companies are also beginning to test AI's reasoning capabilities in the lab. Benchling, a laboratory data software provider, has built its own tests for AI lab reasoning. Their tests include planning cloning experiments, where the Astra model also achieved the highest scores among the models tested.

Industry experts suggest that while these models are not ready to work alone, they have significant potential. Nicholas Larus-Stone of Benchling told R&D World that LLMs can work in the wet lab if they are provided with the right harness and a human remains in the loop. This suggests a future where AI acts as a sophisticated assistant to help scientists process messy data more quickly.