Microsoft AI tool outperforms generalist doctors in complex diagnoses

A new approach to diagnosis

Microsoft has developed a new artificial intelligence tool called the MAI Diagnostic Orchestrator, or MAI-DxO. This system was designed to solve complex medical cases by mimicking the way a panel of clinical experts would approach a patient. Instead of looking at every piece of data simultaneously, it follows a sequential logic to find answers.

Most existing medical AI systems work by assessing symptoms, laboratory results, and scans all at once to produce a result. MAI-DxO differs because it works through a case step by step. It determines which specific questions to ask and which tests to order before it reaches a final diagnosis.

As reported by The Next Web, the tool is capable of gathering information and reviewing results as they come in. It then revises its working diagnosis as new evidence emerges during the process. This mimics the iterative nature of real-world medicine where a doctor's initial theory changes as more data becomes available.

Comparing AI to human doctors

The effectiveness of MAI-DxO was tested in a peer-reviewed study published in the journal Science. Researchers presented the tool with 304 challenging cases originally published in the New England Journal of Medicine. The AI reached the correct diagnosis in 83.2% of those cases, showing a significant lead over human performance.

In the same study, generalist doctors were given the same cases to solve. These doctors were allowed to use the internet and medical textbooks to help them find the correct answers. Despite these resources, the human group only scored 36.1% on the complex cases, meaning the AI was more than twice as accurate as the human participants.

The tool also showed strong results in a separate test involving 325 emergency department admissions. In this scenario, MAI-DxO scored 80.3% compared to a 46.8% score for the human group. These results suggest that the AI's step-by-step reasoning may be particularly effective in high-pressure, high-complexity medical environments.

How the system functions

The underlying technology for MAI-DxO currently runs on OpenAI’s GPT-5. However, the system is flexible and can be adapted to work with different models. Earlier research showed that the orchestrator could be improved using models from other companies, including Google, Anthropic, and Meta.

One of the primary goals of the researchers was to measure how effectively the system selected specific tests. They did not want to simply see if the AI could reach the right answer by guessing. They wanted to see if it could make the correct clinical decisions regarding which pieces of information were necessary to gather next.

Dominic King, a corporate vice president in Microsoft AI’s health unit, noted that many AI evaluations rely on multiple-choice tests. He stated, "AI has become very good at getting a 100% pass rate on those," but noted that patients do not see a summary of those tests in real life. MAI-DxO aims to provide a more useful clinical work-up than a simple summary.

Limitations and future plans

While the results are promising, the study had several important limitations that readers should keep in mind. The doctors in the comparison were generalists rather than specialists in specific fields. Furthermore, the cases were presented as text, meaning neither the humans nor the AI had access to physical examinations or medical imaging.

The study also provided estimates on the cost of the diagnostic work-ups. Researchers found that the AI's work-ups cost less than the human ones, though these figures are based on the cases rather than actual medical bills. These estimates provide a preliminary look at how the tool might impact the financial side of healthcare.

MAI-DxO is not currently being used in clinical practice. Microsoft's researchers are planning to work with Beth Israel Deaconess Medical Center in Boston to begin testing the tool. The goal of this partnership is to see if the system can help triage patients who are waiting in emergency department waiting rooms.