Microsoft Says Its AI Diagnosed Patients Four Times Better Than Doctors

Microsoft says it has built an AI system that diagnoses medical conditions far more accurately than human doctors, and it has a peer-reviewed study to back up the claim.

The research, published Thursday in Science, pits Microsoft's AI tool against physicians on real medical scenarios pulled from case reports and emergency room records. The company calls the tool the Diagnostic Orchestrator. Rather than a single chatbot spitting out a guess, it runs multiple AI agents that work through symptoms and patient history, decide which tests to order, and converge on a diagnosis, mimicking the back-and-forth reasoning process doctors actually use.

"It is finally here," Microsoft AI CEO Mustafa Suleyman told The Verge, referring to the long-promised idea that AI would meaningfully improve healthcare.

The numbers behind the claim

Microsoft built its own benchmark for the study, called SDBench, which it plans to open-source. On cases drawn from hundreds of reports published in The New England Journal of Medicine, the Diagnostic Orchestrator hit roughly 83 percent diagnostic accuracy. Generalist physicians working with reference materials managed about 36 percent; those working without references scored around 22 percent.

The system also outperformed doctors on emergency department cases pulled from a large patient-admissions database at Beth Israel Deaconess Medical Center in Boston, scoring 80 percent accuracy against an average of 47 percent for human physicians. Microsoft says the AI tended to reach its conclusions at a lower estimated cost than its human counterparts, too.

The Diagnostic Orchestrator runs on OpenAI's GPT-5, and the researchers say it also beat frontier AI models working solo, not just human doctors. Worth noting: the peer-reviewed paper doesn't include comparisons against newer models like GPT-6 Astra, Meta's Muse Wonder Full, Google's Gemini 4 Argon, or Anthropic's Opus 5.5. The system was originally built back when GPT-4.1 was state of the art, and Microsoft's researchers say later testing suggests that smarter underlying models don't make the multi-agent approach obsolete, the benefits of having several agents collaborate hold up even as the base model improves.

Still a long way from the clinic

Microsoft is upfront that this is a long way from something you'd find in an actual hospital. The test setup mirrors real diagnostic work more closely than a typical AI benchmark, with the system asking questions, weighing competing explanations, and ordering tests before settling on an answer. But it's all been run against historical cases on paper. Nobody knows yet how it performs with a real patient in front of it.

"It needs clinical validation," Suleyman said. "That is a leap, but this is a major milestone along that journey."

Microsoft's health AI lead and study coauthor, Dr. Dominic King, says the company plans to start testing the system in actual hospital settings, working with Beth Israel Deaconess Medical Center to see how it holds up in live clinical environments. Neither King nor Suleyman committed to a specific date, though King said the trial would likely "get underway in the next couple of months."

Part of a bigger bet

The diagnostic work is one piece of a larger project Microsoft is calling "medical superintelligence," its term for AI systems that could eventually outperform the best human-run healthcare operations. (Microsoft was using "superintelligence" before the term became a broader buzzword in AI and political circles.) King says diagnosis is just one of six areas the company believes AI needs to master, according to a blog post and a companion article in the BMJ's Digital Health & AI journal. The others reportedly include coordinating clinical care, making treatment decisions, and handling the more human-facing side of medicine, things like communication and bedside manner.

Commercialization is on the table too, though Microsoft hasn't said when or how the tool might actually reach patients or providers. The company already has deep ties to US healthcare through its enterprise software, and health-related questions are among the most common uses of its consumer chatbot, Copilot. King says any rollout will need to clear the high regulatory and compliance bars that govern medical technology.

Suleyman is betting the pace of progress will be fast. "Next year we're going to see extremely impressive deployments of intelligent systems in production," he said. "I think that this is just going to change the way care is delivered in a big way."

He's also framed this within a broader "human-centered" philosophy he's talked about before, insisting AI diagnosis tools are meant to support doctors, not replace them. He acknowledged the shift could end up piling more work onto clinicians rather than less, but says his hope is that AI strips out some of the administrative grind so doctors can spend more actual time with patients.

"I'm very optimistic about what's coming next," Suleyman said. "I think this is why we build technology."

The real test now isn't the benchmark. It's whether the Diagnostic Orchestrator's numbers hold up once it's making calls on live patients instead of archived case files, something Microsoft itself admits it hasn't proven yet.