Article 321: Microsoft's MAI-DxO Beat Generalists 83% to 36% on Hard Cases. A Specialist Team Was Never in the Room

The study, published Thursday in Science, is straightforward in its headline: Microsoft's MAI-DxO — a system that coordinates several AI agents acting as a panel of doctors, questioning a patient, ordering tests, and debating a diagnosis — was correct 83% of the time on complex New England Journal of Medicine cases. Primary care physicians and general internists, working individually with reference materials, were correct 36% of the time.

That is a greater-than-twofold gap, and Mustafa Suleyman called it a step toward "medical superintelligence," which Microsoft defines as surpassing the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care.

Read the asterisk first

The comparison group is the story. Cases that complex are, in real hospitals, handled by specialists or physician teams — and those were not in the study. The 36% is not a measurement of medicine. It is a measurement of what a single generalist achieves when handed a case designed to be hard, with no colleagues to consult, no specialist referral, and reference materials. It is a fair benchmark of the model against the "one doctor alone" baseline. It is not a benchmark against the care a patient actually receives.

The study's own data makes the second caveat harder to ignore. Microsoft tested DxO with OpenAI, Google, Anthropic, Meta, and DeepSeek models, and the best results came with OpenAI's GPT-5 and GPT-5.5. The numbers split badly by workload:

• GPT-5, complex cases: 78.9% alone at roughly $8,400 per case. DxO cut about $3,000 of cost and added about four points of accuracy.
• GPT-5.5, complex cases: 72% alone → nearly 87% with DxO, at slightly cheaper cost.
• Emergency department cases: no meaningful gain and no meaningful cost saving from DxO, for either model.
• Gemini 2.5 Flash: 46.4% alone → nearly 69% with DxO — a big lift, but still below what stronger models reach alone.

The pattern: orchestration helps most on complex cases and on weaker models, and the gap between weak and strong models is bigger than the lift orchestration provides. Microsoft's spokesperson concedes the point about diminishing returns — gains shrink as models approach the ceiling.

What DxO actually buys

The strongest reading of this study is not "AI beats doctors." It is that a multi-agent panel converts model capability into accuracy in a way that is measurable in dollars. Cutting $3,000 off an $8,400-per-case run while adding four points is a real result even if the four points are modest, because in clinical decision support the cost of a case run is the thing that determines whether it gets deployed at all.

The weaker reading, and the one the data supports: much of the headline improvement is the underlying model getting better, and Microsoft is measuring its orchestration on top of a moving floor.

Key Takeaway: The right way to hold this study is as a cost-and-complexity result, not a capability verdict. Two things follow. First, the real-world test is the Beth Israel Deaconess partnership Microsoft announced alongside the paper — patients get standard care, consenting patients also engage DxO in parallel, and recommendations get compared against human doctors' in an actual Boston emergency department. That is the study that will matter, and King's own framing ("the complexity and diversity of issues that come from a Boston emergency department on a Friday evening") is the honest admission that the simulation was the easy part. Second, the emergency-department null result is the most informative line in the piece. Complex diagnosis is a reasoning problem that scales with model quality; emergency triage is a time-and-triage problem where more deliberation does not obviously help. If you are deploying this, you deploy it where the case is hard and the clock is slow — which is also where a specialist referral would normally go. That overlap is the unresolved question the paper does not answer.

Sources: Science, "MAI-DxO" study, 22 October 2026; TechCrunch, "Microsoft's Mustafa Suleyman says its AI outperforms doctors on hard cases," 22 October 2026; Microsoft AI, "Building medical superintelligence"