Skip to content

Building Medical Superintelligence | Microsoft AI

Microsoft AI

The multi-agent orchestrator achieved 83% accuracy on complex NEJM cases versus 36% for physicians with reference materials, while reducing diagnostic costs.

Key points

  • SDBench tests multi-step diagnostic reasoning, evidence requests, and costs across 304 NEJM cases and MIMIC-IV-ED emergency data.
  • MAI-DxO coordinates virtual physician agents that debate diagnoses and weigh test value against expense over multiple rounds.
  • On typical emergency department cases from MIMIC-IV-ED, MAI-DxO reached 80% accuracy compared to 47% for physicians.
  • Microsoft open-sourced SDBench benchmark, Gatekeeper, Judge, and cost-estimation code for select NEJM cases.
  • Microsoft partnered with Beth Israel Deaconess Medical Center to evaluate the system directly with patients in prospective hospital studies.

Why it matters

Medical AI researchers gain a standardized sequential benchmark and multi-agent architecture designed to improve clinical accuracy while lowering the cost of diagnostic workups.

Sources