Microsoft has detailed a medical artificial intelligence system that outperformed practicing generalist physicians in diagnostic accuracy during simulated complex medical cases. The system, known as the Microsoft AI Diagnostic Orchestrator or MAI-DxO, achieved an 83.2 percent diagnostic accuracy rate in tests involving complex cases published in The New England Journal of Medicine. By comparison, generalist physicians who had access to reference materials achieved a 36.1 percent accuracy rate.
When physicians were denied reference resources during the evaluation, their accuracy dropped to roughly 22 percent. In a separate test focusing on emergency department admissions, the Microsoft system reached 80.3 percent accuracy, while generalist physicians scored 46.8 percent. The findings were officially published in the peer-reviewed journal Science after initially appearing as a preprint on arXiv last year.
Unlike tools that ingest a complete medical history and list of symptoms all at once, MAI-DxO operates sequentially. It begins with a chief complaint, determines which follow-up questions to ask, and selects necessary tests before arriving at a final conclusion. The architecture relies on multiple AI agents assigned to different roles, such as proposing diagnoses, selecting tests, challenging assumptions, and weighing costs, which researchers note helps counter cognitive biases like anchoring.
Microsoft's AI health lead, Dr. Dominic King, stated that clinical testing of the tool will begin at Beth Israel Deaconess Medical Center in Boston within the next couple of months. While the system demonstrated high accuracy and ordered fewer procedures in simulated environments, medical experts emphasize that these tools are intended to assist human professionals rather than replace them. Widespread clinical adoption will also need to navigate complex legal questions regarding accountability when diagnostic errors occur.
Microsoft Medical AI Outperforms Generalist Doctors in Diagnostic Accuracy Study
Comments