Microsoft develops AI tool that can diagnose ‘sequentially’ like a doctor

Microsoft researchers say an AI tool that diagnoses patients step by step, as doctors do, was more than twice as accurate as general physicians at solving complex medical cases, according to a study published Thursday in Science.

The model identifies and coordinates the questions to ask a patient, as well as the exams and tests to order, using an "orchestrator" that organises how AI tackles a problem by employing a simulated panel of clinical experts collaborating to solve diagnostic cases, according to the report.

Existing medical AI models typically take an all-at-once approach, examining a patient's symptoms, lab results, and imaging studies to generate a diagnosis.

With the orchestrator coordinating the steps to reach a diagnosis, the AI model first decides what to ask about the patient’s history and what ​in the physical exam is significant. Based on what it learns, it decides which tests to order, considers the results, and orders further testing as appropriate.

All the while, it updates its working hypotheses and plans, the research leaders said in an interview.

Tasked with diagnosing more than 300 highly unusual and confusing medical dilemmas, the ​model employing the diagnostic orchestrator was more than twice as accurate at "working up" the patient and determining the diagnosis ‌as ⁠general physicians using textbooks and the internet.

The tool also outperformed human physicians on scenarios drawn from a major database of emergency room admissions.

Assessments comparing these types of older systems to human doctors usually use multiple-choice questions, the researchers said.

"AI has become very good at getting a 100% ​pass rate on those,” ​said Dr. Dominic King, ⁠the corporate vice president of the health unit at Microsoft AI, whose team is developing the new technology.

“But of course, that's not real-world. No ​patient ever came to see me with a nice summary.”

The researchers also ​found that ⁠diagnostic processes cost less when the orchestrator designs the work-ups than when human doctors do.

“We're not just measuring the accuracy of the systems. We're also measuring how judiciously and carefully it goes through a ⁠diagnostic ​work-up,” said Harsha Nori, head of AI at Microsoft AI ​Health.

In a collaboration with Beth Israel Deaconess Medical Center in Boston, the researchers will next test the tool’s ability to ​triage patients in emergency department waiting rooms.