Harvard study in Science: OpenAI o1 outperforms ER doctors in emergency diagnosis accuracy

A study published in Science (DOI: 10.1126/science.adz4433) by researchers at Harvard Medical School, Beth Israel Deaconess Medical Center, and Stanford found that OpenAI's o1 model achieved exact or very close diagnosis in 67% of 76 real emergency room triage cases, compared to 55% and 50% for two attending physicians. The AI was given the same unprocessed electronic medical record data as doctors. The o1's accuracy rose to 82% as more information became available. In treatment plan tasks, o1 scored 89% versus 34% for 46 doctors using conventional resources. The authors urged urgent prospective clinical trials before deployment.