South SFV Today

Artificial intelligence chatbots are increasingly being integrated into medical decision-making processes. A recent study, led by Jonathan H. Chen, MD, PhD, an assistant professor of medicine, explored whether AI chatbots could effectively assist physicians in answering nuanced clinical questions that lack clear-cut answers.

The research found that chatbots alone outperformed doctors who relied solely on internet searches and medical references. However, when physicians were supported by a chatbot, their performance matched that of the chatbot operating independently.

“For years I’ve said that, when combined, human plus computer is going to do better than either one by itself,” Chen noted. He added that the study challenges traditional thinking about the roles of humans and computers in clinical settings.

The study was published in Nature Medicine on February 5th and involved multiple researchers including Adam Rodman, MD from Harvard University as co-senior author along with postdoctoral scholars Ethan Goh, MD and Robert Gallo, MD as co-lead authors.

A previous study conducted in October 2024 demonstrated higher accuracy rates for chatbots over doctors even when they used AI assistance for disease diagnosis. The current research focused on "clinical management reasoning," which involves more complex decision-making scenarios.

Ethan Goh explained this complexity using an analogy: using a map app to find a destination versus deciding how best to reach it given varying circumstances like traffic or road conditions.

In their trial design, the team assessed three groups: a standalone chatbot; 46 doctors with chatbot support; and another group of 46 doctors with access only to internet resources. The decisions made were evaluated against a rubric developed by board-certified doctors.

To their surprise, results showed that the standalone chatbot outperformed those without AI support but performed equally well alongside doctor-chatbot collaborations. The reasons behind these outcomes remain open for further investigation according to Chen.

The implications raise questions about AI's future role in healthcare: “Perhaps it’s a point in AI’s favor,” said Chen while emphasizing the importance of discernment between credible information sources over complete reliance on technology alone.

This research involved contributions from VA Palo Alto Health Care System among others including Microsoft; funding came through various institutions such as Gordon & Betty Moore Foundation alongside Stanford Clinical Excellence Research Center amongst others mentioned earlier within this context too!