Researchers at Beth Israel Deaconess Medical Center (BIDMC) have conducted a study comparing a chatbot’s probabilistic reasoning to that of human clinicians.
The findings, published in JAMA Network Open, suggest that artificial intelligence (AI) could serve as valuable clinical decision support tools for physicians.
Human Challenges in Probabilistic Reasoning
The study’s corresponding author, Dr. Adam Rodman, explained that humans often struggle with probabilistic reasoning, which involves making decisions based on calculating odds. Probabilistic reasoning is a crucial component of diagnosis but is challenging due to its complexity.
The researchers based their study on a national survey of over 550 practitioners who performed probabilistic reasoning on five medical cases.
They fed the Large Language Model (LLM), Chat GPT-4, the same cases and prompts, running the process 100 times to generate a range of responses.
The chatbot’s task was to estimate the likelihood of a diagnosis based on patient presentations.
It then updated its estimates when provided with test results, such as chest radiography for pneumonia, mammography for breast cancer, stress tests for coronary artery disease, and urine cultures for urinary tract infections.
Chatbot vs. Human Performance
When test results were positive, the chatbot demonstrated similar or better accuracy in making diagnoses compared to human practitioners. However, when test results were negative, the chatbot outperformed humans in all five cases.
One notable advantage of the chatbot was its tendency to maintain accurate diagnoses after negative test results.
Humans sometimes overestimate risk after negative results, leading to overtreatment, additional tests, and medication. The chatbot’s consistency in such scenarios may help reduce unnecessary interventions.
Future Directions
Dr. Rodman emphasized the potential of chatbots to support highly skilled physicians in their clinical decision-making.
While chatbots like LLMs don’t calculate probabilities in the same way as epidemiologists or poker players, they align more with how humans make spot probabilistic decisions.
Dr. Rodman highlighted the ease of integrating chatbots into clinical workflows and suggested that future research should explore how AI can complement human decision-making.
The goal is to leverage the strengths of both human and artificial intelligence to improve overall healthcare outcomes.
The study opens the door to exciting possibilities for using chatbots and AI as valuable tools in medical practice, aiding clinicians in making more informed and consistent diagnostic decisions.
If you care about cancer, please read studies about a new method to treat cancer effectively, and this low-dose, four-drug combo may block cancer spread.
For more information about cancer prevention, please see recent studies about nutrient in fish that can be a poison for cancer, and results showing this daily vitamin is critical to cancer prevention.
The research findings can be found in JAMA Network Open.
Copyright © 2023 Knowridge Science Report. All rights reserved.
