Have you ever used the artificial intelligence chatbot called ChatGPT for health advice?
Researchers at the University of Maryland School of Medicine (UMSOM) studied if the advice ChatGPT gives is accurate and reliable.
Their results show that most of the time, it is correct, but sometimes the advice can be wrong or even made up. Their study was published in the journal Radiology.
What the Study Did
In February 2023, UMSOM researchers asked ChatGPT 25 questions about getting screened for breast cancer. They asked each question three times to see what answers the chatbot gave.
Three mammography experts then reviewed the responses. They found that 22 out of the 25 questions got appropriate responses from ChatGPT.
However, there was one answer based on old information and two questions got very different responses each time they were asked.
“ChatGPT answered questions correctly about 88% of the time, which is pretty amazing,” said Paul Yi, the lead author of the study.
He also said that ChatGPT is good at summarizing information so it’s easy for people to understand. The AI accurately answered questions about breast cancer symptoms, who’s at risk, and advice on mammograms.
What Were the Problems?
However, the researchers noticed that ChatGPT is not as thorough in its answers as a Google search.
“ChatGPT only gave one set of recommendations on breast cancer screening from the American Cancer Society, but didn’t mention different recommendations from the Centers for Disease Control and Prevention or the U.S. Preventative Services Task Force,” said Hana Haver, another author of the study.
In one incorrect answer, ChatGPT suggested delaying a mammogram for four to six weeks after a COVID-19 vaccine, which is outdated advice.
The responses were inconsistent when answering questions about personal risk of breast cancer and where to get a mammogram.
“We found that ChatGPT sometimes invents fake journal articles or health organizations to back up its answers,” said Dr. Yi.
He reminded people that these AI tools are new and unproven, and for health advice, they should still rely on their doctor and not ChatGPT.
What’s Next?
Now the researchers are seeing how well ChatGPT does with advice on lung cancer screening. They’re also looking at how to improve the AI’s advice to make it more accurate, complete, and easy to understand for everyone.
Mark T. Gladwin, Dean at UMSOM, said, “As ChatGPT and similar AI technologies evolve quickly, we in the medical community need to test these technologies and protect our patients from any harm that might come from incorrect screening recommendations or outdated health strategies.”
If you care about health, please read studies about why beetroot juice can strongly boost muscle force in exercise, and vitamin D supplements strongly reduces cancer death.
For more information about health, please see recent studies that olive oil may help you live longer, and these antioxidants could help reduce dementia risk.
The study was published in Radiology.
Copyright © 2023 Knowridge Science Report. All rights reserved.
