In a groundbreaking study, researchers have discovered an interesting twist in the way artificial intelligence (AI), like ChatGPT, handles health-related questions.
As it turns out, giving more detailed evidence to ChatGPT can actually make its answers less reliable.
In some cases, the accuracy of its responses dropped to a startling 28%.
This study was unveiled at a major conference dedicated to the study of how computers process human language.
The team behind the research included experts from Australia’s national science organization and The University of Queensland.
They put ChatGPT to the test by asking it 100 different health-related questions. These ranged from common queries about the effectiveness of zinc against colds to more peculiar ones like whether drinking vinegar can help with a fish bone stuck in the throat.
The findings of this study are significant. They come at a time when more and more people are turning to online tools like ChatGPT for health advice.
While these AI tools are becoming popular for their ease of use and access to information, there’s a growing concern about the reliability of the health information they provide.
In the study, ChatGPT was presented with questions in two ways. One set included just the question, while the other set was loaded with additional evidence that either supported or contradicted the question.
Interestingly, ChatGPT performed well when just the question was asked, with an 80% accuracy rate. However, when additional evidence was included, its accuracy dropped significantly. This decrease in reliability was even more pronounced when ChatGPT was allowed to express uncertainty in its answers.
The reason behind this decrease in accuracy isn’t entirely clear. The researchers suggest that the extra information might be confusing the AI, making it harder for it to provide the correct answer.
This study highlights the importance of continuing to research and understand how AI like ChatGPT works, especially as it becomes a more common tool for seeking health information online.
With major search engines beginning to combine AI and search technologies, there’s a critical need to ensure that the health information provided is accurate and reliable.
Looking forward, the researchers plan to explore how the public uses the health information generated by AI tools. Their goal is to better understand the impact of these technologies on public health and ensure that people can trust the information they receive.
If you care about wellness, please read studies about how ultra-processed foods and red meat influence your longevity, and why seafood may boost healthy aging.
For more information about wellness, please see recent studies that olive oil may help you live longer, and vitamin D could help lower the risk of autoimmune diseases.
The research findings can be found in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.
Copyright © 2024 Knowridge Science Report. All rights reserved.
