Artificial intelligence (AI) is becoming increasingly important in healthcare, with applications ranging from helping doctors diagnose diseases to improving the accuracy of surgeries.
However, a new study has raised concerns about the safety and effectiveness of many AI medical devices.
The research, conducted by a team from several institutions including the UNC School of Medicine and Duke University, found that almost half of the AI devices approved by the U.S. Food and Drug Administration (FDA) were not tested on real patient data.
This study, published in Nature Medicine, analyzed over 500 AI medical devices that had received FDA approval.
The researchers discovered that approximately 43% of these devices lacked published clinical validation data, meaning they were not proven to be effective using real patient information.
Instead, some of these devices were tested using “phantom images” or computer-generated data, which do not meet the requirements for proper clinical validation.
“Although manufacturers of AI devices often highlight their FDA approval, this doesn’t necessarily mean that the devices have been thoroughly evaluated for effectiveness in real-world scenarios,” said Sammy Chouffani El Fassi, a medical student at the UNC School of Medicine and lead author of the study.
“We hope our findings encourage the FDA and the industry to improve the credibility of AI devices by ensuring they are properly tested on real patient data.”
The number of AI medical devices being approved by the FDA has skyrocketed in recent years. Since 2016, the average number of approvals per year has increased from just two to 69.
These devices are being used in various areas of healthcare, such as diagnosing diseases from X-rays and predicting how a disease might progress.
AI works by using algorithms—complex sets of rules and calculations that allow the technology to learn from large amounts of data.
For AI medical devices to be truly effective, they must be tested on new, real-world data that the AI has not seen before. This ensures that the AI can accurately analyze and provide results in real-life situations.
The research team was curious about how well these AI devices were being tested, especially as their use in healthcare continues to grow. They looked at all the available submissions in the FDA’s database for AI-enabled medical devices.
Of the 521 devices approved, 144 were validated using retrospective data, meaning they were tested on past patient data.
Another 148 were tested prospectively, meaning they were tested on real-time data from current patients. Only 22 devices were tested using randomized controlled trials, which are considered the gold standard for clinical validation.
The lack of real patient data in testing raises concerns about the safety and effectiveness of these AI devices.
The researchers recommend that the FDA and device manufacturers clearly differentiate between different types of clinical validation studies and ensure that all AI devices undergo proper testing before being approved for use in patient care.
The team hopes that their findings will lead to stronger regulations and encourage more thorough testing of AI medical devices.
“We shared our findings with the FDA, and we hope our work will help improve their regulatory decisions,” said Chouffani El Fassi. “We also want to inspire other researchers and institutions to conduct clinical validation studies on AI medical devices to ensure their safety and effectiveness.”
In addition to analyzing AI devices, Chouffani El Fassi is also working on a project to improve the organ donation process using AI. By developing an algorithm that automates the evaluation and referral of organ donors, the team at UNC Health aims to make the process more efficient and save more lives.
“If this algorithm works, it could make a huge difference in the number of successful organ transplants,” Chouffani El Fassi explained.
While AI has the potential to revolutionize healthcare, this study highlights the importance of rigorous testing and validation to ensure that these technologies truly benefit patients.
With proper oversight and continued research, AI can be a powerful tool in improving healthcare outcomes.
If you care about medicine, please read studies that vitamin D could help lower the risk of autoimmune diseases, and drug for inflammation may stop spread of cancer.
For more health information, please see recent studies about which drug can harm your liver most, and results showing this drug can give your immune system a double boost against cancer
Source: University of North Carolina.
