The FDA is approving an average of 69 medical AI devices per year, but about half of those approvals lack clinical validation data according to an analysis published in Nature. The various uses for Artificial Intelligence in medical settings are rapidly growing, and the FDA only approved an average of two AI devices per year before 2016. The study authors wrote, “Devices that lack adequate clinical validation pose risks for patient care. A new validation standard is proposed to evaluate FDA authorization as an indication of clinical effectiveness in medical AI.”

Researchers Sammy Chouffani El Fassi and Gail E. Henderson led an analysis of over 500 AI medical devices to find that about half lacked clinical scientific data to prove the devices are effective for human use. The authors stated, “Although AI device manufacturers boast of the credibility of their technology with FDA authorization, clearance does not mean that the devices have been properly evaluated for clinical effectiveness using real patient data. With these findings, we hope to encourage the FDA and industry to boost the credibility of device authorization by conducting clinical validation studies on these technologies and making the results of such studies publicly available.”

144 of the devices were “retrospectively validated,” which means the AI is given image data from the past. A better scientific approach is prospective validation, which provides real-time patient data. 148 of the devices in the analysis were approved with prospective validation. 22 of the devices were approved with randomized control trials, which is the gold standard.

The study’s authors said there is a need for the FDA to clearly define and distinguish between the three types of validation. Chouffani El Fassi added, “We shared our findings with directors at the FDA who oversee medical device regulation, and we expect our work will inform their regulatory decision-making. We also hope that our publication will inspire researchers and universities globally to conduct clinical validation studies on medical AI to improve the safety and effectiveness of these technologies. We’re looking forward to the positive impact this project will have on patient care at a large scale.”

The AI medical device market will grow from $15 billion in 2023 to $22 billion in 2024. Estimations say it will reach $97 billion by the year 2028. An executive order from October 2023 lays out concerns and hopes for the development of AI technology for a wide variety of industries.

226 out of 521 authorizations by the FDA lacked validation data from real patients. Some of them contained created images rather than data from real patients.

That executive order states, “Artificial Intelligence must be safe and secure.  Meeting this goal requires robust, reliable, repeatable, and standardized evaluations of AI systems, as well as policies, institutions, and, as appropriate, other mechanisms to test, understand, and mitigate risks from these systems before they are put to use.”

In response to that executive order, the FDA Center for Drug Evaluation and Research (CDER) is forming a new council that will oversee all AI-related technologies within the FDA. This will replace the current AI-related steering committee and additional AI-related working groups.

The council is co-led by three individuals within the FDA. Tala Fakhour, the associate director for data sciences and AI policy, has previously worked for the CDC and ICF International, a consulting firm. Qi Liu, the associate director for innovation and partnership in the office of Clinical Pharmacology, formerly worked for Merck. Sri Mantha, the director of the Office of Strategic Programs, has previously worked for multiple pharmaceutical companies including Pfizer and AstraZeneca.

Chouffani El Fassi said, “We shared our findings with directors at the FDA who oversee medical device regulation, and we expect our work will inform their regulatory decision-making. We also hope that our publication will inspire researchers and universities globally to conduct clinical validation studies on medical AI to improve the safety and effectiveness of these technologies. We’re looking forward to the positive impact this project will have on patient care at a large scale.”

A 2023 research paper outlines several drawbacks for using AI in the medical field, including concerns related to ethics, society, and privacy. AI devices in healthcare require algorithmic models that use a significant amount of private patient data. Researchers have warned that this is a privacy risk because this data can potentially be hacked.

Last year, the World Health Organization (WHO) warned that the rapid adoption of AI technology for medical purposes can cause harm to patients. The WHO said, “Precipitous adoption of untested systems could lead to errors by healthcare workers, cause harm to patients, erode trust in AI and thereby undermine (or delay) the potential long-term benefits and uses of such technologies around the world.”

FDA Director Robert Califf warned last year about the healthcare industry being “swept up quickly by something that we hardly understand.” In March, Dr. Califf spoke at the Coalition for Health AI and raised similar concerns. Califf said, “My concern is that our health systems do not have the infrastructure and tools to make the most important determinations about whether an AI application is “effective” for health outcomes. The ability of algorithms to provide accurate assessments will drift if left untended, often in unpredictable, and sometimes dangerous ways.”

Califf added that it is the responsibility of the manufacturer to ensure products are safe with oversight from the FDA. He said it is unclear how this system can work effectively for AI models that change and adapt over time.

“Not surprisingly, I’m hearing that the “effectiveness” metric being used by health systems to make decisions about incorporating an AI implementation is a financial metric,” Califf said. “Will the algorithm improve the bottom line of the part of the health system making the purchase?  I worry that the main use of AI algorithms will be decisions that optimize the bottom line rather than optimizing the longevity and well-being of patients.  This is counter to the mission of the FDA, where effectiveness means an improvement in a health outcome.”

The HighWire has been reporting about the ongoing concerns with AI, including a conversation between Jeffery Jaxen and Host Del Bigtree about whether AI is an existential threat. In an earlier episode, the Jaxen Report included a warning by Professor Stephen Hawking that AI could destroy mankind. In June, Jaxen discussed AI technology that utilizes human brain tissue.

Steven Middendorp

Steven Middendorp is an investigative journalist, musician, and teacher. He has been a freelance writer and journalist for over 20 years. More recently, he has focused on issues dealing with corruption and negligence in the judicial system. He is a homesteading hobby farmer who encourages people to grow their own food, eat locally, and care for the land that provides sustenance to the community.

Other Headlines

Coronavirus

Yale Study Finds Post-Vaccine Syndrome Shares Symptoms with Long COVID

A preprint study from Yale University researchers examined a condition known as post-vaccination syndrome. The study included 42 individuals with PVS and 22 without. Researchers observed elevated levels of spike proteins in the PVS group for 36 to 709 days because the time that lapsed since the last vaccination varied between participants. It is aContinue reading Yale Study Finds Post-Vaccine Syndrome Shares Symptoms with Long COVID

More news about Coronavirus

Health & Nutrition

USDA Implements New Avian Flu Protocols to Prevent Mass Culling of Chickens

New protocols are being implemented in the USDA regarding its response to the avian flu as Agriculture Secretary Brooke Rollins looks for ways to prevent the mass culling of chickens. Egg prices have continued to soar as millions of chickens have been culled due to avian flu concerns while some states have implemented new cage-freeContinue reading USDA Implements New Avian Flu Protocols to Prevent Mass Culling of Chickens

More news about Health & Nutrition

Vaccines

CDC Fails to Provide Evidence That DNA Integration is “Biologically Impossible” with mRNA Vaccines

The CDC could not provide any evidence or proof to support its statement that “COVID-19 vaccines do not change or interact with your DNA in any way.” This claim is made on the CDC page “Bust Myths and Learn The Facts About COVID-19 Vaccines.” The Informed Consent Action Network (ICAN) issued FOIA requests to obtainContinue reading CDC Fails to Provide Evidence That DNA Integration is “Biologically Impossible” with mRNA Vaccines

More news about Vaccines

Science & Tech

White Coat Waste Project Uncovers $10 Million in Taxpayer-Funded Transgender Animal Experiments

The White Coat Waste Project (WCWP) revealed that $10 million has been spent on taxpayer-funded studies that use mice and monkeys for transgender experiments, including surgeries and hormone treatments. Senator Rand Paul released his annual Festivus report of wasteful government spending days before Christmas and included $33.2 million in transgender monkey research. The HighWire reportedContinue reading White Coat Waste Project Uncovers $10 Million in Taxpayer-Funded Transgender Animal Experiments

More news about Science & Tech

Environment

East Palestine Residents Say Their Voices Are Silenced, Their Health Ignored

An NGO with ties to the Biden administration hand-selected the East Palestine moms who could speak at the Senate hearing, meet with President Biden, and speak with media outlets about the 2023 chemical disaster, according to independent testing expert and Government Accountability Project whistleblower client Scott Smith and residents in the village. Moms Clean AirContinue reading East Palestine Residents Say Their Voices Are Silenced, Their Health Ignored

More news about Environment

Policy

State AGs Launch Investigation Into Fauci Despite Biden’s Preemptive Pardon

South Carolina Attorney General Alan Wilson and a coalition of at least 16 other attorneys general are launching an investigation into potential criminal actions by Dr. Anthony Fauci. The AGs have contacted House Speaker Mike Johnson and Senate Majority Leader John Thune to request relevant information from their investigations. This investigation comes just weeks afterContinue reading State AGs Launch Investigation Into Fauci Despite Biden’s Preemptive Pardon

More news about Policy