Regulatory authorities such as the U.S. Food & Drug Administration (FDA) and the European Medicine Agency (EMA) are heavily regulating the medical landscape when it comes to artificial intelligence (A.I.)-based solutions. The FDA, in particular, took the lead, having issued a specific framework for A.I.-based algorithms.

However, even this leading authority, like other regulatory bodies, does not provide a comprehensive database of these tools that it has approved. One can expect such information to be readily available, especially for a prominent technology like A.I. But the very authorities regulating the landscape failed to provide it.

It’s a booming market and it’s not because of its hype. The number of life science studies published around A.I. rose from 596 in 2010 to 12,422 in 2019. Such smart algorithms promise to augment medical practice from eliminating alarm fatigue through improving prosthetics to managing a pandemic.

Moreover, according to forecasts, the global market size for A.I. in healthcare will soar past its $1 billion valuation in 2016 to $28 billion in 2025. This trend is not showing any signs of slowing down as we march steadily into the A.I. era of healthcare. As such, expectations are high within the medical community.

The TMFI study

Given the importance of A.I. in healthcare, companies have a tendency to overuse the term when describing their solutions. They label their devices or software as A.I.-based when in fact this is not the case. This is a common move to attract investors and to boost the company’s public image. But even major regulatory bodies have shown lacunae in making information on credible A.I.-based medical tools available.

The latest peer-reviewed paper from The Medical Futurist Institute (TMFI) analysed the state of regulation over A.I.-based algorithms. Using the FDA as an example, the authors even pioneered the first open access, online database of FDA-approved A.I.-based algorithms, which the U.S.-based regulatory body should have come up with already.

The results of this study led by Dr. Meskó were recently published in npj Digital Medicine, the first TMFI study to be published in this prestigious journal. Here we break down the main findings, and you can also read the full, open-access paper on npj Digital Medicine.

How the FDA regulates A.I.-based algorithms in healthcare

While novel, A.I.-based medical devices show promise in enhancing medicine, they are inevitably high-risk in nature. There are several unknown consequences of using A.I. in medical decision-making and data analysis. As such, the FDA imposes strict regulatory requirements to licence such medical devices.

Prior to legally marketing their medical hardware or software in the U.S., the company must submit it to the FDA for evaluation. Medically-oriented A.I.-based algorithms have 3 levels of clearance and they must meet specific criteria requirements to be granted a clearance. These clearance levels are 510(k), premarket approval (PMA) and the de novo pathway.

An algorithm gains 510(k) clearance if it is shown to be at least as safe and effective as another similar, legally-marketed algorithm. PMA is issued to algorithms for Class III medical devices, or those with a large impact on human health. Their safety is determined after confirming that their effectiveness is supported by satisfactory scientific evidence. As for the de novo pathway, it relates to novel medical devices for which there are no legal counterparts on the market. General controls offer reasonable guarantee of their safety and effectiveness.

However, despite the FDA’s thorough regulatory processing, Meskó et al. found several issues with its method as well as with accessing relevant information to its approved medical A.I.-based tools.

Issues with the FDA’s regulation

Even if it has strict requirements before a tool is given clearance, the FDA does not ask companies to categorise their technology as A.I.-based. Moreover, its official approval announcements do not clearly state the use of these methods either. This showed a blatant lack of clarity from the leading regulatory body itself regarding a prominent technology. Further clouding clarifications on such matters is the FDA’s official website. Its search engine does not have any feature allowing users to search for specific queries in FDA announcements and summaries. This severely hampers the accessibility of the database.

Nevertheless, the FDA is not the only one with such issues. One can expect regulatory authorities to provide clear descriptions of the devices they regulate, and offer a properly searchable database to assess the implementation of new techniques. Unfortunately, no regulatory agencies offer these possibilities.

With the FDA taken as an example given that it took the leadership in A.I.-based medical device regulation and has the necessary toolkit to assess the credibility of these tools, the purpose of the new paper was three-fold:

1. To provide an insight into the currently available A.I-based medical devices and algorithms approved by the FDA.

2. To create an up-to-date database of FDA-approvals in this field which is open to submissions and might serve as the database that the FDA should have.

3. To raise awareness of the importance of regulatory bodies clearly stating whether a medical device is A.I.-based or not.

Building the first open access database of FDA-approved A.I.-based algorithms

The research team, composed of Dr. Stan Benjamens, Dr. Pranavsingh Dhunnoo and Dr. Bertalan Meskó, took it upon themselves to gather all of the A. I.-based medical tools approved by the FDA into a comprehensive online database. Since the U.S.-based regulatory body does not require a clear definition for these solutions, the study authors agreed on one. They classify a technology as A.I.-based if its development includes a form of machine learning, a computer-based method to create algorithms based on structured databases. A more advanced subtype of machine learning, deep learning, a deep neural network (DNN)-based methodology, for pattern recognition, is also common in A.I.-based technologies.

Since A.I.-based technologies in medicine started gaining traction in the mid-2010s, the authors combed FDA announcements between January 2010 and March 2020. They found 64 A.I.-based, FDA-approved medical devices and algorithms during this timeframe. Out of those, only 29 mentioned any A.I.-related expressions in the official FDA announcement. The remaining 35 were described as A.I.-based technologies on other online sources.

From these results, Meskó et al. have built an online database where the authors independently reviewed each entry. To help in visualising the results, they also created an infographic, which you can see below:

The infographic contains each approved device’s name, a short description, its relation to a primary and a secondary medical specialty and its type of FDA clearance. The same colors are assigned to the same medical specialty.

An open invitation

Creating the database and raising awareness of its importance were only the beginning. It features a submission option where the community can submit FDA-cleared A.I.-based medical solutions that came out after the study’s completion or even if the authors missed any. The authors also cross-check and verify all submissions before making any additions.

Moreover, the authors further encourage regulatory bodies to take over this database or launch their own since they possess the resources to better maintain such a tool. No such database exists other than the one created for this pioneering study.

As A.I.-based devices and algorithms for medical purposes are becoming integral parts of the healthcare landscape, the importance of such an easily accessible and informative database will become more apparent. Regulatory authorities have the duty to maintain one to allow a better oversight of the credible A.I. tools managing our health.

Get the week's top news shaping the future of medicine

Get the week's top news shaping
the future of medicine