AI and the Artificial Intelligence Act: Driver of innovation faced with new regulations – what now?

AI methods are also being used increasingly frequently in other processes, such as diagnosis or treatment. Christopher Boss, director of MedtecLIVE with T4M and Executive Director Exhibitions of NürnbergMesse GmbH explains, “The topic will also be in focus and further developed in the exhibition forum in the coming spring.” Not without reason: with the help of AI, mistakes can be reduced and the use of medical technology optimised. However, as with every innovation, there are not only advantages, which is why in future, the use of AI will be regulated and harmonised to create consistent standards across the EU.

The EU Commission’s “Artificial Intelligence Act” (AIA) is intended to introduce regulations on the use of artificial intelligence. A clear legal framework will enable orientation and classification of any misconduct of the AI. However, regulations, in particular overregulation, which is feared as a result of the AIA, may also make it more difficult for new developments to enter the market and inhibit the development of innovations. Indeed, in the case of AI in medical technology, regulations already exist: The EU Medical Device Regulation (MDR). A clear legal framework is intended to enable orientation and classification of any misconduct of the AI. “The AIA is a horizontal regulation, i.e. independent of sector, and applies equally to computer games and medical technology. Thus, there would be double regulation in combination with the MDR,” explains Natalie Gladkov, Consultant for Digital Medical Devices at the German Medical Technology Association.

Exactly what effects does the AIA have on developments which use artificial intelligence, and what consequences would the new certification regulations have? In order to answer this question, it is first necessary to answer a fundamental question: What is AI, what does it do, and how does it work?

Artificial Intelligence: who are you?

The term sometimes brings scenarios from Hollywood films to mind: an artificial intelligence created by humans develops a life of its own and takes over the world. Luckily, this is not how it works in reality, explains Abtin Rad, Global Director for Functional Safety, Software and Digitalisation at TÜV SÜD: “AI is the capacity of algorithms to imitate human abilities. There are various types, such as machine learning for example, where a written algorithm learns from training data.” According to Rad, the implementation is more complicated and complex, but essentially, artificial neural networks are created which work in a similar way to the human brain and can be trained with data in order to make decisions. This is the method which is used most frequently in medical technology.
Mindpeak also uses this method. The AI, developed in Hamburg, works on image processing with deep learning to support pathology in cancer diagnosis. “Until now, pathologists have been working as they did 100 years ago, searching tissue under a microscope to find tumour cells among several million cells, and they usually only search a few hundred cells per case. This is like searching for a needle in a haystack and takes an extremely long time. Our diagnosis tools, based on artificial intelligence, can analyse several thousand cells on extremely large digitalised images in a fraction of the time, helping pathologists with this complex task. Our AI works approximately ten times faster, making it possible for pathologists to process many more cases in the same period of time and to concentrate on the most difficult cases,” explains Felix Faber, founder and director of Mindpeak.

Other AI methods are also used in the medical branch: Ada Health, for example, uses Bayesian networks – a probabilistic method – rooted on a basis of medical knowledge. “In principle, the technology works like a conversation with the doctor. Based on the main symptom entered, Ada asks adaptive questions which change according to the information provided and take personal risk factors into account. The final report shows the most likely cause for the symptoms and gives advice on the next steps,” explains Jonathan Muck, Senior Communications and Public Affairs Manager at Ada Health. If users share their report with doctors, the AI can also give advice on medical diagnosis. This can be particularly valuable in the field of rare diseases, of which there are approximately 6000.

As we see, AI is already used in some areas of medical technology, such as radiology, pathology or image recognition. The principle of AI is “that systems and algorithms are continually learning. The more data they have already collected, the better the algorithm works,” says Dr. Jörg Traib, Technology Project Manager and Health Director at Bayern Innovativ GmbH in Nuremberg and Director of the forms MedTech Pharma e.V., notional sponsor of MedtecLIVE with T4M. But scenarios such as those shown in films, where the AI becomes independent, will not happen in reality. The AI continually develops itself to a certain degree, by being “trained” by specialists with selected data. If the algorithm is “fed” with sufficiently good-quality, informative data, it will eventually be able to recognise and analyse images which it has not yet seen. Faber adds that, in order for the Mindpeak AI diagnostic tools, such as BreastIHC, to obtain approval, a study had to be carried out to confirm the method’s effectiveness. He sees a challenge for the establishment of AI diagnostic solutions across the healthcare sector, for example, in the existing regulatory conditions. “In principle, it can be said that the MDR primarily covers devices, including fixed objects, and must therefore be revised for SaMD.” According to the MDR, a system must have finished learning when it is approved and then be reapproved when the algorithm is altered by data collection and improvement of the application. But how does approval work and which criteria are focused on in particular?

How does approval and certification work?

In Germany, medical products of all types must undergo a conformity assessment, whereby various aspects are considered in order to determine whether the product delivers on its promises, helps patients and does not harm them. This is the basic condition to ensure that patients are not exposed to any risk. AI must also undergo this process if used for medical purposes. “Artificial intelligence is usually classified as Software as a Medical Device (SaMD), and thus as a medical product. Manufacturers must find a designated certification body, which then checks the product according to the risk class and awards CE certification,” says Muck of the approval process. Faber adds that a study must also be requested in order to confirm the method’s effectiveness: “In principle, it can be said that the MDR primarily covers devices, including fixed objects, and must therefore be revised for SaMD.”

The MDR and the IVDR (EU regulations for in vitro diagnostics) define which criteria must be considered, and while they mention software, they do not consider the special characteristics of AI. For this reason, the certification bodies for AI-assisted SaMD (Software as a Medical Device) also draw on the current status of the technology in order to be able to certify AI to the best of their knowledge and belief. “Aspects which must be considered during approval include for example risk assessment, and the ability to explain AI and its decisions. We also look at verification and validation, such as which data was used for validation and verification, who carried out these processes and the transferability to the target population,” lists Rad. Exactly which criteria are checked depends on the AI’s purpose and models, he explains. In spite of the high complexity, the initial certification itself does not take longer than for a normal medical product and requires between six and twelve months.

If there are changes, for example further developments because the AI has improved itself, under certain circumstances the product must undergo renewed certification, which again takes time. Depending on the extent of the revision, the duration required can vary, but should be shorter than for the initial certification, explains Rad.

The Artificial Intelligence Act – a curse or a blessing?

Each country has its own regulations – at the moment. But the AIA is intended as a Europe-wide regulation, which is intended to regulate the use and approval of artificial intelligence across sectors, establishing consistency. Thus far, there is only an initial draft, which has already had annotations and suggestions for improvement added by many associations. The AIA is a horizontal regulation and therefore applies not only to medical technology, which already has several regulations in Germany. How exactly is this intended to work? “Extensive discussion with all parties is essential. MedtecLIVE with T4M offers a setting for this,” explains Boss.

The Artificial Intelligence Act is the response to current events in the world, as Natalie Gladkov explains the planned regulations: “Some countries are already more advanced with regards to AI, in some cases in ethically questionable manners.” The AIA therefore creates a framework for safety as well as regulatory matters, such as liability. Where the MDR has its limits, the new regulation can also intervene. However, in some points the guidelines overlap: “How exactly this will be implemented remains unclear,” says Gladkov. Regulating only that which needs regulation would be more appropriate. She goes on to say that otherwise, the AIA could mean that the regulation ends up as extensive bureaucracy and that innovations only come onto the market extremely slowly or not at all. “It would be much easier for companies if the focus was placed on that which is practical and the desk work was limited to that which is directly necessary,” according to Faber.

The AIA establishes a framework for safety as well as regulatory questions such as liability, and thus plays an important role. In Germany too, it can provide a framework where the MDR has its limits. However, in some points the guidelines overlap: “As planned by the current draft, there would be double regulation, and how exactly this will be implemented remains unclear,” says Gladkov. The certification bodies are already checking the aspects contained by the MDR, the IVDR and, to a certain extent, also the AIA (Artificial Intelligence Act). Criteria beyond this which have not thus far played a role but are considered in the AIA can then be added gradually when the regulation is introduced. “Additional requirements include for example vigilance of human rights violations or products being used as intended,” explains Rad.

In general, there are two points of view on the potential regulations: “On one hand, it is a chance to establish a lead as a company, but on the other hand, caution is called for with regards to overregulation, particularly in the field of medicine and medical technology,” says Traub. Particularly because, unlike any innovation before it, AI has the opportunity to revolutionise the medical sector from the bottom up, adds Rad. It is becoming increasingly clear that a dynamic, powerful field such as AI must be regulated. While some companies view the MDR as sufficient, “others, and this includes the BVMed, also see potential in the AIA,” explains Gladkov. Faber observes that rules are extremely important, particularly in the healthcare field: “For this reason, I am happy to help as a DIN ambassador in this area with the development of DIN SPECs – the precursor to standards – in order to develop pragmatic solutions for companies.” Sensible and pragmatic regulations also provide trust. “Regulation and innovation need not be opposites. Regulation can break down fearful attitudes, encouraging the use of new technologies,” says Muck. Rad agrees, adding that regulation creates a framework, thereby reducing fear and strengthening trust.

Regulation, directives, audits – this suggests a large amount of work. As an addition to the existing MDR and IVDR, does the AIA impede the development of innovation due to the additional effort required?

AIA – would the regulations be overregulation and block innovations?

When the safety of people is in question, precise consideration is imperative. It is clear that more regulation leads to higher costs. “This is always a challenge initially for start-ups, since suitable staff are currently hard to find on the market,” as Muck explains the problem for the development of new technologies under additional regulation. Faber agrees, adding that large companies would be able to cope with the higher expense, while smaller companies would be more severely affected. “Safety is important, but freedom and opportunities must be maintained, without putting anyone in the market at a disadvantage,” explains Gladkov. However, the AIA is not perceived as an inhibitor of innovation: “It undeniably leads to a requirement for more time and higher costs, but in this sector, we work with human lives, and the additional effort is undoubtedly justified,” says Rad, adding that the AIA connects to worldwide regulations and is therefore legitimate.

How would optimal regulation look?

MDR, IVDR, AIA – reconciling all of these would be complex and demanding for companies, but also for the waiting patients, who could lead a better life with the help of AI. A suggested solution: harmonisation with regional regulations, explains Traub: “It would be optimal for the planned regulation to be integrated into the existing process and not to present an additional hurdle.”

As an example of a good approach, Faber names the FDA (Food and Drug Administration), the US medical agency: “It allows improvements and changes to the product even after certification, with an acceptable amount of effort.” Muck agrees: “Algorithm Change Plans, which act as guard rails, determine the extent to which a product may be changed. In the USA, there is an interesting pilot being run for a new regulation approach. In so-called pre-cert processes, not only a simplified form of the product, but also the company itself are pre-certified, which represents a good precondition for such a development.” An innovative approach, which leaves freedom despite controls and rules. “Clever regulation guarantees safety and is simultaneously a kick-starter and enabler for innovation,” continues Muck.

All things considered, the patients must not be lost sight of, nor that which it costs them when a new technology is simply not introduced. “Regulation can always work out well if there is close cooperation with the market. For this reason, MedtecLIVE with T4M will provide a platform for this and other topics to be expanded upon,” adds Boss. There will undoubtedly be several changes to the draft of the AIA, but the regulation will still be a major topic for exhibitors, visitors and discussions at MedtecLIVE with T4M in May 2022.

 

---

About MedtecLIVE with T4M

The trade fair MedtecLIVE with T4M is the leading European spring event in the field of medical technology and takes place annually, alternately in Stuttgart and Nuremberg. The event covers the entire value chain, connecting the most important medical technology regions in Germany. Here, decision-makers of distributing companies and OEMs meet the most significant suppliers in the field of medical technology.