Artificial intelligence is becoming an essential component of modern clinical trials. It supports patient recruitment, accelerates data analysis, enables adaptive trial designs, and contributes to regulatory decision-making. As sponsors adopt AI systems across various stages of the research lifecycle, they must address the legal and ethical frameworks that govern the use of personal data and algorithmic technologies in healthcare.
This article outlines the core responsibilities of clinical trial sponsors when using AI tools, with a primary focus on European data protection and AI regulations, while also referencing global guidance and emerging standards that shape the broader landscape.

Applying the General Data Protection Regulation

The General Data Protection Regulation applies whenever personal data is processed in the European Union (“EU”), including the European Economic Area (“EEA”) and the United Kingdom (“UK”), or by entities targeting individuals in the EU. When an AI system is used in a clinical trial to process personal data, the sponsor qualifies as a data controller and remains responsible for GDPR compliance of all data processing activities. This includes verifying the legal basis for processing; typically consent or legitimate interest of the sponsor in the area of scientific research; ensuring that the AI system operates in line with the purpose limitation principle and drafting the necessary records and assessments to demonstrate accountability. If an AI tool is introduced after initial data collection, or if its function differs from that initially communicated, the sponsor may need to assess whether the original legal basis still applies or comply with the obligations required for establishing a new legal basis. The sponsor must also ensure that any third-party providing AI services operates as a processor under a compliant data processing agreement and implements adequate technical and organisational measures to protect data confidentiality and integrity.

Understanding the EU AI Act and Its Interception with GDPR

In 2024, the European Union adopted the AI Act, which establishes a legal framework for the development and use of artificial intelligence systems. The regulation applies to all AI systems that are placed on the EU market or used in the EU, regardless of where the provider is based.
The EU AI Act establishes a risk-based regulatory framework that classifies AI systems into four categories: unacceptable, high, limited, and minimal risk. Unacceptable-risk systems are banned outright within the EU because they pose a serious threat to fundamental rights, safety, or democratic values. High-risk AI systems are subject to the most stringent obligations under the Act and can only be deployed if in compliance with such obligations. Limited-risk AI systems are permitted but must meet basic transparency requirements, such as informing users they are interacting with an AI system and ensuring the design does not mislead or deceive. Minimal-risk systems are not subject to specific requirements under the AI Act, but must still comply with other applicable laws, including data protection frameworks.

It is important to clarify the difference between the AI and Data Protection legal frameworks and why they may work simultaneously. While the GDPR applies to the processing of personal data (and if an AI system processes personal data, the person responsible for deploying such system must comply with the GDPR) the AI Act serves to ensure the ethical use of AI in real-world situations. Does the use of AI always require the processing of personal data? Not necessarily. However, in clinical trials, most AI applications, such as patient selection, imaging analysis, and safety monitoring, typically involve personal data.

Does the EU AI Act Apply to Scientific Research?

Despite the broad scope of the AI Act, Article 2(6) and Recital 25 establish a narrow exclusion for AI systems developed and used exclusively for scientific research and development. According to the Regulation, such systems fall outside the AI Act only if they are created solely for the purpose of conducting scientific research, and only if they are not placed on the market or used to produce legal or significant effects on individuals.

This exclusion was introduced to protect academic and experimental research and is designed to avoid imposing the full regulatory burden on AI models used in non-commercial, closed research environments. However, the exemption does not apply in a number of common clinical research scenarios. First, if the AI system is procured or licensed from a commercial provider, rather than developed specifically for the research project, the exclusion cannot be claimed. Second, if the system is used in a clinical trial where it influences patient eligibility, dosing, safety monitoring, or any aspect of the investigational product’s development pathway, the system is no longer considered confined to a purely scientific function. It is then considered to be “put into service,” as defined in Article 3(13) of the AI Act.

In practice, this means that most AI tools used operationally in clinical trials, particularly in interventional or regulatory-driven settings, will not qualify for the scientific research exclusion. The same applies to systems developed in a research environment but intended for future market use, including tools supporting software as a medical device or algorithms subject to future certification.

EU AI Act and Clinical Trials

AI systems used in clinical trials may fall within the high-risk category under the EU AI Act through two regulatory pathways outlined in Article 6. First, under Article 6(1), an AI system is considered high risk if it is a product or a safety component of a product governed by EU harmonization legislation listed in Annex I, such as medical devices under Regulation (EU) 2017/745 or in vitro diagnostic devices under Regulation (EU) 2017/746, and if that product requires third-party conformity assessment. This means that investigational AI tools used for diagnostic decision support, patient stratification based on biomarkers, or real-time safety monitoring may be classified as high risk if they fall within the scope of these device regulations and are subject to notified body review.

Second, Article 6(2) states that AI systems listed in Annex III are also deemed high risk. While clinical research is not explicitly mentioned in Annex III, an AI system used in a trial may fall under this category if it materially influences decisions that affect participants’ health or fundamental rights, particularly where profiling is involved or medical decision-making is impacted. Sponsors must assess whether the AI system qualifies under either of these routes, as both may lead to a high-risk designation with corresponding regulatory obligations.

If a clinical trial sponsor deploys a high-risk AI system (e.g. for patient selection, safety signal detection, or diagnostic support), it must comply with the EU AI Act by ensuring the system is used according to the provider’s instructions, assigning trained human oversight, retaining system logs for at least six months, and monitoring the system’s performance. The sponsor must report any serious incidents or risks to the provider and relevant authorities without delay, ensure input data is relevant and representative, inform trial participants of the AI system’s use, and where applicable, perform a fundamental rights impact assessment and complement the existing GDPR Data Protection Impact Assessment (DPIA) with AI-specific risks.

The Role of Data Protection Impact Assessments

When AI systems are used in clinical trials and involve the processing of sensitive health data or automated decision-making, a Data Protection Impact Assessment may be required under the GDPR. This assessment should include a description of the processing, the purpose of the AI system, the legal basis for data use, and an evaluation of the risks to data subjects. Where the AI system falls under the AI Act’s high-risk category, the sponsor must also maintain a risk management framework aligned with the requirements of the Regulation, including appropriate levels of human involvement, accuracy monitoring, and transparency in system design.

Global Context: Ethics and Emerging Regulatory Approaches

While the European Union provides one of the most comprehensive legal frameworks for AI in healthcare, other jurisdictions are developing their own regulatory and ethical approaches. The United States Food and Drug Administration (FDA) has issued an action plan for AI in medical devices and emphasizes good machine learning practices, particularly in software that evolves over time. Health Canada has issued draft guidance for AI-enabled medical devices, and Australia has adopted a regulatory sandbox model for early-stage AI testing.

The World Health Organization has published the Ethics and Governance of Artificial Intelligence for Health report, which sets out guiding principles such as transparency, accountability, inclusiveness, and respect for human autonomy. These principles are intended to guide all stakeholders involved in health-related AI, including researchers and sponsors. Even where specific legal obligations may not yet exist, adherence to ethical standards is increasingly expected by ethics committees, funders, and regulatory agencies. Sponsors are encouraged to align with these international standards and document their governance processes accordingly.

Conclusion

The application of the EU AI Act follows a phased approach. The Regulation entered into force in August 2024, with key provisions becoming applicable in stages. Rules concerning prohibited AI practices and AI literacy take effect from February 2025. Obligations for general-purpose AI systems, including transparency, documentation, and risk mitigation, will apply from August 2025. Requirements for high-risk AI systems, such as conformity assessments, risk management, and human oversight, come into force from August 2026. For AI systems embedded in medical devices that require notified body involvement, the relevant obligations apply from August 2027.

At the same time, jurisdictions such as the United States, Canada, the United Kingdom, and Australia are developing or implementing new legal frameworks to govern the use of AI in healthcare and clinical research. As global standards continue to emerge, clinical trial sponsors should design compliance programs that align with both European regulations and international expectations. A harmonized approach will help ensure ethical, legal, and operational consistency when deploying AI tools in trials across multiple regions.

The advent of new technology always ushers increasingly complex developments in the ever-evolving landscape of drug development. The uptake of Artificial Intelligence (AI) technologies has been ubiquitous in all areas of drug development, including clinical research where digital health solutions are being employed to increase clinical trial efficiency and decrease the associated time and costs.

Clinical trials are fraught with the resource-intensive hurdles of cost, time, and complexity. A promising application of AI being used to address these issues is digital twins. Digital twins are digital replicas of physical objects or systems connected by bidirectional data and information flow. Popular in the aerospace and manufacturing industries, digital twins are also being used in clinical trials to replicate biological systems or processes to simulate real time biological processes and to model outcomes.

Digital twins can model biological components ranging from cells and tissues to organs and environments in a patient’s body. A digital twin is generated from preexisting data, AI modeling and incorporates real time data to predict outcomes to optimize decision making. These twins are versatile and have several applications, some of which include drug discovery, drug repositioning, personalized treatments based on digital patient profiles, recruitment into trials as virtual patients, in-silico clinical trial design and safety monitoring.

Featured are a small variety of companies demonstrating the creative applications of digital twin technology in clinical trials:

• Unlearn –  Unlearn has a platform to generate digital twins aiming to aid in designing more efficient trials, reducing sample sizes, boosting power, and making faster, more confident development decisions. PROCOVA™ is a statistical methodology developed by Unlearn.AI for incorporating prognostic scores derived from trial participants’ digital twins into the design and analysis of phase 2 and 3 clinical trials.

This methodology has been qualified by the EMA and is covered under the FDA’s guidance on Adjusting for Covariates in Randomized Clinical Trials for Drugs and Biological Products as a special case of ANCOVA statistical method.

• BOTdesign –  Botdesign has ORIGA, Europe’s first web-based platform for augmenting clinical data with deep learning. It enables healthcare manufacturers and researchers to generate realistic artificial patients, while guaranteeing data confidentiality and regulatory compliance. ORIGA is based on advanced generative AI models called Variational Autoencoders (VAEs) used to create synthetic patients. This can be particularly useful in increasing size and diversity in research especially for rare and underrepresented cohorts.
• Aitia –  Aitia has built a causal AI engine (REFS®) that uses high-performance computational power to turn massive amounts of multiomic and patient outcome data into fully realized, unbiased and causal in silico models of human disease called “Gemini Digital Twins” that can be used to discover new causal human drug targets and biomarkers, candidate patient subpopulations for clinical trials, and optimal drug combinations.
• Bayer – Bayer has used digital twins to create virtual trial arms or “external control arms”, which can replace control/placebo arms in some clinical trials. This can help fill evidence gaps e.g., where an RCT (randomized control trial) is not feasible or ethically sound, in addition to reducing costs, overall development time and/or trial recruitment time.
• Sanofi – Sanofi uses quantitative systems pharmacology (QSP) modeling of a disease and available clinical trial data from live patients to create digital twins of the human patients seen in the clinic. All of the available data on disease biology, pathophysiology, and known pharmacology, is taken and integrated into a single computational framework.

Although digital twins can’t fully substitute real humans, they can help streamline clinical trials by reducing costs and timelines. As is the case with any technology there are associated ethical, technological and regulatory risks and challenges. The accuracy and predictive power of a digital twin heavily depend on the quality of input data, and issues with generalizability currently limit scalability. Given their extensive reliance on patient data, digital twins must comply with the varied privacy and security laws globally. Nevertheless, the advancement of AI technologies lends potential for digital twins to revolutionize drug discovery and development even further.

Let’s do something a properly trained writing AI probably would not do: Start with an I statement.

I get the feeling that each time ADRES reaches out to me about possibly contributing an article to the BioStartup Standard, it turns into me writing about parts of my personal journey in the biotech space.

It is the same again, this time, because four (widely spaced) personal events, or perhaps rather encounters, over the course of that journey, together inform this piece, which, stripped of those anecdotes, boils down to little more than a small piece of advice regarding training: Training not, as the context of AI may suggest, in the sense of feeding training data into an AI model, but training in its traditional sense – the training of people.

1. The first touchpoint was reading an article about knowledge loss (organizational forgetting) in the chemical industry. This was more than 15 years ago, and I did not keep a copy of the article, so unfortunately, I cannot attribute the exact source. Part of the methodology included interviews with retired lead engineers from several chemical plants and the role they continued to play as consultants post-retirement. It highlighted how there was no impact on routine operations with the loss of key knowledge assets (the lead engineers), but that that changed as soon as troubleshooting was required, be it due to quality issues, breakdowns, or changes such as planned expansion or process improvement.

• Other than the first, the second touchpoint was directly related to biotech and to AI – or rather to Natural Language Processing (NLP), i.e., one of the key concepts in machine learning and language models, because back then – in the early 2010s – no one I knew called it AI yet. But the ideas were already there and I was discussing their potential application to biotech (specifically, to the analysis and presentation of data from clinical trials) with friends in Cambridge who were doing NLP research. While we could already envision, if dimly, what would be possible in the future (was is possible now!), in the short- to midterm we saw that the limitations of the (then available) technology would place it firmly as a tool for a human expert, like an advanced word processor or statistical programming suite.

• Fast forward to 2025, and we have AI established and growing in importance across industries (including, of course, biotech). And along with that we have a growing body of criticism as well, which is where the third touchpoint comes in: A couple of weeks ago (end of June 2025, that is), a friend recommended a draft paper[1] to me, covering a study on neural and behavioural consequences of using AI assistance in (academic) writing tasks. The authors’ concluded that aside from positive effects the use of AI also “came at a cognitive cost”, impacting critical evaluation of AI outputs and potentially reinforcing “echo chamber” environments in which outputs from AI systems get critically checked less and less as their users get primed by previous exposure.

• Then, shortly thereafter, the final piece to this puzzle, the one that made everything click into place, came into play when colleagues at ADRES reached out with the call for contributions to the issue of the BioStartup Standard you are currently reading. And right there, in the middle of the technical guidelines for submitting an article, I read “AI tools can assist, but substantial revision and personalization are required” and found that mildly funny – the call for contributions to “the AI issue” was critical of relying fully on AI. Initially, my somewhat vague intention had been to write about implementation of “behind the firewall” systems in small scale organisations or something similar more operations oriented. But I felt myself constantly drawn back to this critique of AI in a call for AI and it got me thinking in an entirely different direction. One by one the above memories came up: My first – abortive (I would be lying if I said that we implemented anything of what we discussed in Cambridge) – concepts for utilising AI in trial analysis and reporting as a tool for human experts; the recent paper on cognitive cost of using AI – specifically in an educational (learning!) setting; the long ago read about organizational forgetting caused by personnel turnover; it all started to fit together.

Let me pause here briefly to state (if that did not become clear from the Cambridge anecdote) that I am not an AI-luddite who tries to warn you about how dangerous this technology is and that you better not use it. We are using it. And we should be using it. It is a powerful tool, as I am sure a lot of the colleagues contributing to this issue will highlight in their own articles.

As we are adding new and powerful tools to your toolbox, we need to make sure to also have the right users for these tools, not just the right tools themselves, and that also means not neglecting the training of your next generation of users – not just in using the tools, but in the fundamentals.

The current generation of professionals in our space has still acquired their skills and experience outside an AI echochamber, they are experts able to deliver without AI support, who become further empowered by new AI tools, and are able to critically review what a system delivers, feeding into a continuous improvement cycle.

But this generation is not here to stay forever. What is needed, thus, is to ensure that the next generation, as well, will understand the underlying science and processes, and often enough the art and craftsmanship to do the same – to function and deliver without AI, to make the most use out of the AI systems available, to check whether the systems are performing, and to improve them going forward.

Invest in AI. But do not neglect to invest in people.

AND not OR.

1 N Kosmyna, E Hauptmann, YT Yuan, J Situ, X-H Liao, AV Beresnitzky, I Braunstein, P Maes, ‘Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task’, https://arxiv.org/abs/2506.08872 (retrieved 30-Jul-2025)

The convergence of artificial intelligence (AI) and organoid technologies is beginning to reconfigure the early stages of drug development. These two innovation domains, each advancing rapidly on their own, are now intersecting in ways that promise to improve the predictive value of preclinical testing, reduce the cost and duration of development pipelines, and ultimately produce safer, more effective therapies. Yet alongside this opportunity lies a complex set of technical, ethical, and regulatory challenges. For the scientific and biotech community, navigating this evolving landscape will require not only technological adaptation but also institutional coordination and policy foresight.

A New Convergence in Preclinical Modeling

Organoids – three-dimensional, multicellular constructs derived from stem cells – have emerged as biologically relevant in vitro systems that recapitulate aspects of human tissue architecture and function. Their ability to model complex human phenotypes has led to growing use in oncology, infectious disease, toxicology, and regenerative medicine. Compared to animal models or two-dimensional cultures, organoids offer advantages in terms of genetic fidelity, species relevance, and personalization. However, their adoption in industrial drug pipelines remains limited by variability in culture protocols, inconsistencies in functional readouts, and a lack of data harmonization across producers and laboratories. These limitations have motivated increasing interest in computational approaches to standardize interpretation and enhance comparability—enter AI.

Machine learning and deep learning approaches, when applied to the outputs of organoid systems, can extract latent patterns in high-dimensional data, such as transcriptomics, high-content imaging, and pharmacological response profiles. AI has shown promise in identifying phenotypic signatures, classifying tissue states, and predicting drug responses. In theory, these tools could accelerate compound screening and guide mechanism-informed lead selection. Yet AI systems trained on organoid data inherit the uncertainties and inconsistencies of their biological source material. As a result, successful integration depends on improving both experimental standardization and data quality—two prerequisites for effective model training, validation, and interpretation.

Regulatory Realignments and the Burden of Proof

The regulatory environment is evolving in parallel. In the United States, the passage of the FDA Modernization Act 2.0 in 2022 formally removed the requirement for animal testing prior to human trials. This shift has created space for new approach methodologies (NAMs), including organoids, computational simulations, and other alternatives, to support investigational new drug (IND) applications. The FDA’s Model-Informed Drug Development (MIDD) initiative encourages the use of simulation and predictive modeling throughout the development process. Simultaneously, the agency has begun developing frameworks for AI/ML-based software, focusing on algorithmic transparency, real-world validation, and risk mitigation. While regulatory acceptance of AI-derived predictions remains cautious, the direction is clear: tools that are well-characterized, traceable, and biologically grounded are increasingly welcome in preclinical and regulatory workflows.

In the European Union, a more prescriptive and comprehensive regulatory framework is emerging. The Artificial Intelligence Act, adopted in 2024 and set to be enforced in phases from 2025, represents the first region-wide legislation governing AI. Biomedical applications—particularly those with potential implications for health outcomes—are designated as “high-risk” under the Act. Developers must meet requirements related to data governance, explainability, human oversight, and post-market monitoring. Although the Act is technology-neutral, its implications for AI-driven drug development are significant, especially when organoid-derived or patient-specific data are involved. Unlike sector-specific guidance, the AI Act applies horizontally across domains, which presents both a compliance burden and an opportunity to build AI tools that are safe, auditable, and trustworthy by design.

Toward a Predictive and Accountable Innovation Ecosystem

Despite their promise, the integration of organoids and AI into drug development raises systemic challenges. A persistent lack of protocol standardization continues to limit reproducibility across labs and platforms. Biological heterogeneity, while valuable for capturing patient diversity, also complicates benchmarking and model generalization. The ethical use of patient-derived tissues and associated data requires robust consent procedures and governance structures that can support both research and commercial applications. On the computational side, many AI models function as black boxes, limiting interpretability and regulatory acceptability. Moreover, the successful deployment of these technologies depends on interdisciplinary teams—yet the integration of wet-lab biology, computational modeling, and regulatory expertise remains rare in most research environments.

Nevertheless, a growing body of academic, industry, and regulatory stakeholders is working to address these gaps. Efforts to create interoperable organoid databases, define reference standards, and foster precompetitive data-sharing frameworks are underway. Some regulatory agencies are exploring sandbox initiatives that allow developers to test AI models in controlled settings with early feedback. Ethical frameworks for the secondary use of patient-derived data in AI training are also gaining attention, although global harmonization remains limited.

In the years ahead, the integration of AI and organoid platforms could enable a more human-relevant and predictive approach to drug development—one in which computational models are trained on real biological complexity, and preclinical decisions are informed by tissue-specific responses. But realizing this potential will require more than innovation. It will demand transparency, shared standards, and a long-term commitment to collaborative infrastructure. The scientific community must work not only on the frontiers of technology, but also at the interface of governance, ethics, and reproducibility.

In this context, the convergence of AI and organoid science is not simply a technical advance. It is a shift in how we conceptualize preclinical research—away from generalized proxies and toward systems that integrate human biology, computation, and regulatory science in a coherent, scalable, and accountable way.