What is "Human-in-the-Loop"?

The newly proposed EU GMP Annex 22 on Artificial Intelligence (AI) establishes requirements for the use of AI and Machine Learning (ML) in the manufacturing of active substances and medicinal products. It sets up requirements for the selection, training, and validation of AI models. The document emphasizes the concept of "Human-in-the-Loop". What does this mean?

Human-in-the-Loop (HITL)

The draft document does not apply to generative AI and Large Language Models (LLMs), which should not be used in critical GMP areas. However, if they are used in non-critical GMP applications that do not directly impact patient safety, product quality, or data integrity, personnel with the appropriate qualifications and training must ensure that the outputs from these models are fit for purpose. In particular, the following applies:

• Where a model is used to inform a decision made by a human operator, the description of the intended use should include the operator's responsibilities. In this case, the operator's training and consistent performance should be monitored as with any other manual process.
  
  
• When a model is used to inform a human operator's decision, records of this process should be kept. Depending on the criticality of the process and the level of testing of the model, this may imply consistent review and/or testing of every model output according to a procedure (human review).
  

More information is provided in the draft EU GMP Annex 22: Artificial Intelligence.