Validation - Revision of Annex 15: Industry's "problems" - Results of an ECA Industry Survey

Recommendation

2/3 May 2024

Process Validation - Live Online Training

With the publication of a draft on the revision of Annex 15 at the beginning of February 2014 movement was brought into the GMP requirements on validation and qualification - also in Europe. We have known about changes - especially concerning the topic of process validation - by means of the new FDA Process Validation Guidance from the USA since the beginning of 2011. But the changes which might result from revision of Annex 15 should not be underestimated either. The European Compliance Academy (ECA) has carried out a survey to determine how industry assesses these possible changes and the uncertainties that exist concerning the revision draft. The following is a conclusion on the analysis of the survey:

A total of 116 persons participated in the survey. Not everybody answered each question, however. But the fact that some of the questions were skipped is also a statement and leaves room for interpretation. Almost 50% (45.2%) of the participants in the survey work for companies with more than 500 employees while almost 1/3 (32.3%) work for companies with fewer than 100 employees (see graph 1).

Graph 1: Distribution of participants in the survey by company size

A total of 68.9% of the participants are employed in the pharmaceutical industry. 13.1% of the people participating in the survey work in the API producing industry (chemical API and biotechnologically produced API). The majority of participants stating their function within the company (61%) had a function in the "quality" sector. Almost 10% (9.2%) have validation functions or work as consultants. That is all as far as demographic data is concerned.

The first question concerned the "ongoing validation strategy" which, according to the draft, should be described in the validation master plan. The question was whether more explanation was needed on this term. More than half of those (51.3%) answering this question answered in the affirmative. About 1/3 (34.5%) did not need further explanation. 10 persons answering "yes" substantiated their answer. Some of the responses were: A prepared format including a table of contents; some general conditions on the level of detail expected would be useful; specify minimum requirements; more details; is that revalidation or "traditional" trending? - Where does an annual product quality report end and where does an ongoing validation strategy start? It is not clear as to whether the term relates to a future change control and revalidation strategy or whether it relates in part, or in full, to ongoing process verification. More clarification concerning acceptability is necessary since everybody uses the term of "ongoing validation strategy" in a different way. An extension to verifications is required with the exception of legacy systems. A further comment suggested that Annex 15 should require an SOP for the "ongoing validation strategy".

Question number 2 as to whether using conditional approval to the next stage in a case of deviations with a documented assessment in connection with qualification/validation activities should be possible was answered by 62.2% with "yes" (if approved by the Quality Unit). A further 20.3% would allow this approach only in "exceptional circumstances". 13.5% regard it as a good idea. Only 4.2% would not allow this approach.

57.7% of the participants who answered question 3 on how the new requirement that Factory and Site Acceptance Tests become a request in how equipment qualification is judged said they would only use it on new complex equipment. 19.7% would use it for all new equipment. 18.3% considered it to be a very good idea and only 4.2% of those participants answering said that the tests were not required.

More than 50% (53.6%) expect further explanation on the cancellation of the chapter qualification of (established) equipment in use. Almost 1/3 (29%) of the participants even think the chapter should be retained. But 17.4% of them consider cancellation to be a very good idea.

The question 5  "Do you miss alternatives (e.g. ASTM E 2500) for the equipment qualification" answered more than 50% (54,3%) with "not clear yet". The reason for this might be that ASTM E 2500 is not widely known in Europe. Also 28,6% answered with "no" and only 17,1% with "yes". 

Regarding question 6 on how to judge the fact that 3 validation batches are still mentioned, 39.1% answered that more explanation was needed. 30.4% are glad that this is the case. About 14.5% of the persons answering the question stated that they wished one batch were sufficient. The percentage of persons who would prefer that the 3 validation batches were not mentioned any more in order to prevent confusion is about equal (15.9%). 

Graph 2: Distribution of the answers concerning the 3 validation batches

Question 7 ("Do you understand the term "hybrid approach"?")  answered approx. 50% (45,6%) with "not clear yet". 33,8% answered with "yes" and 20,6% with "no". Only three participants gave an answer to the question about what clarification is needed. Two of the three persons who answered the question asked for more examples and for a more specific definition. It was also asked when the "hybrid approach" was considered as sufficient to justify the release of batches.

The possibility of a bracketing approach regarding process validation (question 8) was judged to be positive by the vast majority of persons answering. For 56.7% of them, it is a good idea but the approach must be very well-justified and 28.4% even think the idea would be very good. 10.4% of them would use this approach only in exceptional cases and 4.5% would forbid it altogether (see Graph 3).

Graph 3: Distribution of the answers to the possibility of a bracketing approach

52.2% answered "no" to question 9 on whether the differences between "continuous process verification" and "ongoing process verification" are understood but nearly as many (47.8%) answered with "yes". Some of the comments of participants who answered "no" are listed as follows: Clear definition/expectations of what is meant (4x); means the same thing (2x); wish for an overview and steps for each (of the two processes) in Annex 15. One participant commented that it is not clear how material that was released within a continuous process verification can be revalidated.

Question 10 "Do you appreciate the fact that the "Verification of Transportation" and "Validation of Packaging" chapters are now part of Annex 15?" was answered by 79.9% with "yes" and by 9.2% with "no". The answer is not clear yet for 13.8%.

Question 11 "Would you prefer more than the PDE-acceptance criteria in a cleaning validation (e.g. 10 ppm)?" leads to an interesting result. The distribution of answers was relatively even: 37.5% of the participants said "yes" while 32.8% said "no" and the question is not yet clear for 29.7%. It is not easy to cluster the comments of the participants who answered "yes" and were asked to specify their answer. Further acceptance criteria mentioned were, for example, toxicological studies, pharmacological studies and properly justified organoleptic controls. Several times, the wish was expressed that the use of the 10 ppm acceptance criteria and acceptance criteria with respect to daily dosages should still be possible. One comment argued in favour of the lowest value after the calculation of quantitatively visibly clean, 10 ppm and 1/1000 dosage criteria. The wish was expressed to clarify where the NOEL data should come from since there is no data on factors F1-5 for many actual active ingredients. One commentator wanted clarification in Annex 15 that cleaning validation only concerns equipment coming into contact with the product. Furthermore, the wish for clarification between acceptance criterion and routine criterion was expressed. One comment was: PDE data is not always available. Then the 10 ppm or the dosage criterion should be possible. One interesting contribution suggested that when using a PDE concept, one could abstain from validation of the detergent if an EU-certified detergent for household purposes is used.

Question 12 on whether a clear statement regarding the number of validation batches for cleaning validation would be preferred was answered in the affirmative by 63.1%. 26.2% of the participants said "no" while 10.8% still regarded the answer as unclear.  

The penultimate question 13 was about the wish for more conformity with FDA Process Validation Guidance. 60.9% answered "yes", 20.3% answered "no" and for 18.8% it is not clear yet. Some of the comments of the participants answering "yes" were: more details on stages 1, 2 and 3; necessity for a statistical approach - 3 batches are not sufficient; it would be very nice to have one single guideline covering US and EU requirements (or two identical ones). With regard to world-wide distribution, in addition to harmonisation with the FDA, one comment wants harmonisation with KFDA, Japan, Anvisa etc. (to avoid an extra workload on validation activities). The traditional and the hybrid approach should be dropped completely; more clarity with respect to expectations concerning statistics and terminology (stages 1-3) would significantly simplify harmonisation; I would like to see more conformity.

Graph 4: Distribution of the answers to the question no 14  "how to you judge the draft in general?"

Not that many requests for improvement were expressed. Some comments on this topic are listed as follows: Not much is clearly stated ... a lot of promises for many years ... still ongoing revisions and now process validation and cleaning validation surprises ... these key points will again be the cause for disagreements and a lot of discussion! Instead of becoming harmonised, we are again drifting apart; it is a general and heterogeneous mixture of old and new concepts (more conformity at least with the FDA Guidance), more background information on how to obtain PDE values. And the 3-batch concept is stated for the traditional process validation but not for the cleaning validation. No integration with the ASTM verification life cycle, no alternative for the equipment qualification; more conformity with the FDA Guidance; clearer definitions in each chapter. One commentator referred expressly to chapter 1.5, criticising the fact that acceptance criteria are not very meaningful in a validation master plan if the master plan is a company-wide document - unless it is about the way in which deviations from acceptance criteria are handled. The commentator goes on to explain that the assessment of resources can only be carried out in the form of a general overview as this differs from one project to another. The last point the participant commented on was that confirmation of materials would be described in a protocol but not in a validation master plan. Then there was a question concerning item 2.6 ("Changes") in the draft: Is this intended to be about changes concerning acceptance criteria or about typographical errors, additional tests? Why would the scientific justification be sufficient?

Conclusion
The participants in the survey have a surprisingly positive opinion of the draft of Annex 15. 57.2% of participants considered the document to be very good or good. Another 31.7% said the draft was "satisfactory". This is somewhat surprising since the desire for more "clarity" and for further explanations/examples was expressed rather often by the participants in the survey. This concerned the following points in particular: 

• ongoing validation strategy 
• hybrid approach
• difference between continuous and ongoing process verification 
• number of cleaning validation runs

A lower percentage of participants compared to the items mentioned above demanded more details, explanations and examples concerning the PDE concept in connection with the cleaning validation. For 30% of the participants, the question concerning alternative acceptance criteria to PDE within the scope of cleaning validation was not (yet) clear at the time. Irritation was also caused by cancellation of the chapter on qualification of (established) equipment in use where the vast majority of persons answering the question demanded further explanations. Almost 1/3 even think the chapter should be retained.

The following points were considered to be rather positive:

• the possibility of a conditional approval to the next stage in the case of deviations in connection with qualification/validation activities 
• the possibility of a bracketing approach regarding process validation 
• inclusion of the "verification of transport" and "validation of packaging" chapters

Generally, the wish for more conformity (at least) with the FDA Process Validation Guidance was expressed with the goal of (world-wide) harmonisation.

The ECA will put the results of the survey at EMA's disposal for information purposes.