gms | German Medical Science

In February 2013, the European Medicines Agency (EMA) published the “Reflection paper on GCP compliance in relation to trial master files (paper and/or electronic) for management, audit and inspection of clinical trials”. This paper was the EMA’s response to a question from the EMA Inspection Working Group on how to handle the drastic increase in minor and major inspection findings in the trial master file (TMF) from 2011 to 2012, and in particular how to address TMF quality issues in the future [1], [2], [3], [4].

The essential documents in the ICH GCP (International Conference on Harmonization of technical requirements for registration of pharmaceuticals for human use – Good Clinical Practice), published in 1996 [5], is the minimum list of documentation but is not a comprehensive content list for the TMF [2]. From 1996 to the present, the environment of clinical trials has changed drastically, requiring additional documentation in the TMF. All documents associated with a clinical trial that are needed to reconstruct the course and conduct of the trial are relevant for inclusion in the TMF [2]. Therefore the core documents as listed in the Drug Information Association (DIA) TMF Reference model Version 2 (2012) [6] were considered more germane and are included in the risk assessment that we performed (see below).

Inspectors from the EMA have frequently identified problems with the TMF, including that sponsors often fail to provide a comprehensive TMF. The EMA stated in its “Reflection paper on GCP compliance” that TMFs should be complete and accurate [2]. This raises the question: When can a TMF be considered accurate and complete? At present, no quality characteristics have been defined to ensure a TMF is up to standard. The EMA “Reflection paper on risk-based quality management in clinical trials” requested that tolerance limits should be established [7]. However, at present no detailed regulatory guidance is available with regard to tolerance limits, nor are we aware of any publication proposing acceptable tolerance limits for the completeness of a TMF.

The main objective of this cross-company working party was to establish explicit expectations as to when a TMF can be accepted as sufficiently accurate and complete. Risk management methods were employed to define quality expectations and tolerance limits.

The impact on “Safety, rights and wellbeing of patients” as well as “data integrity” was used as the basis for the risk assessments. This is in accordance with the EMA’s view that these are the ultimate principles in GCP and that they should guide the assessments of quality in clinical trials [7].

As a first step, a risk assessment was performed for all 148 required types of core documents included in the DIA TMF Reference model Version 2 (2012) [6] and provided as data publication at Dryad [8]. For each document type the impact with regard to “Safety, rights and wellbeing of patients” as well as “data integrity” was determined based on the assumption that a missing document would indicate that the underlying process was not performed (worst case scenario).

The team members assessed the impact of the missing process on patient rights and safety and trial data integrity, using a 10-point scale. The impact was rated as “critical” (score between 8 and 10) if the missing process would have a direct effect on patient rights and safety or trial data integrity. “Major” (score between 5 and 7) was used if it would have possible effects on patient rights and safety or trial data integrity. A score between 2 and 4 was chosen for a “minor” impact that would have no expected effects on patient rights and safety or trial data integrity. Finally a score of 1 was applied if the missing process would not have any effect on patient rights and safety or trial data integrity but only impact the documentation of the clinical trial. Examples of each type of document are provided in Table 1 [Tab. 1]. The full list of all assessed document types is provided as data publication at the Dryad repository (see [8]).

As stated above, 148 types of documents were included in the risk analysis. Sixty-nine types of documents (47%) were categorized as critical, 54 types of documents (36%) were categorized as of major importance, and 26 types of documents (18%) were categorized as of minor importance, as their absence was not expected to have impact on patient rights, safety or data integrity (Figure 1 [Fig. 1]).

In a second step a risk assessment was performed to determine how much effort would be required to replace or substitute a missing document in the TMF, assuming that the associated process was performed for the clinical trial (that is, that the document had been generated during the trial but was not available in the TMF).

For all core documents assessed in step 1 the level of effort required for tracing or replacing the document was assessed. If the original was available the document type received a risk score of 1; if a copy could be filed in the TMF, the missing document received a score of 2. When the document was missing, but the process could be proved to have taken place using other documents, the effort of this verification was assigned a score of 3 to 6.

The overall risk assessment combined the results of step 1 (the impact of a process that was not performed) with step 2 (the effort to replace or substitute a document). The overall risk assessment was calculated using the following formula:

Because the importance of patient rights and safety was considered much more important than the effort required to substitute or replace a document, the score for the impact of the missing process was squared in the formula above. The resulting risk priority number (RPN) was plotted on a risk-ranking matrix (Figure 2 [Fig. 2]) in order to assess the overall risk category. The numerical values of the risk categories are displayed in Table 2 [Tab. 2].

Categorization of all 148 required types of core documents in the DIA TMF Reference model indicated that only 8% of all document types in a very high risk category, whereas 14% are in a high risk category, 38% of the document types are in a medium risk category, and 40% of all document types are in a low risk category (Figure 3 [Fig. 3]).

Based on the first risk assessment step in which the impact of missing documents/processes on patient rights and safety and/or trial data integrity were assessed, the team specified tolerance limits. The tolerance limit indicated what level of completeness is necessary to ensure acceptable quality of a TMF. The tolerance limit for “stand alone documents” (for instance, the clinical trial protocol) was 0%. Table 3 [Tab. 3] shows the defined tolerance levels.

To illustrate the effect of the defined quality expectations, the team estimated the number of documents for a trial with 100 sites and calculated the acceptable number of missing documents according to Table 3 [Tab. 3]. For instance, assuming each site was provided investigational material three times during the trial, this would amount to 300 documents. The quality expectation of <1% would still be fulfilled if 2 documents were missing (Table 4 [Tab. 4]).

The full list of all assessed document types is provided as data publication at the Dryad repository (see [8]).

The TMF should adequately document trial processes and thereby ensure that patients’ rights are respected, that their safety is assured, and that the trial data are reliable. Not all documents in the TMF are of equal value in documenting these outcomes. The absence of some documents (and their associated processes) may have a critical impact on these outcomes, whereas others may have almost no impact at all. Therefore, when considering how to assure an adequate-quality TMF, it is important to assess the importance of the individual documents rather than to simply consider the overall number of documents filed. Therefore, tolerance limits for missing documents cannot be specified uniformly and should be assessed based on the impact on patient rights and safety and trial integrity. Our approach determines the importance of a missing document, and in addition (if the trial process has been performed, but the document is absent), indicates the amount of effort required to replace it. We have generated a list of document types of very high and high importance which should be focused on in order to assure an adequate high-quality TMF, while on the other hand identifying lower risk areas which require less emphasis and attention during quality control steps without endangering the integrity of the entire TMF. This list could be of major assistance to anyone working with TMFs, e.g. helping to ensure adequate and continuous TMF maintenance or to prioritize efforts in an inspection preparation in case of short timelines and limited resources.

The team also rated other deficiencies (e.g. poor scanning quality) observed in QC checks. Respective quality expectations, tolerance limits and respective QC procedures are not included here but will be published in a separate article as it requires a comprehensive treatment.

Data for this article are available from the Dryad Repository: http://dx.doi.org/10.5061/dryad.t2f61 [8].

The authors declare that they have no competing interests.

We acknowledge Dr. Jill Holbrook who provided medical writing services on behalf of BI Pharma GmbH & Co. KG.