gms | German Medical Science

GMS Medizinische Informatik, Biometrie und Epidemiologie

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e.V. (GMDS)

ISSN 1860-9171

Current practice and perspectives in CRO oversight for Biostatistics & Data Management services – survey performed among members of the German Association of Research-Based Pharmaceutical Companies (vfa)

Aktuelle Praxis und Perspektiven beim CRO-Oversight für Biostatistik- & Data Management-Dienstleistungen – Umfrage unter den Mitgliedern des Verbandes Forschender Arzneimittelhersteller (vfa)

Research Article

  • corresponding author Michael Hennig - GlaxoSmithKline GmbH & Co. KG, Munich, Germany
  • Dietrich Knoerzer - Roche Pharma AG, Grenzach-Wyhlen, Germany
  • Armin Schüler - Merck KGaA, Darmstadt, Germany
  • Daniele Compagnone - AbbVie Deutschland GmbH & Co. KG, Ludwigshafen, Germany
  • Ferdinand Hundt - Doc-Dog-Consulting, Berlin, Germany
  • Thorsten Ruppert - vfa, Berlin, Germany

GMS Med Inform Biom Epidemiol 2018;14(1):Doc01

doi: 10.3205/mibe000179, urn:nbn:de:0183-mibe0001795

Published: January 22, 2018

© 2018 Hennig et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at http://creativecommons.org/licenses/by/4.0/.


Abstract

Outsourcing of activities is common and increasing in the pharmaceutical industry, not only for clinical trial conduct, or other trial related aspects but also for biostatistics and data management.

This raises the question of whether and how sponsors retain the capability to select and to control the CRO(s) involved and what biometrical expertise still has to be present in the sponsor company to ensure adequate oversight. Not only in the case of partial outsourcing but especially in the case of complete outsourcing sponsors have to find the right balance between a complete “hands off” approach and a resource intensive in-house shadowing of activities to fulfil the requirements.

In order to answer these questions, a survey was conducted among the German vfa member companies. This article discusses the specific oversight topics for the outsourcing of data management and biostatistics services, e.g. requirements for data quality, coding, and statistical analyses.

This study shows that the majority of the companies use the same CRO for biometrical services as for rest of study services and that there is a preferred provider for these services in 65% of the companies. There are established, specific requirements for the main deliverables of biometrical services: For coding and data standards these are predominantly industry standards (MedDRA, CDISC). For statistic outputs and data quality mainly sponsor standards are used. The adherence to the requirements is mainly checked via spot-checks, and standardized checklists are mainly used for checking data standards.

Generally, in all companies participating in the survey, there is an awareness of the necessity of biometrical oversight. Although not necessarily perfect yet, all companies took measures to ensure good biometrical oversight and there is a common understanding, that it is not an option to resign from biometrical oversight.

Keywords: clinical trial, outsourcing, CRO (Clinical Research Organisation), vendor, oversight, supervision, quality management, biostatistics, data management

Zusammenfassung

Auslagerung („Outsourcing“) von Aktivitäten ist üblich und nimmt in der pharmazeutischen Industrie weiter zu – dies trifft nicht nur für die Durchführung von klinischen Studien, sondern insbesondere auch für Biostatistik- und Datenmanagement-Dienstleistungen zu.

Dies wirft die Frage auf, ob und wie Sponsoren die Möglichkeit behalten, CROs adäquat auszuwählen und zu kontrollieren und welche biometrische Expertise noch beim Sponsor vorhanden sein muss, um eine angemessene „Oversight“ sicher zu stellen. Nicht nur im Falle der partiellen Auslagerung, sondern vor allem im Falle der kompletten Auslagerung müssen Sponsoren die richtige Balance zwischen einem kompletten unkontrollierten Überlassen aller Aktivitäten und einer Ressource-intensiven Wiederholung aller Aktivitäten finden, um ihrer Sponsoren-Verpflichtung gerecht zu werden.

Um diese Fragen zu beantworten, wurde eine Umfrage unter den deutschen vfa-Mitgliedsunternehmen durchgeführt. Dieser Artikel beschreibt die spezifischen Oversight-Themen für die Auslagerung von Diensten aus den Bereichen Data Management und Biostatistik; es geht somit um Anforderungen hinsichtlich Datenqualität, Codierung und statistischer Analysen.

Diese Studie zeigt, dass die Mehrheit der Unternehmen die gleichen CROs für biometrische Dienstleistungen wie für andere Dienstleistungen verwenden und dass es einen bevorzugten Anbieter („preferred provider“) für biometrische Dienstleistungen in 65% der Unternehmen gibt. Es gibt etablierte, spezifische Anforderungen für die Ergebnisse der biometrischen Dienstleistungen hinsichtlich Codierung und Daten-Standards (Industriestandards wie MedDRA und CDISC). Für die Darstellung der Ergebnisse von statistischen Analysen und für die Datenqualität werden vor allem Standards der Sponsoren verwendet. Die Einhaltung der Anforderungen wird vor allem auf Stichproben-Basis überprüft, während standardisierte Checklisten vorwiegend bei der Überprüfung von Datenstandards Anwendung finden.

Grundsätzlich gibt es in allen Unternehmen, die an der Umfrage teilgenommen haben, ein Bewusstsein für die Notwendigkeit der biometrischen Aufsicht. Obwohl noch nicht überall unbedingt perfekt implementiert, ergreifen alle Unternehmen Maßnahmen, um eine gute biometrische Kontrolle zu gewährleisten. Es gibt ein gemeinsames Verständnis dafür, dass es keine Alternative zum effektiven biometrischen Oversight gibt.

Schlüsselwörter: Klinische Studie, Outsourcing, CRO (Clinical Research Organisation), Anbieter, Aufsicht, Beaufsichtigung, Qualitätsmanagement, Biostatistik, Data Management


Introduction

Outsourcing of activities is common and increasing in the pharmaceutical industry, not only for clinical trial conduct, or other trial related aspects but also for biostatistics and data management [1].

According to a recent study by Getz and Lamberti [2] outsourcing is the fastest growing area of R&D spending, exceeding 60 billion US Dollars in 2016. Biostatistics and data management plays an important role as shown in the 2016 NICE CRO Outsourcing Survey, where biostatistics and general toxicology/clinical trial data management were included in the top 5 list of needed services for preclinical/clinical trials [3]. Hatcher and Hughes [4] reported that the ratio of sponsors reporting a better performance (in terms of cost and speed) compared to those reporting a worse performance of their outsourced programs was 1:3. A similar conclusion is made by Roberts et al. [5] who stated that interactions between study site teams and CROs are not always efficient or productive. CROs are widely perceived as adding bureaucratic burden to trial conduct, increasing costs, and contributing to delays in protocol conduct, without a demonstrable benefit in the quality of data collection or improvements in patient safety.

It has to be stated that the highly important topic of quality of services – yet to be defined – and especially the topic “CRO Oversight” is rarely mentioned in peer reviewed journals or in publications from the internet. Roberts et al. [5] highlighted the challenge by measuring performance with easily measurable but potentially abused metrics like number of data queries generated and resolved.

Not only in the case of partial outsourcing but especially in the case of complete outsourcing, sponsors have to find the right balance between a complete “hands off” approach and a resource intensive in-house shadowing of activities to fulfil the requirements as specified in ICH E6 “the ultimate responsibility for the quality and integrity of the trial data always resides with the sponsor”, ICH [6]. As recently described by Stammer [1] a clear differentiation between measures for qualification before establishing an outsourcing contract and measures to ensure quality during conduct of the trial is needed. The need to implement efficient approaches is acknowledged in last year’s update (R2) to ICH E6 where it is stated in the introduction that the aim of the amendment is “to encourage implementation of improved and more efficient approaches to [...] oversight [...]”, ICH [7]. This awareness within regulatory agencies is also shown in the EMA reflection paper on risk based quality management in clinical trials: “the current practice can however be expensive and there are too many trials in which avoidable quality problems arise”, EMA [8].

Having this development in mind a survey was conducted among the German Association of Research-Based Pharmaceutical Member Companies (vfa; Verband forschender Arzneimittelhersteller). A total of 43 leading research-based pharmaceutical companies are currently organized in the vfa which represents two-thirds of the pharmaceutical market in Germany. The survey results are described on an overall level by Hennig et al. [9]. This article discusses in more detail the specific oversight topics for the outsourcing of data management and biostatistics services, e.g. requirements for data quality, coding, and statistical analyses.


Methods

In a joint project of the vfa sub-committee on clinical research and quality assurance (UA KliFo/QS) and the Biostatistician working group within the vfa, a questionnaire covering the major aspects on the current practice of CRO-selection and oversight was developed. Twenty-five vfa member companies are involved in these committees. The questionnaire for this survey was developed by the Biostatistics working group of the vfa.

The questionnaire referred to interventional clinical trials of phases II–IV, as trials of these phases are similar with regards to the outsourced services. It started with a section, in which the key elements were defined, to ensure a common understanding and interpretation of these elements, as shown below:

  • The term “CRO oversight” is used for any measure to control the performance, the deliverables and the efficiency of Contract Research Organizations (CROs) performing outsourced tasks on behalf of the pharmaceutical company or acting as the sponsor of a clinical study – not covered in this questionnaire: insourcing/temporary employment. Other terms typically used in this context include “CRO management”, “CRO supervision”.
  • The term “preferred provider” is used for any outsourcing model, in which one or several CROs are selected as primary supplier by a pharmaceutical company in order to perform defined tasks for a series of clinical studies. Other terms typically used in this context include “strategic (alliance) partner/vendor/CRO”.
  • The terms “local” and “global” refer to international companies with local subsidiaries in various countries. Here “global” refers to the CRO outsourcing on the international level within a company, whereas “local” refers to the German subsidiary and studies on the local German level – if applicable.

The questionnaire consisted of three sections:

The first part asked general questions about outsourcing models, the outsourced services, the selection and decision-making. Here it was assessed whether the outsourcing is organized locally or globally as well as the reasons for outsourcing. The global and local perspectives were addressed separately as the vfa member companies are acting with a global and local focus.

The second section dealt with the procedures ensuring CRO oversight and covered issues like CRO qualification, audits, SOPs, other oversight tools and escalation processes.

The third part covered specific oversight topics for the outsourcing of data management and biostatistics services, e.g. requirements for data quality or coding. This publication focusses on the third part. The first two parts were already published by Hennig et al. [9].

The complete questionnaire covered 52 items. The survey was conducted from August to October 2015 and captured the companies’ outsourcing status quo effective at this point in time. English language was selected for this questionnaire for ease of use within the companies. The questionnaire was sent out electronically by the vfa. The completed questionnaire was returned to the vfa and blinded. This ensured that no identification of the companies was possible for the analysis team, which was led by one of the authors. Before analysing the questionnaire descriptively, several quality control measures were performed in order to clean any data deficiencies and inconsistencies. In case of obvious data errors (e.g. an initial question was not answered, but the follow-up question was answered) the corresponding missing data was substituted.

The following five questions make up the third part of the questionnaire, which is the subject of this publication:

1.
In case of outsourcing Biostatistics/Data Management (BDM) services to a CRO: Is Biostatistics/Data Management (BDM) always outsourced to the same CRO as the rest of the study or is it always a different CRO?
2.
In case of outsourcing Biostatistics/Data Management (BDM) services to a CRO: Is the BDM department involved appropriately in the process of creating the contract with the CRO?
3.
Do you work with preferred CRO-partners especially for Biostatistics & Data Management?
4.
Do you have specific Biostatistics & Data Management requirements for the CRO?
5.
How do you check the adherence of the specific requirements (see question above)?

Questions No. 4 and 5 were divided into subcategories for coding, data standards, statistics (Question 4 only), data quality, and further specific requirements.

In addition, relevant articles were identified in a systematic literature search in Embase, Medline and other internet sources. Based on the initial systematic literature search performed for the first paper a new review, focussing on literature published after the initial search identified a total of 99 publications of potential relevance. After screening of the abstracts and full-texts a total of 6 relevant articles remained.


Results

18 (72%) out of the 25 companies of the UA KliFo/QS subcommittee participated. Three companies provided multiple responses: One company provided two questionnaires – one covering the local (German) outsourcing practice and one covering the global outsourcing practice. One company provided three questionnaires: one covering the local practice, one for the global practice and one additional questionnaire covering the outsourcing of monitoring activities only. Finally, one company divided their answers on two questionnaires: one for partly outsourcing activities, the other for full outsourcing activities. The following results include the multiple feedback from the three companies.

Most of the companies use the same CRO for BDM services as for rest of study services (Figure 1 [Fig. 1]).

Whereas for 12 of 16 companies (75%) the BDM departments were reported to be appropriately involved in the process of creating the contract, for the remaining 4 companies (25%) this was not the case (Table 1 [Tab. 1]).

There is a preferred provider for BDM services in 65% of the companies (Table 2 [Tab. 2]).

There are established, specific requirements for the main deliverables of BDM services: For coding and data standards these are dominantly industry standards (MedDRA, CDISC) (Table 3 [Tab. 3], Table 4 [Tab. 4]). For statistic outputs and data quality mainly sponsor standards are used (Table 5 [Tab. 5], Table 6 [Tab. 6]). Further sponsor requirements beyond coding, data standards and data quality are available in 45% of the cases (Table 7 [Tab. 7]).

The adherence to the requirements is mainly checked via spot-checks, and standardized checklists are mainly used for checking data standards (Table 8 [Tab. 8], Table 9 [Tab. 9], Table 10 [Tab. 10], Table 11 [Tab. 11]).


Discussion

This study describes the current practice of biometrical oversight of research based pharmaceutical companies in Germany and their biostatistics and data management departments (BDM) organized within the vfa. CRO oversight is deemed to be of increasing importance as it is assumed that by 2020 around 3/4 of all clinical trials will be performed by CROs [10]. The findings are based on a sub-study on biometrical oversight, results regarding CRO oversight on all other relevant aspects can be found in Hennig et al. [9].

This survey represents a large proportion of the current practice in these pharmaceutical companies/BDM departments. Sampling representativeness is limited by three factors: the selection process of this survey, in which 25 pharmaceutical companies represented in the vfa were considered, a return rate of 72% and the fact that three companies submitted multiple questionnaires. Overall 43 research-based pharmaceutical companies are organized in the vfa, representing two-thirds of the pharmaceutical market in Germany. Those 25 companies participating in the vfa committee “Clinical research and quality assurance (UA KliFo/QS)” and in the vfa working group “Biostatistics” were involved in the survey (see chapter 3). Small biotech companies with no biostatistics or data management (BDM) department are not represented within the 25 companies. Therefore, the sample of 25 companies contains all major companies with a BDM department. Another limitation of the study is that a potential split into answers from global functions and local organizations is retrospectively not possible, which may lead to some degree of inconsistency.

Effective biometrical oversight can only be ensured, when biometrical tasks are quality-controlled. This necessitates biometrical expertise on the sponsor’s side. This expertise is not a distant ideal but an indispensable necessity (“the ultimate responsibility for the quality and integrity of the trial data always resides with the sponsor”, ICH [6]). In larger companies to ensure consistency between headquarters and across all affiliates, there should be a common set of rules on all aspects of biometrical oversight. These rules are normally set up by headquarters and resemble the norm, that must be achieved.

There are various ways to achieve the above-mentioned necessity and the derived aspects, that are to be monitored/controlled: The expertise

  • (i) already exists in the company,
  • (ii) can be used from another function of the organization or
  • (iii) can/must be outsourced to a third party (e.g. consultant).

Ad (i): Own functions for biometrical oversight are a standard in global organizations. In this survey, there was not a single response indicating a lack of a biometrical function in the global organization. In headquarters, responsible for the pivotal studies and the submissions to the authorities, this is not surprising. It is important that these standards are maintained for clinical trials and non-interventional studies and projects like re-analysis of secondary data and other epidemiologic studies.

In local organizations a separate biometrical function is not necessarily standard so that scenarios (ii) and (iii) apply.

Ad (ii): In some instances the global function is willing to support the local organization with biometrical oversight. This seems to be an easy approach in cases, where headquarter and affiliate are in the same country.

Ad (iii): This scenario carries the most uncertainty as external personnel supervise other external service providers in the name of the sponsor. None of the vfa companies responding to the survey has chosen this scenario.

Although there is a quality gradient in general (not necessarily in every case) between scenarios (i) to (iii) each of them is better, than abstaining from biometrical oversight

Having this as general goal, the survey looked more closely into specific aspects.

Most survey participants have requirements for coding and these refer to the global standards like MedDRA. It’s worth noting, that 1/3 of the participants additionally have sponsor specific coding standards in certain areas. This reflect the fact, that coding is interpreted in a wide sense, such, that guidance on e.g. preparation of sponsor-specific analytical data sets is also regarded as coding. CDISC is the common data standard requested (3/4 SDTM, 2/3 ADaM). The fact that CDISC is not mandatory in all cases might reflect the local component of the survey. It might also reflect the view of sponsors that the additional effort associated with the implementation of CDISC is not outweighing the benefits due to the standardisation in case no submission to FDA or PMDA is planned.

The fact, that 11% of survey participants answered, that they have no specific requirements to check data quality does not mean that there is no data quality checking at all. Quality checking is often implemented within the CRO contract. Quality checks remain crucial especially for the sponsor, independent of previous checks by the CRO.

Two-thirds (65%, 11/17) of all survey participants have specific preferred partners on biometrical tasks. This is in order to expect higher quality in biometrical tasks, because of better trained external partners. This answer does not necessarily mean, that the remaining 1/3 does not have similar quality steps in place. Two possible explanations are:

  • The overall company strategy may generally follow various options to outsource:
    (a) full-service-CRO (all tasks delegated to one single service provider),
    (b) selective outsourcing (different CROs for different tasks) or
    (c) preferred partners solely for biometrical tasks.
    It is obvious, that in case (a) no specific preferred partners for biometrics exist, nevertheless there are well trained partners at hand.
  • The cooperation with preferred partners depends on the type of studies: local studies might have different cooperation models then global studies. Cooperation models might differ between interventional and non-interventional studies as well as between early phase studies and pivotal or late-phase studies. Therefore, there is no uniquely correct answer to report in the survey.

It is surprising though, that the BDM department is involved appropriately in the process of creating the contract with the CRO solely in 75% (12/16) of the cases and not bydefault. This is to some extent worrisome as the contract for a given project is a cornerstone for general as well as for biometrical oversight. Various authors emphasize the importance of both measures for qualification before establishing an outsourcing contract and measures to ensure quality during the conduct of a trial [11], [1]. It seems that other departments have not realized the high relevance of maintaining good biometrical oversight, which starts with the contract, and it is the BDM department’s duty to continuously point out this importance internally. Also, it could be that these BDM departments have no further specific statistics and data management requirements, as holds true for 55% of the responding participants. This fact may result in using rather generic oversight-chapters for the contract with the CRO, without further involvement of the BDM department.

There is a further aspect which is not completely covered by the CRO oversight, which is the oversight on technical providers [12]. Full oversight necessitates that this aspect is covered, too.

In general: It is not clear, if the return rate is a biased sample of solely those companies, that are aware of the necessity and whether a survey including companies not in the vfa might result in a different picture. But generally, in all companies participating in the survey, there is an awareness of the necessity of biometrical oversight. Although not necessarily perfect yet, all companies took measures to ensure good biometrical oversight and there is a common understanding, that it is not an option to resign from biometrical oversight.

This survey provided a first insight into current practices of Biostatistics & Data Management (BDM) departments within the UA KliFo/QS subcommittee and the working group Biostatistics of the vfa. Companies without a BDM department did not participate, so it has to be further evaluated to what extent this survey within this subset of pharmaceutical companies is representative for the general practice.

In addition to the aspects above, a continuous effort resulting in a better understanding of the difference in oversight across sponsors is needed. For example, 60% of the participants have sponsor standards besides using industry standards like MedDRA, CDISC. The authors see a need to heighten the awareness of CRO oversight within the statistical and data management community by bringing this on the agenda of scientific meetings which could trigger further exchanges across companies to get a deeper insight. Perspectively a regular and continuous exchange of representatives of BDM departments of various companies is put in place to ensure:

  • Evaluation of new trends
  • Learning from experiences
  • Ensuring common standards

Notes

Competing interests

The authors declare that they have no competing interests.


References

1.
Stammer C. Qualification measures and monitoring of the quality of CROs and subcontractors in clinical trials. Pharm Ind. 2017:198-202.
2.
Getz K, Lamberti MJ. Hard truths about executing outsourcing relationships. Pharm Outsourcing. 2016 Jul 21;17(4). Available from: http://www.pharmoutsourcing.com/Featured-Articles/189343-Hard-Truths-about-Executing-Outsourcing-Relationships/ External link
3.
Another exciting Year for Clinical Research Outsourcing. Am Pharm Rev. 2016 Jan 31. Available from: http://www.americanpharmaceuticalreview.com/Featured-Articles/183037-Another-Exciting-Year-for-Clinical-Research-Outsourcing/ External link
4.
Hatcher A, Hughes J. Applying best practices from other industries to sponsor-CRO relationships. Pharm Outsourcing. 2016 Jan 31;17(1). Available from: http://www.pharmoutsourcing.com/Featured-Articles/182575-Applying-Best-Practices-from-Other-Industries-to-Sponsor-CRO-Relationships/ External link
5.
Roberts DA, Kantarjian HM, Steensma DP. Contract research organizations in oncology clinical research: Challenges and opportunities. Cancer. 2016 May;122(10):1476-82. DOI: 10.1002/cncr.29994 External link
6.
International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). Guideline for Good Clinical Practice E6(R1) – Current Step 4 version dated 10 June 1996. ICH; 1996. Available from: https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R1_Guideline.pdf External link
7.
International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2) – Current Step 4 version dated 9 November 2016. ICH; 2016. Available from: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R2__Step_4_2016_1109.pdf External link
8.
European Medicines Agency. Reflection paper on risk based quality management in clinical trials. EMA/269011/2013. London: EMA; 2013. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2013/11/WC500155491.pdf External link
9.
Hennig M, Hundt F, Busta S, Mikus S, Sanden PH, Sörgel A, Ruppert T. Current practice and perspectives in CRO oversight based on a survey performed among members of the German Association of Research-Based Pharmaceutical Companies (vfa). GMS Ger Med Sci. 2017;15:Doc02. DOI: 10.3205/000243 External link
10.
Rizk MA. Data trust and transparency: New era of clinical outsourcing and partnerships. Pharm Outsourcing. 2017 Mar 17;18(2). Available from: http://www.pharmoutsourcing.com/Featured-Articles/336106-Data-Trust-and-Transparency-New-Era-of-Clinical-Outsourcing-and-Partnerships/ External link
11.
Schinzel S, Hundt F, Theobald K, Theis F, Buchmann A, Herbold M. CRO Precontract Audits. Applied Clinical Trials. 2009 Jun 02. Available from: http://www.appliedclinicaltrialsonline.com/cro-precontract-audits  External link
12.
Nedelcheva Y. The dynamics of outsourcing: A bibliometric study. Pharm Outsourcing. 2017 Mar 17;18(2). Available from: http://www.pharmoutsourcing.com/Featured-Articles/336116-The-Dynamics-of-Outsourcing-A-Bibliometric-Study/ External link