gms | German Medical Science

GMS German Medical Science — an Interdisciplinary Journal

Association of the Scientific Medical Societies in Germany (AWMF)

ISSN 1612-3174

Quality assurance in non-interventional studies

Research Article

  • corresponding author Karlheinz Theobald - Sanofi-Aventis Deutschland GmbH, Clinical Operations, Frankfurt/Main, Germany
  • author Müge Capan - Technical University (TU) Berlin, Institute of Machine Tools and Factory Management, Dept. Quality Science, Berlin, Germany
  • author Marlis Herbold - Sanofi-Aventis Deutschland GmbH, Clinical Operations, Frankfurt/Main, Germany
  • author Stefan Schinzel - Sanofi-Aventis Deutschland GmbH, Clinical Operations, Frankfurt/Main, Germany
  • author Ferdinand Hundt - Sanofi-Aventis Deutschland GmbH, Clinical Operations, Berlin, Germany

GMS Ger Med Sci 2009;7:Doc29

doi: 10.3205/000088, urn:nbn:de:0183-0000881

This is the English version of the article.
The German version can be found at: http://www.egms.de/de/journals/gms/2009-7/000088.shtml

Received: May 14, 2009
Revised: October 5, 2009
Published: November 9, 2009

© 2009 Theobald et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en). You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.


Abstract

Nowadays, drug research and surveillance after authorisation becomes more and more important for several reasons. Non-interventional studies (NIS) investigate various aspects of drug use including efficacy and safety under real life conditions. Such kind of health services research should be on a high scientific, methodological and organisational level. Therefore accompanying measures to improve or to keep the quality are highly recommended. The aim of quality management is: first to avoid bias of results by using an appropriate study design and an adequate data analysis, second to assure authenticity, completeness and validity of the data and third to identify and resolve deficiencies at an early stage. Basic principles are laid down in corresponding guidelines and recommendations of authorities, institutes and societies. Various guidelines for good epidemiological practice (GEP) were published by the U.S. Food and Drug Administration (FDA) and international and regional societies for epidemiology. In addition in Germany the Federal Institute for Drugs and Medical Devices (BfArM) together with the Paul Ehrlich Institute (PEI) and the German Association of Research-Based Pharmaceutical Companies (VFA) have published respectively recommendations dealing with quality aspects of non-interventional observational studies. Key points are the advanced publishing of information about the project, developing of a study plan/protocol containing the scientific objectives, a sample size justification and a description of the planned analyses and the publishing of a summary of the results timely after completion of the study. The quality of the data can be improved by using standardized case report forms (CRF) and the CRF should be reviewed and tested before start of study by some participants. A source data verification (SDV) should be performed in randomly selected centres – in between 2% and 5% of the centres depending on the number of participating centres. Before start of statistical analysis a statistical analysis plan (SAP) should be created. The use of standardized tables and figures is highly recommended. The basis of the report writing should be the STROBE-statement “Strengthening the Reporting of Observational studies in Epidemiology Initiative” containing a checklist of 22 points to be covered in the report. The development of own standard operating procedures (SOP) describing the processes during planning, conduct and evaluation of a non-interventional study as well as the quality management and the regular training of all involved people is also highly recommended. All accompanying measures to improve or to keep the quality of the NIS should not violate the concept of non-intervention.

Keywords: non-interventional studies, quality management, Good Epidemiological Practice (GEP), study plan, standard operating procedures (SOP)


1 Introduction

Since a long time, non-interventional studies (NIS; e.g. registries) have been making valuable contributions to medical research, especially in Germany. These studies are on a high scientific, methodological and organisational level and therefore require accompanying quality assurance measures.

Basic principles for this kind of research are laid down in guidelines and recommendations for assurance of good epidemiological practice (GEP) [1] and other additional international guidelines [2], [3], [4]. The aim of GEP is to establish a quality standard for epidemiological research. The guidelines contain partially detailed recommendations regarding the topics ethic, research questions (e.g. a priori defined hypothesis), study plan, biological sample databases, quality assurance, data storing and data documentation, analysis, data protection, contractual provisions and interpretation of research results. Measures for quality assurance are treated relatively unspecific.

According to the EU Directive 2001/20/EG article 2c, non-interventional studies, in contrast to clinical trials, were described as “studies where the medicinal products are prescribed in the usual manner in accordance with the terms of the marketing authorisation. The assignment of the patient to a particular therapeutic strategy is not decided in advance by a trial protocol but falls within current practice and the prescription of the medicine is clearly separated from the decision to include the patient in the study.” [5]

According to § 4 (23) German Drug Law, non-interventional studies are studies where “findings from the treatment of persons with drugs according to the terms of the marketing authorisation are analysed with epidemiological methods; thereby treatment, including diagnosis and control, does not follow any predefined study plan but only the medical practice.” [6]

In summary, non-interventional studies (non-GCP studies) fulfil the following criteria:

  • The collected data will be analysed with epidemiological methods.
  • The diagnostic methods and other observational procedures are according to medical practice. There are no other diagnostic methods disposed.
  • The patient is treated within the current practice. There are no rules for the physician for how to treat the patients.
  • Only marketed and commercial drugs or medical devices are used according to the information stated in the approval.

Types of non-interventional studies are:

  • case-control-studies
  • registries
  • cohort-studies (especially “Anwendungsbeobachtungen” = AWB = post authorisation observational product studies)

Cohort-studies include surveys to compare different therapies.

The Federal Institute for Drugs and Medical Devices (BfArM) together with the Paul Ehrlich Institute (PEI) had developed recommendations (draft) for planning, conducting and analysing non-interventional studies, especially for AWB [7]. Furthermore, the German Association of Research-Based Pharmaceutical Companies (VFA) has published recommendations for improving the quality and transparency of non-interventional studies (NIS) in January 2007 [8]. In these recommendations the following quality criteria for planning, conducting and analysing NIS are mentioned:

  • advice from federal state law established Ethics Committees
  • written patient information and informed consent
  • information about (planned) NIS in public accessible registries
  • publication of a summary of the results within one year after completion of NIS
  • development of standard operating procedures with description of:
    • processes for planning, conducting and analysis
    • measures for quality assurance, especially for representativeness and validation of the data raised

The aim of quality assurance is to make valid, scientific statements based on the results in NIS, meaning to avoid possible bias of results by using an appropriate study design and an adequate data analysis, assure authenticity, completeness and validity of the data and to identify and resolve deficiencies at an early stage [9].

Laws [6], guidelines [5], [10] and recommendations [7], [8], [11] relating to NIS are very helpful to implement quality assurance measures.

Within this paper, possibilities shall be identified which measures of quality assurance in AWB or other NIS like registries can be used without compromising the character of non-intervention.


2 Measures of quality assurance at different stages of NIS

The quality of a NIS includes the requirements on scientific/professional qualification as well as the personal, technical and spatial equipment of persons/sites involved in the NIS (quality of structure). Quality includes the requirements on the processes during planning, conducting, analysis and report writing, publication and archiving (quality of processes) and quality includes the requirements on the quality of the data and methods for analysing the data (quality of results).

2.1 Measures in the planning phase

In contrast to clinical trials, it is strongly recommended when planning a NIS to preserve the aspect of non-intervention for all planned measures [12].

Quality assurance is guaranteed by the description of the contents and the responsible persons. Form and scale of quality assurance is orientated on the timely, organisational, technical and financial basic conditions set in the study concept. The following measures are to be considered:

2.1.1 Study plan

Essential for the quality of the NIS is the development of a mainly standardised study plan. NIS require specifications in advance but no details for intervention are allowed to be defined. The following statements shall be covered in the study plan [7]:

  • Formulation of one (or more) precise questions and working hypotheses,
  • justification of the sample size,
  • description of measures for quality assurance and
  • description of the statistical methods.

In addition to these statements, the guideline 3 (study plan) of the “Guidelines and recommendations for assuring good epidemiological practice” (GEP) [1] also contains further details which shall be covered by a detailed and binding study plan: study type, study population, selection and recruiting process of study participants, definition and determination of primary variable, expositions respectively risk factors, potential confounder, data collection and concept of archiving, measures for assuring data protection and ethical principals and a timeline with definitions of responsibilities.

Also the appropriateness of the chosen data base, e.g. in registries, should be discussed in the study plan.

Furthermore, the acquisition of serious adverse events (SAE) should be considered in addition to the normal acquisition of adverse drug reactions (ADR). This documentation is part of the due diligence and ensures also a better comparability of the data with the results of clinical trials. ADR/SAE and concomitant diseases should be coded (e.g. with MEDRA [13], [14]) as well as concomitant drugs. The required processes shall be described in detail in the study plan.

2.1.2 Quality plan

The quality plan describes which quality control and quality assurance activities concerning the NIS have to be conducted. Thereby all quality relevant tasks are defined, concrete quality objectives are described as well as responsibilities for single tasks are named. Beyond this, the resources which need to be at disposal are defined accordingly.

2.1.3 Design of case record forms (CRF, electronic-CRF)

Standardised case record forms ensure the completeness of the data collected, develop the comparability among the different studies and ease working processes. They need to be adjusted according to the target awareness respectively the study objective. The check of the CRF by e.g. tentative completion of CRF drafts by participating physicians – patient report forms by the patients – before conducting the NIS contribute to quality assurance of the content and design of the CRF and therefore lead to a better data quality. It is to be ensured that the nature of data collected can answer all questions of the NIS.

2.1.4 Ensuring the representativeness

In addition to clinical trials, NIS can provide results from routine medication. Therefore, applicable measures should make sure that the participating patients and physicians of the NIS as well as therapeutic processes result in representative picture of the “medical practice”. An even regional distribution of physicians/sites shall be ensured. According to project, the allocation of physicians/sites shall consider other criteria as practice size, professional qualifications etc. For this purpose, internal and external surveys about practice prevalence in the indication are useful. If applicable, (stratified) random selection of physicians/sites is to be made. The amount of enrolled patients per physician/site should be limited to avoid that one big site or a few big sites dominate the results and therefore compromise the representativeness.

A measure to achieve the representativeness of the patients is the demand to the physician to include and to document consecutively the first patients in question in the NIS. An assessment of the representativeness can be done during the data analysis by comparing the investigated patient population with other studies and literature.

2.1.5 Training

All NIS participants like the data collecting staff (physicians, nurses, etc.), the data entry staff, the data manager, the biostatisticians, the medical personnel of the institute or the pharmaceutical company etc. are to be selected carefully and trained on the project specific aspects [8].

For the conduction of the required training and for knowledge tests, besides other forms of training, e-learning tools are proper; especially in case of a huge amount of employees and a decentralised organisation where the possibility for presence training and group training is not given. For the planning of a NIS, meetings are recommended with the participating data collecting staff and the study team and, if applicable, employees of the contracted Clinical Research Organisation (CRO). During these meetings objectives, study plan and case record forms of the NIS shall be discussed. If applicable, study accompanying meetings of the parties for a second training and for process optimisation are useful and necessary.

2.1.6 Advice from the Ethics Committee

An opinion from a federal state law established Ethics Committee is recommended [7], [8], [11].

According to the professional rules for physicians (§ 15 Research, Muster-Berufsordnung für Ärztinnen und Ärzte) [15]before conducting biomedical research in human beings – excluding exclusively epidemiological research projects –, physicians must ask for opinion of a Ethics Committee build by a medical association or a medical faculty about the ethical and legal questions connected to the objectives.

With the implementation, several federal medical councils (Landesärztekammern (LÄK)), e.g. Hessen and Bavaria, have included the epidemiological research in § 15 in their individual professional rules.

LÄK Hessen [16]: “The physician needs the opinion of a Ethics Committee built by a medical association or a medical faculty about the ethical and legal questions connected to his project before conducting biomedical research in human beings and epidemiological research .

LÄK Bayern [17]: “The physician needs the opinion of a Ethics Committee built by a medical association or a medical faculty about the ethical and legal questions connected to his project before conducting biomedical research in human beings or epidemiological research with individual-related data .

In case of multicenter studies Ethics Committees often require a new consultation. In contrast the LÄK Berlin [18] disclaims a new consultation in case the opinion of an Ethics Committee established according to national law is available.

2.2 Ensuring the transparency

It is recommended that [8]: “with study start, information about planned NIS (study title, objectives, name of study manager, planned amount of study sites as well as aspired sample size) are published in a public accessible registry” (e.g. online registry of VFA: http://www.vfa.de/de/forschung/nisdb/ and/or international registry: http://www.clinicaltrials.gov/).

Furthermore, according to German Drug Law (§ 67 article 6), a general disclosure duty is demanded for pharmaceutical companies: “The pharmaceutical employer should announce investigations which are designed to collect cognition in the practice of marketed or registered drugs, to the National Association of Statutory Health Insurance Physicians (kassenärztlichen Bundesvereinigungen), the national confederations of health insurance funds (Spitzenverband Bund der Krankenkassen) and the assigned higher federal authority (Bundesoberbehörde) immediately. Thereby site, time and objective (study plan) of the NIS are to be indicated and all physicians involved are to be named.[6]

The reward of the participating physicians has to be reported to the National Association of Statutory Health Insurance Physicians (kassenärztlichen Bundesvereinigungen), the national confederations of health insurance funds (Spitzenverband Bund der Krankenkassen) and the corresponding Ethics Committees. The amount of the reward should not stimulate the physician to prescribe or advocate the medical product under investigation. In general the medical fee schedule (GOÄ) will be used as indication for an adequate reward (e.g. 120 € per beginning hour [19]).

2.3 Measures for data collection and data entry

2.3.1 First check of completeness

After the receipt of the filled in CRF, a first check of completeness is done (e.g. signature of physician, check of adverse events).

2.3.2 Source data verification

The quality of the data collected can be checked by a comparison of the physician- and patient-documentation. This check can be done in form of a source data verification (SDV) of clinical results at the physician’s practice. SDV is a comparison of the data between the CRF and the medical records and vice versa. Discrepancies can be resolved by queries to the physician (in case of an AWB this is recommended after the site has completed the study). Advisable are random checks (random sample “physician” and/or random sample “patient” and/or random sample “data”). Depending on the project, at x% of all physicians/sites (simple random sample or stratified random sample) the primary variable/the main objective parameters and the safety data are checked for all patients. The size of the random sample (x%) is project related and is orientated by the total amount of participating physicians/sites, the expected homogeneity respectively heterogeneity in relevant structure variables of the sites or the patients as well as the number of patients per physician. A survey of VFA companies revealed that between 2% (e.g. for a big amount of participating sites) and 5% (up to 10% if necessary) of the sites were chosen randomly (personal communication, [20]).

2.3.3 Verification and validation

Quality assuring measures shall be established during data entry. This can be done in form of a quality control by which the recorded data and the data in the CRF were compared (e.g. double data entry or single data entry with randomized check of the database with the data documented in the CRF).

The verification and validation of critical variables (e.g. adverse drug reactions) is done in agreement with the physician. Ambiguity through a lack of readability, incompleteness and implausibility can be solved by queries from the data management.

As a basis for the plausibility checks, standardised and/or project specific criteria are recommended (e.g. range checks or cross checks of variables). A data review meeting before data base lock, which involves all parties, to discuss the data and remaining existing implausibilities (with the consequence of a possible exclusion of data) is also recommended.

2.4 Measures during the analysis

It is recommended to standardize the analysis of NIS through specification of standard tables and standard figures in order to promote the use of validated standard analysis programs and so to achieve time and cost savings and better comparability of the results amongst different studies. Templates allow the standardised analysis and presentation of results of the collected data. Before start of the analysis, it is highly recommended to develop a statistical analysis plan (SAP) including a description how to handle missing or implausible data (e.g. indicate imputation strategies). This strategy ensures the transparency and consistency of the analysis strategy. According guidance is provided by the guidelines for clinical trials as e.g. ICH Topic E9 “Statistical Principles for Clinical Trials” [21].

2.5 Measures during the creation of NIS reports

The final report which is developed after analyses shall fulfil quality requirements as traceability, transparency and objectivity. Traceability means that all activities and used methods within the scope of the study are presented quite clearly for the reader. Transparency means in this context that all study results are presented and no information is hidden. Objectivity is used to eliminate subjective influences on the interpretation of the results. In this regard, “Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Initiative” provides a set of quality assuring measures for developing reports in epidemiological studies [22].

An international group of epidemiologists, statisticians and publishers of medical journals has developed these recommendations for writing scientific papers in which the results of such observational studies are reported. All details to be noted were presented in a checklist with 22 points (http://www.strobe-statement.org/index.php?id=checklists). Amongst other this includes the presentation of the used study design, the predefined hypothesis, the complete description of statistical methods, the details of the data management activities and the discussion of possible sources for bias of the results. The format of the report is flexible and can be adapted to authors or journals needs. The STROBE statement can improve the quality and comparability of reports of epidemiological studies. It is expected that the quality of the report/publication in epidemiological studies is also improved as it was already the case after the publication of the CONSORT (Consolidated Standards of Reporting Trials), Statements for randomised clinical trials [23], [24].

Furthermore, at least a short report of the results of NIS shall be sent to the participants.

In the case that a publication is not possible, at least a summary of the results shall be published in a public registry (e.g. http://www.clinicaltrialresults.org/).

2.6 Measures for archiving NIS

After finalisation of a NIS, the report and all other documents shall be archived (e.g. according to an archiving SOP). This includes the electronic archiving of the data analysed. Within the recommendations of the BfArM from 1998, article 14 says: “It is recommended that all documents for later access and analyses are to be archived for a minimum of 10 years[11].


3 Example

A non-interventional, cross-sectional, multicenter study [25] was conducted to compare the direct diabetes-specific treatment costs of two basal insulins. Besides the creation of a scientific study plan, the following quality assurance measures were implemented:

  • feasibility assessment of the case record forms
  • patient information and informed consent, verbal and in writing
  • telephone monitoring after inclusion of the first patient
  • double data entry
  • extensive review of the CRF and data checks for completeness and plausibility of the data
  • source data verification in 10% of the physicians

The quality of the data concerning completeness and plausibility was comparable with the quality of data from clinical trials. Only in rare cases data were excluded from analysis.


4 General quality assurance measures

The development of own standard operating procedures (SOP) describing the processes during planning, conduct, evaluation and reporting of a non-interventional study as well as a description of the accompanying quality measures to achieve a high quality of the data is highly recommended.

A regular training of all involved people regarding the legal conditions and the recommendations of the agencies and expert societies is also highly recommended.

In addition the quality and validation of the data management and data analysis systems has to be assured continuously.


5 Discussion

For all quality assuring measures the aspect of non-intervention should be discussed, but should be respected for AWB.

The objective of NIS is the presentation of an unaffected picture of daily medical practice. It is not always possible to make clear boundaries between “medical practice” and “intervention”. For example, it is still unclear if the assignment of patient’s questionnaires or patient’s diaries already violate the character of non-intervention. The same applies to the written patient's information and informed consent forms but eventually also to the advice of the Ethics Committee. Surely, there are serious concerns about that quality assuring measures influence the behaviour of physicians, medical staff and patients. Therefore the quality assuring measures shall not have (any) influence on the behaviour of patients and physicians and so on the results of NIS.


6 Conclusion

In Germany, important steps for improving quality or adherence of high quality of non-interventional studies are the consideration of guidelines and recommendations for assuring good epidemiological practice (GEP) [10], the combined draft recommendations of the BfArM and PEI for planning, conducting and analysing “Anwendungsbeobachtungen” [7] as well as the VFA recommendations for improving the quality and transparency of non-interventional studies [5]. In this case, the organisations and pharmaceutical companies which conduct NIS should train their employees and all participants in NIS on a regular basis. The aspect of non-intervention should be respected when applying measures of quality assurance.


Notes

Conflicts of interest

None declared.

Acknowledgements

The authors thank Nicole Garten for assistance in preparing the English version of the manuscript.

Remark

Parts of this publication are included in the diploma thesis of Müge Capan, student at the faculty for transportation and engine systems (V) at Technical University (TU) Berlin, Institute of Machine Tools and Factory Management, Dept. Quality Science.


References

1.
Arbeitsgruppe Epidemiologische Methoden der Deutschen Arbeitsgemeinschaft für Epidemiologie (DAE). Leitlinien und Empfehlungen zur Sicherung von Guter Epidemiologischer Praxis (GEP). [Stand: April 2004]. Available from: http://www.gesundheitsforschung-bmbf.de/_media/Empfehlungen_GEP.pdf External link
2.
US Department of Health and Human Services, Food and Drug Administration; Center for Drug Evaluation and Research (CDER); Center for Biologics Evaluation and Research (CBER). Guidance for Industry – Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment. March 2005. Available from: http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM126834.pdf External link
3.
International Society for PharmacoEpidemiology (ISPE). Guidelines for Good Pharmacoepidemiology Practice (GPP) [issued 1996, revised August 2004 & April 2007]. Bethesda: ISPE; 2009. Available from: https://www.pharmacoepi.org/resources/guidelines_08027.cfm External link
4.
International Epidemiological Association (IEA) European Federation. Good Epidemiological Practice (GEP) – IEA Guidelines for proper conduct of epidemiological research. November 2007.
5.
Directive 2001/20/EC of the European Parliament and of the Council of 4 April 2001 on the approximation of the laws, regulations and administrative provisions of the Member States relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use. Official Journal of the European Union 2001;L 121, 1/5/2001:34-44. Available from: http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/vol-1/dir_2001_20/dir_2001_20_en.pdf External link
6.
Gesetz über den Verkehr mit Arzneimitteln (Arzneimittelgesetz – AMG) Arzneimittelgesetz in der Fassung der Bekanntmachung vom 12. Dezember 2005 (BGBl. I S. 3394), das durch Artikel 1 der Verordnung vom 28. September 2009 (BGBl. I S. 3172) geändert worden ist. Berlin: Bundesministerium der Justiz; 2009. Available from: http://www.gesetze-im-internet.de/amg_1976/BJNR024480976.html External link
7.
Bundesinstitut für Arzneimittel und Medizinprodukte (BfArM); Paul-Ehrlich-Institut (PEI). Gemeinsame Empfehlungen des Bundesinstituts für Arzneimittel und Medizinprodukte und des Paul-Ehrlich-Instituts zur Planung, Durchführung und Auswertung von Anwendungsbeobachtungen (Entwurfsfassung vom 9. Mai 2007 zur Kommentierung durch die Fachöffentlichkeit). 2007. Available from: http://www.bfarm.de/cln_012/nn_1198726/SharedDocs/Publikationen/DE/Arzneimittel/1__vorDerZul/klin-pr/nichtInterventPruef/EmpfAWBEntwurf,templateId=raw,property=publicationFile.pdf/EmpfAWBEntwurf.pdf External link
8.
Verband Forschender Arzneimittelhersteller e.V. (VFA). VFA-Empfehlungen zur Verbesserung der Qualität und Transparenz von nicht-interventionellen Studien [31. Januar 2007]. 2007. Available from: http://infomed.mds-ev.de/sindbad.nsf/0/f99eec1f4951d5c6c12572c800335eee/$FILE/VFA-Empf-NIS_070131.pdf External link
9.
de la Haye R, Herbold M. Anwendungsbeobachtungen, Leitfaden für die praktische Durchführung. 2nd ed. Aulendorf: Editio Cantor Verlag; 2006.
10.
Commission Directive 2005/28/EC of 8 April 2005 laying down principles and detailed guidelines for good clinical practice as regards investigational medicinal products for human use, as well as the requirements for authorisation of the manufacturing or importation of such products. Official Journal of the European Union 2005;L 91, 9/4/2005:13-9. Available from: http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/vol-1/dir_2005_28/di r_2005_28_en.pdf External link
11.
Bundesinstitut für Arzneimittel und Medizinprodukte (BfArM). Empfehlungen zur Planung, Durchführung und Auswertung von Anwendungsbeobachtungen, 12. November 1998. BAnz. 1998;229:16884.
12.
Koch A, Windeler J, Abel U. Anwendungsbeobachtungen: zu Begriff und Nutzen [Therapeutic indications: on the concept and applications]. Med Klin (Munich). 1996;91(2):103-5.
13.
European Parlament. EudraLex – Volume 9: Pharmacovigilance guidelines [September 2008]. 2008. Available from: http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/vol9_en.htm External link
14.
Kübler J, Vonk R, Beimel S, Gunselmann W, Hommering M, Nehrdich D, Köster J, Theobald K, Voleske P. Adverse Event Analysis and MedDRA: Business as Usual or Challenge? Drug Inf J. 2005;39(1):63-72.
15.
(Muster-) Berufsordnung für die deutschen Ärztinnen und Ärzte der Bundesärztekammer, – MBO-Ä 1997 – in der Fassung der Beschlüsse des 100. Deutschen Ärztetages 1997 in Eisenach […] geändert durch den Beschluss des Vorstands der Bundesärztekammer am 24.11.2006. 2006. Available from: http://www.bundesaerztekammer.de/downloads/MBOStand20061124.pdf External link
16.
Landesärztekammer Hessen. Satzung zur Änderung der Berufsordnung für die Ärztinnen und Ärzte in Hessen [geändert am 10. April 2007]. Hessisches Ärztebl. 2007;5:325-30. Available from: http://www.aerzteblatt-hessen.de/pdf/haeb07_325.pdf#search=%22berufsordnung%22&view=%22fit%22 External link
17.
Bayerische Landesärztekammer. Berufsordnung für die Ärzte Bayerns: Bekanntmachung der Neufassung vom 1. August 2005. Bayerisches Ärztebl. 2005;60(Spezial 2). Availabe from: http://www.blaek.de/presse/aerzteblatt/2005/SD_Berufsordnung.pdf External link
18.
Ärztekammer Berlin. Neufassung der Berufsordnung der Ärztekammer Berlin vom 30. Mai 2005, zuletzt geändert durch den 2. Nachtrag vom 26.09.2006. [Stand 11.01.2007]. 2007. Available from: http://www.aerztekammerberlin.de/10arzt/30_Berufsrecht/06_Gesetze_Verordnungen/30_Berufsrecht/331_BerufsO.pdf External link
19.
Koyuncu A. Vertragsgestaltung und Ärztevergütung bei Anwendungsbeobachtungen und anderen nichtinterventionellen Studien. Dtsch Z klin Forsch. 2009;3/4:54-61.
20.
Hahn M, Bethke TD, Hecht A, Henn D, Ruppert T, Hundt F. Qualitätssichernde Maßnahmen in nicht-interventionellen Studien: Ergebnisse einer Umfrage unter den Mitgliedsunternehmen des Verbandes Forschender Arzneimittelhersteller [Quality assurance measures in non-interventional studies: Results of a survey among the members of the Association of Research-Based Pharmaceutical Companies]. GMS Ger Med Sci. 2008;6:Doc12. Available from: http://www.egms.de/static/en/journals/gms/2008-6/000057.shtml External link
21.
European Medicines Agency. ICH Topic E9, Statistical Principles for Clinical Trials [Stand: September 1998]. London: EMEA; 2006. Available from: http://www.emea.europa.eu/pdfs/human/ich/036396en.pdf External link
22.
Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M; STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Epidemiology. 2007;18(6):805-35. DOI: 10.1097/EDE.0b013e3181577511 External link
23.
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang T; CONSORT GROUP (Consolidated Standards of Reporting Trials). The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001;134(8):663-94.
24.
Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C, Gaboury I. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust. 2006;185(5):263-7.
25.
Schön, Theobald, Kallert, Schädlich. How to gain high data quality in a non-interventional trial taking the LIVE-COM study as example; Poster accepted for ISPOR 12th Annual European Congress; October 2009.