gms | German Medical Science

50. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds)
12. Jahrestagung der Deutschen Arbeitsgemeinschaft für Epidemiologie (dae)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie
Deutsche Arbeitsgemeinschaft für Epidemiologie

12. bis 15.09.2005, Freiburg im Breisgau

Measures of Completeness of Follow-up Data, exemplified by an Individual Patient Data Meta-analysis of Survival and Second Malignancies in Hodgkin’s Lymphoma

Meeting Abstract

Search Medline for

  • Jeremy Franklin - Universitaet Koeln, Koeln
  • Annette Pluetschow - Universitaet Koeln, Koeln

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. Deutsche Arbeitsgemeinschaft für Epidemiologie. 50. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (gmds), 12. Jahrestagung der Deutschen Arbeitsgemeinschaft für Epidemiologie. Freiburg im Breisgau, 12.-15.09.2005. Düsseldorf, Köln: German Medical Science; 2005. Doc05gmds337

The electronic version of this article is the complete one and can be found online at:

Published: September 8, 2005

© 2005 Franklin et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.



Introduction and Aims

Incomplete or inaccurate follow-up of patients can lead to bias in meta-analyses of long-term outcomes such as survival or late toxicities [1]. We propose measures of completeness of follow-up, specifically for a systematic review comparing survival and second malignancy risk after use of various treatment modalities for Hodgkin’s lymphoma (HL) [2]. We also consider how the effect of incomplete follow-up on meta-analysis results may be assessed.

Materials and Methods

We sought to quantify the following aspects of the set of individual patient data (IPD) of censored time-to-event nature contributing to a meta-analysis:
(1) Length of follow-up: by median follow-up length (MFU)
(2) Information lag: by (a) median time interval between date of last information (DLI) and latest date of collection of trial data (MIL); or (b) interquartile range in the DLI (IQR-DLI).
(3) Follow-up imbalance: by p-value from the log-rank test of differences in distribution of last information date between treatment arms.

Medians and quartiles were calculated using Kaplan-Meier techniques to allow for censoring.

These measures were computed for all trials contributing to the HL meta-analysis, which included IPD from 37 trials enrolling between 24 and 1136 HL patients between 1966 and 2000. The meta-analysis involved four randomised comparisons between treatment modalities.

We then proposed and tested a method for restricting the data analysis to the more complete follow-up periods of each contributing trial. For this purpose, all follow-up times in a particular trial were censored at the date at which, in that trial, 75% of patients were still being followed up (i.e. the lower quartile of the DLI, termed 'cut-off date'). The results of treatment comparisons using this 'sensitivity' analysis were compared with those of the main analysis for two endpoints, namely progression-free survival (PFS) and second malignancy rate (SM).

These analyses were complemented by (a) a questionnaire completed by trialists on other aspects of trial procedures relevant to follow-up quality and (b) comparisons of observed SM rates with those expected on the basis of cancer registry data.


Of the 37 trials in the HL meta-analysis, 12 trials had a MFU between 4 and 9 years, 13 trials between 10 and 19 years and 12 trials between 20 and 32 years.

Since no date for latest collection of trial data was available to us, we substituted the latest DLI in the trial concerned in order to estimate median information lag (MIL). The MIL was less than 1 year in 14 trials, between 1 and 5 years in 16 trials and between 5 and 11 years in 7 trials. The IQR-DLI was less than 1 year in 11 trials, between 1 and 2 years in 6 trials, between 2 and 4 years in 10 trials and between 4 and 13 years in 10 trials. The IQR-DLI was less than 10% of the MFU in 12 trials, between 10% and 20% in 8 trials, and between 20% and 97% in 17 trials. Comparing the various trials, MIL and IQR_DLI were approximately linearly related with correlation coefficient 0.90. Both MIL and IQR-DLI tended to increase linearly with MFU up to a MFU of about 20 years; for trials with a MFU longer than 20 years MIL and IQR-DLI were lower (between 0 and 5 years). This may indicate increasing loss to follow-up as trials get older, but that the oldest trials where data are still available have employed effective update campaigns or used registry data to improve completeness.

The distribution of lengths of follow-up was also compared between treatment arms within trials. In one trial only, a significant difference was detected. Bearing in mind the multiple comparisons involved, therefore, no evidence of preferential follow-up in particular treatment arms was found.

Censoring follow-up at the cut-off date led to a median 'loss' (among the 75% of observations which were thus censored) of between 0.0 and 9.0 years of follow-up for the various trials. Estimated 'Peto' odds ratios and p-values for treatment comparisons for the endpoint PFS were only negligibly affected by censoring at the cut-off date. For the endpoint SM, minor differences between the two analyses were obtained for all treatment comparisons.


The MFU is a well-established measure of length of follow-up. The MIL is introduced here to quantify the amount of 'missing' follow-up and thus indicates a potential for bias. A disadvantage of this measure is the need to define the latest date of data collection in the trial concerned. Substitution of the latest DLI is plausible, but makes the result dependent on a single observation. The use of the IQR of the DLI avoids this problem, and quantifies the 'scatter' in the extent of follow-up and thus the potential for bias. Both MIL and IQR must be considered in relation to the MFU.

The use of censoring at the cut-off date to remove less completely observed time periods is a possible method for reducing the potential for bias. The effect of such cut-off depends upon the number of events observed and their distribution over time: analyses with few events or with mainly late events (as with SM in our example) will tend to be most strongly affected by cut-off.

In conclusion, measures of completeness of follow-up are needed for assessment of bias potential in the meta-analysis of time-to-event data, and some feasible measures are proposed and illustrated here. We also propose a technique to improve completeness or to assess the influence of incompleteness. Further aspects of the quality of follow-up data may be investigated using a trialists' questionnaire.


Our gratitude is due to all trialists who contributed data to the HL meta-analysis and to the Deutsche Forschungsgemeinschaft for supporting the meta-analysis financially.


Clark TG, Altman DG, De Stavola BL. Quantification of the completeness of follow-up. Lancet 2002; 359: 1309-10
Franklin JG, Paus M, Pluetschow A, Specht L. Secondary malignancy risk following treatment for Hodgkin’s disease: A meta-analysis of the randomised trials. Eur J Haematol 2004; 73 (suppl. 65): 21 (abstr. L06)