gms | German Medical Science

GMS Medizinische Informatik, Biometrie und Epidemiologie

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e.V. (GMDS)

ISSN 1860-9171

Passive versus active follow-up to investigate the efficacy of primary prevention programs

Passives versus aktives Follow-up zur Beurteilung der Wirksamkeit primärer Präventionsprogramme

Original Article

Search Medline for

  • corresponding author Martina Kron - Department of Biometry and Medical Documentation, University of Ulm, Ulm, Germany
  • Wilhelm Gaus - Department of Biometry and Medical Documentation, University of Ulm, Ulm, Germany
  • Josef Högel - Department of Biometry and Medical Documentation, University of Ulm, Ulm, Germany

GMS Med Inform Biom Epidemiol 2005;1(1):Doc02

The electronic version of this article is the complete one and can be found online at: http://www.egms.de/en/journals/mibe/2005-1/mibe000002.shtml

Published: April 7, 2005

© 2005 Kron et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en). You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.


Abstract

Before general application of a primary prevention program its efficacy has to be demonstrated. For this purpose a randomized controlled trial with active or passive follow-up may be conducted. In the last 5 years, the ratio of controlled trials with passive versus those with active follow-up was 1:13. However, under certain circumstances a passive follow-up may be more appropriate and useful to overcome the drawbacks of an active follow-up, as e.g. high costs and many drop-outs. In a randomized controlled trial, a passive follow-up is based on the reporting of cases by physicians or hospitals instead of actively following up all study participants individually. The statistical evaluation can be carried out using a one-sample chi2-test. Advantages and limitations are discussed. A passive follow-up may be advantageous in situations with low incidence, large number of participants, complete ascertainment of conditions with obligatory notification or effective disease registries and should be preferred in such a context.

Keywords: active follow-up, passive follow-up, study design, randomized controlled trial, primary prevention program

Zusammenfassung

Vor dem Einsatz primärer Präventionsprogramme auf Bevölkerungsebene ist deren Wirksamkeit zu prüfen. Standardmäßig werden dazu randomisierte kontrollierte Studien mit aktivem oder passivem Follow-up durchgeführt. In den letzten 5 Jahren kamen auf 1 Studie mit passivem Follow-up 13 Studien mit aktivem Follow-up. Unter bestimmten Voraussetzungen kann ein passives Follow-up jedoch geeigneter und sinnvoll sein und dazu dienen die Nachteile eines aktiven Follow-ups, wie z.B. hohe Kosten und viele Drop-outs, zu vermeiden. In einer randomisierten kontrollierten Studie mit passivem Follow-up werden alle Patienten mit der interessierenden Erkrankung von den behandelnden Ärzten oder den Krankenhäusern der Region direkt an das Studiensekretariat gemeldet. Die statistische Auswertung kann mit einem 1-Stichproben-Chi2-Test erfolgen. Die Vor- und Nachteile dieses Vorgehens werden diskutiert. Ein passives Follow-up kann in Situationen mit niedriger Inzidenz, großer Anzahl von Studienteilnehmern, Gewährleistung einer vollständigen Erhebung der Krankheitsfälle durch Betrachtung einer meldepflichtigen Erkrankung und gut geführte Krankheitsregister vorteilhaft sein und sollte dann durchaus bevorzugt werden.

Schlüsselwörter: aktives Follow-up, passives Follow-up, Studiendesign, randomiserte kontrollierte Studie, primäres Präventionsprogramm


Introduction

For medical therapies a proof of efficacy is mandatory but also for primary prevention programs, e.g. vaccination against a serious disease, efficacy must be demonstrated before general application. The conventional design for studies to prove efficacy is a randomized controlled trial (RCT) with two parallel groups: one group receives the primary prevention and the other group does not receive this intervention. All participants of the study will be followed up individually, i.e. by active follow-up, and at the end of the observation period the incidence rates in both study groups will be compared, e.g. using a two-sample chi2-test [1]. However, an active follow-up of all study participants often poses problems particularly if a large sample is involved over a long time [6]. The consequences are high expenses and a lot of drop-outs due to e.g. non-compliance or change of residence. Therefore - in spite of high costs - the power of the study might be low.

Alternatively, a passive follow-up may be used to overcome these drawbacks, as e.g. in a large randomized controlled trial on vaccination against polio in 1954 [5]. However, in the last 5 years, the ratio of controlled trials with passive follow-up versus those with active follow-up was 1:13 (numbers are based on a search in PubMed). In the following, types of passive follow-up will be described. An example for the purpose of comparing active and passive follow-up is given. Assumptions, advantages, and limitations of both types of follow-up will be discussed afterwards.


Passive follow-up

Suppose the efficacy of a primary prevention program is investigated with a RCT with two parallel groups (one intervention group and one non-intervention group). This study can be double-masked, i.e. neither the study participants nor the physicians know who received true intervention and who sham-intervention (placebo). Moreover, the comparison of two (or more) active treatments (e.g. new treatment versus standard treatment) is also possible. In a RCT with passive follow-up no individual follow-up of study participants is carried out. Instead of an individual follow-up, all "cases" are generally determined and recorded by their physician or any hospital of a pre-defined area. Here, "cases" are patients suffering from the disease which should be prevented by the investigated intervention.

With starting the study, physicians and hospital staffs are informed about the study by circular letters, study meetings, and public media. Reporting of cases to the study secretariat might be done by a surveillance or monitoring system of hospital admissions or discharges. One scenario is that eligible cases are selected from the hospital discharge listings. Their diagnoses are classified by well trained nosologists and can serve as the basis for reviewing selected medical charts from hospital records in order to arrive at a standardized ascertainment of cases. Another possibility would be a physician-initiated notification. Thereby, during the study, physicians and hospitals report all cases to the study secretariat. Additionally, the study secretariat inquires about cases e.g. every 3 months if no cases have been reported.

Study participants among reported cases can be identified by retrieving from the study file whether the case participated in the study and if so, whether the case received true intervention or non-intervention (placebo, control). Then the number of diseased and recorded study participants in the two study groups can be compared.

The data can be evaluated by calculation of incidence rates in the study groups, simply dividing the number of diseased study participants in each group by the number of participants randomized. However, this incidence rate may be prone to bias since some participants might not have been recorded as cases during the study period [3]. Therefore, the true incidence rate might be underestimated. If the drop-out rate is the same in both groups, underestimation of the incidence will occur in both groups to a nearly similar extent. A sensitivity analysis for different drop-out rates may be done to get an impression of the consequences. In case the drop-out rate can be estimated from other studies, this can be used to correct for that bias. Moreover, incidence differences and incidence ratios can be calculated. For interpretation of the incidence ratio, the drop-out rate is irrelevant as long as there is non-differential case ascertainment by treatment group, otherwise the ratio estimate is biased. Confidence intervals for incidence differences or incidence ratios can be calculated by use of standard statistical methods [4].

To demonstrate the efficacy of the intervention statistically, a one-sample chi2-test (see e.g. [1]) may be applied instead of a two-sample chi2-test in order to avoid estimation of the number of participants receiving the intervention and not dropping out. For a one-sample chi2-test the proportion of cases in the intervention group among all diseased study participants will be calculated. Under the null-hypothesis of no intervention effect or even a worsening under intervention this proportion is greater than or equal to 50% of all cases under the assumption of a 1:1 randomisation. Therefore, if it can be shown by application of a one-sample chi2-test that the proportion of cases of the intervention group among all diseased study participants is less than 50%, the efficacy of the primary prevention program is demonstrated.


Example

In the following fictitious example assumptions for planning a study are presented; calculated sample sizes and potential results for a RCT with active follow-up and a RCT with passive follow-up are given and compared.

Assume that in a population with a 4-year cumulative incidence of tuberculosis of 8% a primary prevention program (e.g. a vaccination or a short term training program in hygiene) is to be tested. For planning purposes it is presumed that the cumulative incidence of the disease can be reduced from 8% to 4% by the intervention.

For planning a RCT with active follow-up a one-sided significance level of 5% and a power of 90% is chosen. Using a two-sample chi2-test, these assumptions lead to a sample size of 602 participants per group. After motivating the population for participation in the study, altogether 1204 participants, who give informed consent, are recruited. They are randomized to the true intervention or the sham-intervention group (602 participants in each group). The treatment is given in a double-masked manner. After 4 years of individual follow-up the data are collected and might look such as presented in table 1 [Tab. 1].

According to the data shown in table 1 [Tab. 1] there is a 4 year cumulative incidence of disease of 7.3% (41 of 562) in the control group and of 4.8% (27 of 558) in the intervention group. The incidence difference is 2.5 points of percentage and the incidence ratio is 1.51. Testing with a two-sample chi2-test whether the incidence in the intervention group is smaller than that in the control group leads to a one-sided p value of 0.043. Therefore, the efficacy of the primary prevention program could be demonstrated by the use of a RCT with active follow-up.

Assume that the same 1204 participants were treated and surveyed in a study using passive follow-up. This study may deliver results as presented in table 2 [Tab. 2]. Table 2 [Tab. 2] includes some more cases in the study population that are detected since the disease under study is notifiable and therefore, the register of the local health authority may be used to glean all cases.

The fictitious data in table 2 [Tab. 2] show that in 4 years altogether 154 cases were determined. Of these, 86 cases do not belong to the study population, 41 cases belong to the non-intervention group and 27 to the intervention group. Incidences in control and intervention group can be estimated as 6.8% (41 of 602) and 4.5% (27 of 602). Therefore, the incidence difference is 2.3 points of percentage and the incidence ratio is 1.52. The proportion of cases in the intervention group among cases in the study is 39.7% (27 of together 68 cases in both groups). The application of a one-sample chi2-test to demonstrate that the proportion of cases in the intervention group among all cases in the study is smaller than 50% leads to a one-sided p value of 0.045. Thus, the efficacy of the intervention can also be proven evaluating the data from the RCT with passive follow-up, supposing that the same number of cases as in the trial with active follow-up is detected.

A sample size calculation could have been done for a one-sample chi2-test in the RCT with passive follow-up using the assumptions above. These assumptions lead to a ratio of 4:8 cases (4 cases in the intervention and 8 cases in the control group). Consequently, the proportion of cases in the intervention group among all study cases is 4/12. Assuming a one-sided significance level of 5% and a power of 90% for a one-sample chi2-test results in a sample size of 73 diseased participants. To have 73 cases in the sample, 609 participants per group have to be recruited if a 1:1 randomisation is performed and a proportion of registered cases is 4% under intervention and 8% under non-intervention. Hence, under the assumption that all cases are registered, in the RCT with passive follow-up only a few more participants had to be recruited per group to achieve the same power as in the RCT with active follow-up.


Discussion

For the evaluation of the efficacy of a primary prevention program it generally is important that medical care in the study area is sufficient. In both designs, this ensures that (nearly) all cases will consult or be sent to physicians and hospitals of that area. Passive follow-up may have considerable benefits under certain conditions like obligatory notification, effective disease registries, geographic isolates, institutionalized populations. If the disease under study is notifiable, this ensures that almost all cases are registered. In both designs the required number of study participants mainly depends on the incidence of the disease in the study groups, the level of significance and the power of the statistical test.

For both designs it is best to regionally concentrate the study activities and motivate people throughout the whole selected region to participate in the study. However, there will always be some inhabitants who refuse to participate in the study. Consequently, by passive follow-up probably some cases will become known who are not study participants. These cases will add no power to the demonstration of efficacy but may be evaluated to check the representativeness of the study participants.

An assumption which must be fulfilled when applying a passive follow-up is that case determination rates are the same for the intervention and the control group. This is true if the intervention is masked and/or the disease which should be prevented is serious enough to require hospital treatment or at least ambulatory care. If, on the other hand, the intervention is unmasked and the disease under study is less serious, case determination rates under intervention and non-intervention may differ because primary prevention measures may also affect patterns to seek medical care independent of disease status. If participants of the intervention group seek medical care more often, this may result in a detected incidence in the intervention group greater than that expected. Therefore, a bias occurs which may result in the judgement that the primary prevention program is non-efficacious although it is. If the contrary is true, the difference in the estimated (observed) incidence between the intervention and the control group is larger than it is in reality, i.e. the primary prevention program is said to be efficacious although it is not.

There are several requirements for a passive follow-up. The study endpoint must be symptomatic so that the disease leads to seeking medical care. For ascertainment of coronary heart disease, passive follow-up would not necessarily be appropriate due to the high proportion of silent myocardial infarctions. It is necessary that cases always come to the attention of a medical professional and that there exist no barriers to seeking medical care. Further, the medical professional must always record the correct diagnosis in the proper medical record and always notify the study agency of selected diagnoses. Quality assurance procedures to protect completeness of endpoint ascertainment and of quality of data should be built into the process of care seeking and of medical care.

A pre-requisite for conducting a passive follow-up is either the existence of a comprehensive disease registry (as e.g. for notifiable diseases) or the active co-operation of a large group of physicians or hospitals involved in the care of study participants. This group of physicians will not always be easily defined. Moreover, it is difficult to motivate physicians, personnel and health-care workers to contribute data to a project they do not consider "theirs", particularly if there is no financial incentive. If e.g. general practitioners are requested to report on cases in their practices, this might be very laborious. Physician-initiated notification might also be selective and lead to undercounting. Under these circumstances, case determination might sometimes be worse in the design with passive follow-up than in the design with active, i.e. individual, follow-up. However, if e.g. a notifiable disease is under investigation it might be the other way round, i.e. the drop-out rate in the design with individual follow-up is high whereas recording of cases in the design with passive follow-up is (nearly) complete.

Other assumptions for the passive follow-up design are that the compliance rate (adherence to randomized treatment) and the drop-out rate in both study groups is nearly the same and that there is non-differential case ascertainment by treatment group. Obviously, the numbers of drop-outs due to change of residence or death are likely to be similar in both study groups since participants are randomly allocated. Thereby, this is true for both, cases and not diseased participants. If important risk factors for the disease are known, stratified randomisation is recommended to obtain homogeneous subgroups for comparison. If additionally the intervention can be masked, compliance and case ascertainment rates are similar in both groups. If the intervention cannot be masked, this can lead to different compliance and case ascertainment rates in the group of intervention and non-intervention. In case that more participants in the intervention group than in the other group do not adhere to the treatment randomized, the number of diseased in this group will rise. If many participants in the non-intervention group take particular care of their health or apply the intervention due to own motivation, the number of diseased in this group will decrease. In both situations, the incidence ratio of the study groups might be closer to one than expected, i.e. the study has less power to demonstrate efficacy, although the intervention is efficacious. If, on the other hand, two active treatments are compared and participants applying the new treatment are more motivated than participants receiving standard treatment. Application of a statistical test may result in a type one error since the observed incidence ratio might be more extreme than it should be.

For differential case ascertainment by treatment group two situations might be distinguished. If participants of the intervention group are more sensitive and pay more attention to symptoms of the disease, than the number of cases detected in this group will rise and the incidence ratio is biased towards one. However, if this is true for participants of the non-intervention group, the incidence ratio is biased away from one and this might lead to a type one error because of overestimating the treatment effect. Beside masking, motivating participants throughout the study to adhere to the treatment assigned might help to ensure that compliance rates are similar and effect estimates are prone to less bias. Of course, differential compliance rates by treatment group constitute a problem for both active and passive follow-up. However, control of compliance is much easier in case of active follow-up. Moreover, participants and physicians should be motivated for case ascertainment in an equally extensive way. Thereby, the more serious the disease the easier to achieve non-differential case ascertainment.

Both designs are open to selection bias in a different way and magnitude. Factors or events having potential for bias differently affecting the two designs are for example:

• participants living in the catchment area but lost to follow-up only affect the active follow-up design,

• participants moving away from the catchment area strongly affect the passive follow-up design,

• treatment associated deaths may strongly affect the passive follow-up design (but registers of deaths may be used to retrieve this information),

• participants hospitalized for a competing cause may strongly affect the passive follow-up design (but will be recognized if discharge records are accessed to get information about participants),

• treatment has side effects with impact on compliance affects both designs equally,

• treatment has side effects with impact on seeking medical care strongly affects the passive follow-up design.

The explicite advantages of a passive follow-up may be the following. It can lead to cost savings because individual follow-up is avoided and no inquiry of healthy people has to be carried out. This especially is true, if the incidence is low. Because then, many healthy study participants have to be followed up in the active follow-up design. A better practicability of the passive follow-up design may result from a less extensive study documentation and a reliable documentation of hospital cases. Moreover, a rather complete reporting of cases seems to be more feasible than achieving a small drop-out rate in active follow-up because especially for a serious disease all cases are likely to be registered at a hospital. Compliance of study participants concerning contacts with the study team during the active follow-up is replaced by the reporting carried out by medical staff. It seems that costs of motivation, education and follow-up are shifted from the study participant to the medical professionals. But that is not necessarily true since information is retrieved for the study which is recorded routinely by physicians. Quality assurance programs can be instituted to reduce the magnitude of undercounting and to estimate the magnitude of the under-reporting, so the number of cases discovered by hospital documentation will not be smaller than the number of cases detected by information from an individual follow-up. Quality assurance programs might be expensive but also other inexpensive ways are possible, e.g. a surveillance or monitoring system of hospital admissions or preferably discharges. Although surveillance of hospital discharge diagnoses would have considerable advantages of cost, it would obviously be restricted to conditions that require hospitalization and would still be subject to selection bias due to barriers to care and referral to medical care and/or hospitalization. However, probably the most appropriate way is to select all eligible cases from hospital discharge listings whereby diagnoses are classified by well trained nosologists. Afterwards selected medical charts can be used to arrive at a standardized ascertainment of cases.

A statistical test for the design with individual follow-up compares two independent groups, while the design with passive follow-up requires a one-sample test. The two-sample chi2-test always is more powerful than the one-sample chi2-test. However, for low incidence differences in power are small. A comparison of the power of one-sided chi2-tests (one sample vs. two samples) for incidence rates between 4% to 16% is given in table 3 [Tab. 3] for levels of significance of 5% and 1%. There it is assumed that all data are complete and reports on all participants are available, i.e. the reported number of cases is the same in both designs. As a result, no considerable loss of power has to be concluded for the design with passive follow-up. Assume that the same amount of money is invested in study activities under both designs. Then, more participants can be recruited for studies with passive follow-up. For the situations with active follow-up considered in table 3 [Tab. 3], sample sizes needed in order to achieve the same power for passive follow-up were calculated. Furthermore, we calculated how many times higher the sample size under passive follow-up with respect to situations considered in table 3 [Tab. 3] should at least be. From this it follows that, if active follow-up per participant is at least 1.2 times more expensive than passive follow-up, studies with passive follow-up will be more powerful because more participants can be recruited when investing the same amount of money. Assume e.g. the situation with hospital discharge records available in electronic format. Information retrieval in such a dat base will only take a few minutes each time it is done. In total, only a couple of hours have to be investigated, even if conducted regularly, e.g. each quarter of a year. On the contrary, an active follow-up consisting of letters and, in case of non-response, of telephone calls would be much more time consuming and presumably, less complete. Thus, if active follow-up is expensive, difficult or long, there will be a gain in power because case reporting will be more complete than active follow-up of individual participants and more participants can be enrolled without spending more money. Sample sizes for designs with passive follow-up depending on the incidence in the control group and the pre-specified relevant reduction of this incidence by the intervention are given in table 4 [Tab. 4]. Results support the conclusion that savings in costs can be high, especially if the incidence is low.

The advantages of the passive follow-up design over the design with active follow-up may predominate in some situations and a summarizing overview of advantageous and limiting situations described above in detail is given in table 5 [Tab. 5]. Passive follow-up may have considerable benefits under certain conditions, such as low incidence, large number of participants, complete ascertainment of conditions with obligatory notification, effective disease registers, geographic isolates, institutionalized populations, as e.g. residents of long-term care units or employees of a company. Therefore, a passive follow-up would seem to be a design feature that would also be of interest in group-randomized trials [2].


References

1.
Altman DG. Practical Statistics for Medical Research. London: Chapman & Hall; 1991. p. 230-252.
2.
Donner A, Klar N. Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold; 2000.
3.
Fleming TR. Evaluating the safety of interventions for preventions of perinatal transmission of HIV. Ann NY Acad Sci. 2000;918:201-211.
4.
Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologic research. New York: Van Nostrand Reinhold Company; 1982. p. 296-300.
5.
Meier P. Polio trial: an early efficient clinical trial. Stat Med. 1990;9:13-16.
6.
Tuberculosis Research Centre (ICMR). Fifteen year follow-up of trial of BCG vaccines in south India for tuberculosis prevention. Indian J Med Res. 1999;110: 56-69.