gms | German Medical Science

EbM zwischen Best Practice und inflationärem Gebrauch
16. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin

Deutsches Netzwerk Evidenzbasierte Medizin e. V.

13.03. - 14.03.2015, Berlin

Evidence for selection bias in influenza vaccine effectiveness studies: systematic review

Meeting Abstract

Search Medline for

EbM zwischen Best Practice und inflationärem Gebrauch. 16. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin. Berlin, 13.-14.03.2015. Düsseldorf: German Medical Science GMS Publishing House; 2015. Doc15ebmP9f

doi: 10.3205/15ebm112, urn:nbn:de:0183-15ebm1127

Published: March 3, 2015

© 2015 Harder et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License. You are free: to Share - to copy, distribute and transmit the work, provided the original author and source are credited. See license information at



Background and Study Question: Due to the lack of randomized controlled trials, observational studies on influenza vaccine effectiveness (VE) are frequently used to inform national immunization policies. However, they are prone to two forms of selection bias, namely confounding by indication and healthy vaccinee bias. If not addressed properly, both forms of bias might strongly distort the evidence base, which might have adverse consequences for the decision-making process. We performed a systematic review to assess the occurrence and consequences of selection bias in influenza VE studies.

Methods: Systematic review of observational studies reporting influenza VE and indicators for selection bias. We assessed risk of selection bias for each study and calculated ratios of odds ratios (crude/adjusted) to quantify the effect of confounder adjustment. VE estimates obtained during and outside influenza seasons were compared to assess residual confounding by healthy vaccinee effects.

Results: We identified 22 studies reporting on ten clinical outcomes. Eighteen studies (82%) showed high risk of selection bias (n=13 confounding by indication, n=2 healthy vaccinee bias, n=3 both forms). Adjustment for confounders increased VE against all-cause mortality, hospitalization and influenza-like illness by 12% (95%CI: 7-17%), 9% (95%CI: 4-14%) and 7% (95%CI: 4-10%), respectively. Despite adjustment nine studies showed residual confounding as indicated by significant off-season estimates. These were observed for five different outcomes (all-cause mortality, major adverse vascular events, hospitalization due to influenza/pneumonia, acute coronary syndrome, influenza-like illness) but were more likely regarding all-cause mortality compared to other outcomes (p=0.03), and more often occurred in studies which indicated healthy vaccinee bias at baseline (p=0.01).

Conclusions: Both forms of selection bias are likely to operate simultaneously in observational studies on influenza VE. Although adjustment can correct for confounding by indication to some extent, the resulting estimates are still prone to healthy vaccinee bias, at least as long as unspecific outcomes like all-cause mortality are used. Conventional study designs are unlikely to provide unbiased estimates of influenza vaccine effectiveness on the population level. Consequently, other study designs are needed to address this important public health issue.