Systematic review and meta-analysis concerning the efficacy and effectiveness of seasonal influenza vaccines – (part 1)Archived

Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysisOsterholm MT, Kelley NS, Sommer A, et al.The Lancet Infectious Diseases, Early Online Publication, 26 October 2011Accompanying commentary by Kelly and Valenciano

This is an authoritative independent evidence-based review and meta-analysis of the efficacy and effectiveness of influenza vaccines. It confines itself to trials and observational studies where diagnostic tests confirmed influenza infection as the end point.  Moreover it differs from some previous reviews of influenza vaccine performance in that the authors have used particularly restrictive study inclusion criteria to minimise bias and confounding and excluded randomised controlled trials (referred to below as trials) in which the comparison group did not receive either placebo or a vaccine other than influenza.  In this summary ECDC will mostly address the issues arising from use of trivalent seasonal vaccines though the review itself also looks at the special circumstances of the 2009 pandemic and monovalent vaccines.

After applying these stringent criteria the review noted important gaps in the evidence base for some age groups with regard to efficacy data for trivalent injected inactivated seasonal influenza vaccines, (injected vaccines) and live attenuated seasonal influenza vaccine given nasally (live vaccines). The latter live vaccines are generally not available in Europe.

In the methodology the authors searched Medline for trials assessing a relative reduction in influenza risk of all circulating influenza viruses during individual seasons in controlled trials (efficacy studies) and in the field with carefully performed observational studies meeting inclusion criteria (effectiveness studies). The articles eligible for inclusion were published between the 1st Jan 1967 and the 15th Feb 2011 and used RT-PCR or culture for confirmation of influenza. The authors estimated random-effects pooled estimated of effectiveness for seasonal injected vaccines and live vaccines when data were available for statistical analysis. In total, 5707 articles were screened and 31 of them were identified as eligible studies (17 randomised controlled trials and 14 observational studies). Ten trials assessed the efficacy of injected seasonal vaccines. Another ten trials assessed that of live seasonal vaccines. The authors also noted how many influenza seasons were covered by the studies since with genetic drift in influenza and the consequent mis-match of the vaccine with circulating viruses it is possible for effectiveness of the same vaccine to vary from season to season and so individual season results may be misleading.

The results pertaining to efficacy (trials) and effectiveness (experience as observed in the field) are presented separately in the paper. The efficacy of injected seasonal vaccines was found in 8 of the 12 seasons under consideration analysed in 10 trials giving a pooled efficacy of 59% [95% Confidence Intervals of 51—67%] in younger adults (aged 18 to 65 years).  No such trials met this reviews demanding inclusion criteria for children aged 2 to 17 years or for older adults. The efficacy of live seasonal vaccines came from 9 of the 12 seasons analysed in ten trials giving a pooled efficacy of 83% [5% CI of 69—91%] in children aged 6 months to 7 years. No trials met the inclusion criteria for children aged 8 to 17 years.  In relation to vaccine effectiveness, this was variable for seasonal influenza. Nine studies showed significant protection against medically attended influenza in the outpatient or inpatient setting though because of season to season variation it was not possible to come up with a single estimate.  The authors conclude that influenza vaccines can provide moderate protection against virologically confirmed influenza, but such protection is greatly reduced or absent in some seasons. In addition, they note that the evidence for protection in adults aged 65 years or older is generally lacking but they emphasised that this does not mean that the vaccine does not show effectiveness in that age group.  Young children aged between 6 months and 7 years was the group in which live vaccine’s efficacy was consistently highest. The authors emphasised the need for new vaccines with improved effectiveness if reductions of influenza-related morbidity and mortality are to be achieved.

The review is accompanied by a commentary by Heath Kelly (from the Victorian Infectious Diseases Reference Laboratory in Australia) and Marta Valenciano (from the European organisation EpiConcept and the I-MOVE collaboration – supported by ECDC). Kelly and Valanciano highlighted that that the evaluation of influenza vaccines against non-specific outcomes, such as influenza-like illness, hospital admission due to pneumonia or all-cause mortality, always confuses the understanding of the true burden of influenza and the effect of influenza vaccines and that serological endpoints (used in regulatory studies) were also problematic. They suggested that now would be an appropriate time to use revised estimates of the most probable effectiveness of influenza vaccines to re-examine the effectiveness and cost-effectiveness of some policy options, using highly specific laboratory-confirmed outcomes to assess influenza burden.

ECDC Comment (27th October 2011):

Vaccine Efficacy and Effectiveness

Estimating, understanding and communicating about influenza vaccine effectiveness are never easy.(1-3)  Even the terminology can be difficult. In this review the terms vaccine efficacy and effectiveness are well defined and used conventionally for an intervention (4,5).That is:

Efficacy – an estimate of the likelihood of an intervention preventing an outcome under optimal circumstances – for example as determined in randomised controlled trials.

Effectiveness – the likelihood of an intervention preventing an outcome when it is applied in the field – often determined by well designed observational studies allowing for possible confounding factors.

In addition in regulation of influenza vaccines there is also serological efficacy. A confusing term concerning how each new vaccine is considered to meet regulatory requirements where the antigen has been changed because limited trials with the new vaccine results in a pre-determined serological response in humans.(6,7)  This is sometimes confused with meaning that the vaccines have high effectiveness when in fact as has been repeatedly documented by ECDC and the I-MOVE Project, as well as this review the effectiveness estimates for seasonal influenza vaccination are like those documented by Osterholm et al.(1-3, 8)  The way forward is to be realistic and not overly optimistic and include independent estimates of effectiveness in post-marketing surveillance of vaccines.(9) Individual European member states such as the Netherlands have already implicitly taken this approach in making public statements.(10)

There are also difficulties in the end points of studies of influenza vaccines, whether they are laboratory confirmed infections, hospitalisations or deaths. As indicated by the Commentary non-specific end-points like influenza like-illness are confusing and should be avoided.(2) It is sometimes stated that there should be  large randomised trials to resolve these uncertainties.(11) This is simply unfeasible as well as probably being unethical as they would mean offering placebos to persons who would benefit from a intervention of proven effect.  Also because of the season to season change in effectiveness, especially with what is a new seasonal influenza since 2009 (12), any single trial will not be generalisable to current and future experience.  Well-conducted trials contribute to this field but they have to be complemented by observational studies and the Osterholm et al review and the Dutch statement are to be commended in considering both.