Forecasting COVID-19 cases and deaths in Europe - new hub will support European pandemic planning

This is the catalyst for the new European COVID-19 Forecast Hub, an initiative of the European Centre for Disease Prevention and Control (ECDC), with the support of the Centre for Mathematical Modelling of Infectious Diseases (CMMID) at the London School of Hygiene & Tropical Medicine.

Based on similar projects in the US, Germany and Poland, the hub collates weekly short-term forecasts from modelling teams around the world, predicting the number of COVID-19 cases and deaths four weeks ahead in 32 countries: 30 EU/EEA Member States, the UK and Switzerland. These are combined into an ‘ensemble’ and displayed alongside the individual forecasts.

“We are excited about European modelling teams collaborating together with others from around the world to develop accurate and timely predictions of COVID-19 cases and deaths in Europe. We clearly foresee the potential impact on public health policy. This initiative yields insight now and also strengthens our capacity to respond to infectious disease threats in the future” says Helen Johnson, Mathematical Modeller, ECDC.

During this pandemic, many forecasts have been produced, some very accurate, others less so. Scientists and their models need to remain responsive and flexible, especially given the increasing complexity of the pandemic, including changes in behaviour, “pandemic fatigue”, vaccine roll-out and newly-emerging variants. Some models account for these evolving factors less than others, leading to potentially biased forecasts. Collating forecasts into a single ‘ensemble’ forecast makes it possible to counteract the biases from different models, providing more accurate and reliable forecasts than a single model. This approach has been used successfully during outbreaks of other diseases, including Ebola, dengue, and influenza.

According to Sebastian Funk, Professor of Infectious Disease Dynamics, London School of Hygiene & Tropical Medicine, UK: "By constructing an ‘ensemble’, we can leverage the contribution from each of the modelling teams, with the aim of creating the best possible forecast.”

As Dr. José L Aznarte, Associate Professor of Artificial Intelligence, UNED Madrid, Spain, puts it: "[...] the weekly aggregation of all our predictions into an ensemble model which consistently outperforms every single one, is the perfect example of scientific cooperation for social good.”

Every week on Mondays, the teams submit their forecasts of cases and deaths, predicting up to four weeks ahead. The hub's website displays current and past forecasts from each of the models, the ensemble forecast, and various evaluations of the submissions. The latest round of submissions, collated on 19 April 2021, contains 33 models representing a wide range of modelling methodologies.

According to Dr. Johannes Bracher, postdoctoral researcher in Statistics, Karlsruhe Institute of Technology, Germany: "Our experience from running a German and Polish COVID-19 Forecast Hub has been that this format provides a very useful feedback loop for modellers. It creates a track record of what models said at different points in time, allowing for apples-to-apples comparisons, systematic evaluation and, ultimately, improvement of models."

This is confirmed by Dr. Tyll Krüger, Professor at Wrocław University of Science and Technology, Poland: " I consider the European Forecast Hub to be an extremely valuable tool for policy makers, epidemiologists, public health professionals, modellers and the public. The Forecast Hub is the ideal platform to compare and ultimately improve forecasts from many different groups in a scientific and transparent way. It also provides a clear representation of scientists' certainties and uncertainties about the epidemic dynamics in the near future. The ‘ensemble’ forecast, which is composed of many individual forecasts, should receive broad public coverage in the media of every European country."

However, the European Forecast Hub provides more than forecasts. It creates a platform for the participating modellers to meet colleagues from across Europe and elsewhere. A weekly online meeting is held, during which the teams share ideas and thoughts, representing a collaborative project that will be of benefit for years to come. During these meet-ups, the most recent forecasts are discussed and teams have the opportunity to present and discuss their forecast models or related work. There are currently 28 teams participating, representing eight different countries: Poland, Spain, Germany, Czechia, Italy, UK, Australia, and the USA. The Hub is also looking forward to welcoming further teams in the future.

“I very much enjoy attending the weekly on-line meetings, which both myself and my PhD student, Barbara, have always participated in. [...] We also very much appreciate the spirit of transparent co-operation among the different working groups, which I find very stimulating” says Professor Paolo Giudici, Professor of Statistics at the University of Pavia, Italy.

As Dr. José L Aznarte explains: “The variety of approaches pursued by the different international teams represents a huge opportunity to learn from the expertise of other researchers, usually coming from different fields."

This project was enabled by ECDC, supported by the Centre for Mathematical Modelling of Infectious Diseases (CMMID) at the London School of Hygiene & Tropical Medicine, and supported through collaboration with the University of Massachusetts-Amherst, the Karlsruhe Institute of Technology, and the Robert-Koch Institute.

Full length quotes

“I very much enjoy attending the weekly on-line meetings, which both myself and my PhD student Barbara have always participated in. Our research benefits a lot from the presentations and the following discussions, both in terms of new modelling knowledge and comparison of the findings of our models with others. We also very much appreciate the spirit of a transparent co-operation among the different working groups which I find very stimulating.” Prof. Paolo Giudici, Professor of Statistics, University of Pavia

“The online meetings of the Forecasting Hub are a perfect way of sharing advances and experiences in predictive modelling of COVID-19. The variety of approaches pursued by the different international teams represents a huge opportunity to learn from the expertise of other researchers, usually coming from different fields. Furthermore, the weekly aggregation of all our predictions into an ‘ensemble’ model which consistently outperforms every single one, is the perfect example of scientific cooperation for social good.” Dr. José L Aznarte, Associate Professor of Artificial Intelligence, UNED (Madrid)

"Our experience from running a German and Polish COVID-19 Forecast Hub has been that this format provides a very useful feedback loop for modellers. It creates a track record of what models said at different points in time, allowing for apples-to-apples comparisons, systematic evaluation and, ultimately, improvement of models. On the accompanying calls there has been a lot of constructive discussion about why predictions differed and which aspects were decisive at a given time. We are happy to contribute to this new European effort to foster this kind of exchange." Dr. Johannes Bracher, postdoctoral researcher in Statistics, Karlsruhe Institute of Technology, Germany

“There are many good reasons for decision-makers to consider the outputs from multiple models constructed independently. By constructing an ‘ensemble’, we can leverage the contribution from each of the modelling teams with the aim of creating the best possible forecast.” Sebastian Funk, Professor of Infectious Disease Dynamics, London School of Hygiene & Tropical Medicine

"I consider the European Forecast Hub to be an extremely valuable tool for policy makers, epidemiologists, public health professionals, modellers and the public. The Forecast Hub is the ideal platform to compare and ultimately improve forecasts from many different groups in a scientific and transparent way. It also provides a clear representation of scientists' certainties and uncertainties about the epidemic dynamics in the near future. The ensemble forecast, which is composed of many individual forecasts, should receive broad public coverage in the media of every European country." Dr. Tyll Krüger, Professor at Wrocław University of Science and Technology