Comparison of the 2021 COVID-19 roadmap projections against public health data in England

Comparison of the 2021 COVID-19 roadmap projections against public health data in England
Like

Background

On the 30th January 2020, the first two cases of COVID-19 were detected in the UK [1]. Despite efforts being made to contain the virus, by the 26th March 2020 the number of confirmed cases had risen to over 20,000 with 1,841 confirmed deaths [2], and due to increasing pressures on health care services the UK was put into its first lockdown, obligating all but essential workers to stay at home [3] .

Lockdown measures were highly effective in limiting the spread of the SARS-CoV-2 virus within England, reducing the considerable pressure being placed on the healthcare system and preventing hospitals from becoming overwhelmed Such non-pharmaceutical interventions (NPIs) remained the primary means of pandemic control across the remainder of 2020 and into 2021. However, whilst NPIs were effective at reducing transmission, they were also  extremely costly both economically and socially, with effects upon businesses, access to education for young people and impacts on mental health and well-being as a result of isolation. It was therefore important to use these measures in such a way that these adverse impacts were as low as possible, whilst at the same time minimising  the risk of health services being overwhelmed.

The Roadmap out of lockdown

At the beginning of 2021 England was facing a renewed threat from the newly emerged Alpha variant, and on the 6th January 2021 the decision was made to enter a third national lockdown. Concurrently however, COVID-19 vaccination rollout had begun for the first time (beginning in England on 8th December 2020), providing protection for the most vulnerable against severe disease and decreasing the level of susceptibility within the population as a whole. It was of crucial importance to understand the dynamics of an increasingly infectious disease within an increasingly protected population, to determine how this affected the balance between the health risks associated with COVID-19 disease, the economic damage of preventing the normal operation of many businesses, and the social damage of restricting personal interaction.

In order to support this understanding and inform government decision making, three main academic groups— from the University of Warwick (the model we concentrate on here), the London School of Hygiene and Tropical Medicine [4] and Imperial College London [5]— worked to produce epidemiological model projections within a series of six Roadmap documents, coordinated through SPI-M-O, the Scientific Pandemic Influenza Group on Modelling, Operational - a subgroup of the Scientific Advisory Group for Emergencies (SAGE). These long-term projections together with epidemiological, behavioural and public health insights were presented to government by the Chief Medical Officer and the Chief Scientific Advisor where they were translated into the steps of England’s Roadmap out of lockdown:

  •       Lockdown (4th January - 7th March 2021). Stay at home order: Shopping for basic necessities only; non-essential retail, hospitality and personal care services closed. Work from home where possible. Can leave home for exercise (with household, support bubble or one other person), to meet support bubble, or to seek medical care. Remote learning in schools except key workers and vulnerable. Clinically extremely vulnerable advised to shield.
  •       Step 1a (8th March 2021). Schools reopen, with twice-weekly testing of staff and pupils.
  •       Step 1b (29th March 2021). Meeting outdoors with 6 people or 2 households allowed. Outdoor sports facilities can re-open.
  •       Step 2 (12th April 2021). Non-essential retail, personal care and public buildings can re-open. Outdoor table service in hospitality venues.
  •       Step 3 (17th May 2021). Meeting outdoors with up to 30 people, and indoors with 6 people or 2 households allowed. Most businesses can reopen, including indoor hospitality.
  •       Step 4 (19th July 2021). Originally set for 21st June 2021, after this step all legal limits on social contact were removed.

Mathematical modelling reflections

While the model projections produced to support these decisions were made using our best understanding of the epidemiological dynamics at the time, it is important to make the distinction  that these are not forecasts— they are rather designed to illustrate “what if” scenarios— though it is still important that such projections remain broadly consistent with the realised epidemic if they are to be of use in public health planning. Model projections rely on robust estimates of epidemiological parameters (usually inferred by matching models to data), estimates of vaccine characteristics and assumptions about population behaviour in terms of testing, isolation, social mixing and vaccine uptake - errors in any one of these can affect the projected rate of exponential growth resulting in large deviations in the predicted waves of infection and disease.

With the benefit of hindsight, we are now in the position where we can look back on the projections made in each Roadmap document and compare the projections to the corresponding data, to assess whether they were sufficiently robust to help inform policy. This study focuses on the outputs of the Warwick model, and provides strong evidence that, contrary to the beliefs of critics, in general there is a high level of agreement between the model projections and the subsequent trajectory of the epidemic. This level of agreement generally  increases across the six roadmap documents, driven by advances in our understanding and corresponding model complexity. In general we believe the models acted as a useful policy tool; aiming to provide scientific evidence before each step in the relaxation of controls [6], suggesting when the steps would lead to a minor increase in cases or when greater caution may be needed, in our view the six Roadmap documents fulfilled their purpose.

Caricatures of the development of Warwick SARS-CoV-2 transmission and COVID-19 disease model.

Lessons for the future

The work highlights two clear lessons for the future:

First, the difficulties of making detailed long-term predictions due to uncertainties in both vaccine and disease parameters (especially in the early stages of vaccine roll-out or a novel variant), as well as in human behaviour.  Estimates of the effects of vaccination and the level of protection it infers against severe disease effects, infection and transmission became progressively more confident over time, as data accumulated. We also gained an increasing understanding of social behaviour; how precautionary behaviour increases rapidly in response to imposed restrictions or perceived risk, and the gradual decline in social caution at other times, as individuals seek a return to normality during periods of relative stability. Some significant behavioural anomalies would always remain impossible to predict however, such as the dramatic increase in mixing in late June/July 2021 due to the UEFA European Football Championship, and the later increased isolation caused by the ‘pingdemic’.

Second, the difficulties of communicating projections to a policy and lay audience—this being evidenced by a lingering level of mistrust despite the model projections having proved to be a useful policy tool and mechanism for translating epidemiological knowledge into accurate assessments of the likely range of public health burden.

 

Severe disease projections made at the time of Roadmap step 4.

References

[1] Lillie, Patrick J., et al. "Novel coronavirus disease (Covid-19): the first two patients in the UK with person to person transmission." Journal of Infection 80.5: 578-606 (2020).

[2]  Gov.uk. “UK Coronavirus dashboard.” https://coronavirus.data.gov.uk/details/deaths (2022)

[3]  Institute for government. “Timeline of UK government coronavirus lockdowns and restrictions.” https://www.instituteforgovernment.org.uk/charts/uk-government-coronavirus-lockdowns (2022)

[4] Davies NG, Barnard RC, Jarvis CI, Russell TW, Semple MG, et al. “Association of tiered restrictions and a second lockdown with COVID-19 deaths and hospital admissions in England: a modelling study.” The Lancet Infectious Diseases 21(4):482–492 (2021).

[5] Sonabend R, Whittles LK, Imai N, Perez-Guzman PN, Knock ES, et al. “Non-pharmaceutical interventions, vaccination, and the SARS-CoV-2 delta variant in England: a mathematical modelling study.” The Lancet 398(10313):1825–1835 (2021).

[6] Sage “Introduction to epidemiological modelling, October 2021.” https://www.gov.uk/government/publications/introduction-to-epidemiological-modelling/Introduction-to-epidemiological-modelling-october-2021. (2021)

Please sign in or register for FREE

If you are a registered user on Nature Portfolio Health Community, please sign in