September 2020 was a turbulent time in the UK. COVID-19 infection numbers had been relatively low over the summer but were rising rapidly in most areas. Schools and many workplaces were open; students were going back to universities. Vaccines and lateral flow tests were not yet available. Within the month many policy changes were introduced including travel restrictions, local lockdowns, self-isolation support payments and fines, the “rule of six” limiting social gatherings, and pubs closing at 10pm. In the midst of all this, the NHS COVID-19 app for England and Wales was launched on 24 September.

The app was designed to quickly alert people at higher risk of having been infected so that they could self-isolate (or, later, take a test) and hopefully avoid passing on the virus. Built on the Google Apple Exposure Notification system, it uses Bluetooth to sense when other phones with the app installed are nearby. If a user reports a positive test result, they can choose to anonymously notify other users with whom they have been in recent close contact. (See “how the app works” for more details.)

In our paper we considered data from the first year of the app, September 2020 to September 2021. That year included the emergence of the Alpha and Delta variants, local restrictions, national lockdowns, gradual lifting of restrictions, vaccination roll-out, increasing use of lateral flow tests, and self-isolation policy changes (including recommendation to take a PCR test from Spring 2021 onwards). We describe how the app’s role and significance varied against this changing backdrop. Our estimates include a lot of uncertainty because the app is privacy-preserving by design, meaning that it collects only the minimal data necessary to ensure its correct operation. Nevertheless, we felt it was important to evaluate it, both to understand its role in the pandemic and to inform the development of any similar future digital interventions. 

We show that the app played an important role in England and Wales. It had high uptake and engagement, its notifications identified people at higher risk of infection quickly and accurately, and this led to around 1 million cases being averted, corresponding to about 44,000 hospitalisations and 10,000 deaths avoided. Here I will outline our main results; see the paper for the full details with all the scientific caveats and discussion. Selected figures are reproduced here, with numbering to match that of the paper.

Figure 1a shows that uptake - the number of devices with the NHS COVID-19 app installed, and those with contact tracing enabled - varied over the year, notably increasing during the steps of the “roadmap out of lockdown” which saw many public venues re-opening and requiring visitors to check-in, with the app’s QR function a convenient way to do this. Uptake also varied geographically, as can be seen from Figure 1b. Early fluctuations in the uptake seen in Figure 1A were caused by missing or duplicated data packets; the issue was fixed via an app update in December 2020. 

When users install the app they are asked for the first half of their postcode and to confirm that they are over 16 years old; we don’t know how accurately users respond to these questions. The app does not collect data about the ages of its users, their GPS location or vaccination status. Nevertheless, we can get a measure of app engagement by calculating the percentage of “eligible” (aged over 16) cases which appeared on the government dashboard which were also reported through the app each day. This measure was over 20% throughout the year and peaked at over 60% in July 2021 (Figure 3b).

The average number of exposure notifications per positive test entered in the app gives a measure of population mixing - how much people are contacting each other. We see from the red line in Figure 4b that at times of heavy social restrictions, only 1 or 2 notifications were received per positive test, meaning that app users were not interacting much. By June 2021 when there were more app users, cases were concentrated among young adults, and people were able to mix more freely, an average of 7 notifications were received per positive test reported through the app. The high number of positive tests and the resulting notifications during June-July 2021 were referred to in the media as the “pingdemic”; this was followed by a drop-off in uptake (Figure 1a). 

In total there were about 7 million notifications during the app’s first year. It is important then to ask whether these notifications were working as intended, reaching individuals at higher risk of having been infected. Data available for measuring this is quite limited. We can see whether a user entering a positive test result has recently been notified, but we cannot measure the “leaky pipeline” between the two events: how many users delete the app upon receiving a notification? How many then take a test? And, if positive, how many report it through the app within the expected timeframe? From anecdotal conversations it seems likely that this pipeline is fairly leaky, particularly following the negative media attention of the pingdemic. Nevertheless, we estimated the proportion testing positive after exposure notification - a proxy for the “infectiousness” of the people testing positive each day - and, whilst acknowledging that it is likely to be an underestimate, we found that it decreased with vaccination roll-out and increased again with the emergence of the Delta variant (Figure 5c). As a measure of the accuracy of the app, we found that a notified app user was more likely to test positive than a randomly-selected member of the public (Figure 5d) or a non-notified app user (Figure 5e) by a factor which varied considerably over time but was always at least “twice as likely”, and the latter peaked at “77 times as likely” (95% confidence interval: 54, 106) in April 2021. 

A key benefit of the app is that users can be alerted quickly, usually within 4 hours of a positive test being reported. Of course, if an app notification arrives only after an individual has already found out they are at risk of having been infected, for example via word of mouth, traditional contact tracing, developing symptoms and/or testing positive, then the app alert is likely to be redundant. (It may still play some role, for example, an individual developing a sore throat might exercise more caution if this is combined with an app notification, but to be conservative in our estimates we did not attribute any effect to an app notification if it arrived after an individual had “realised another way”.) Unfortunately there is no good data on the relative timings of these other alert mechanisms, nor on how much they (or the app) caused people to change their behaviour, so our estimates have wide uncertainty. We performed a sensitivity analysis (running our analysis repeatedly with a range of different assumptions) to try to cover a wide range of plausible timings and behaviours. 

We estimated that the app’s contact tracing function alone averted about 1 million cases (sensitivity analysis 450,000-1,400,000) during its first year, corresponding to 44,000 hospital cases (20,000-60,000) and 9,600 deaths (4,600-13,000). We found that the effectiveness of the NHS COVID-19 app in reducing cases was concentrated over two key time periods: December 2020 to January 2021, and July to September 2021 (Figure 6a). These periods were when the app had most “opportunity” to act: when there was high uptake, case rates were high and/or rising, and there was reduced reliance on other interventions, e.g. people were mixing more freely. The effectiveness of the app in averting deaths was particularly concentrated in winter 2020 to 2021, before vaccinations considerably reduced the case fatality rate. Building on previous findings, we found that the effectiveness of the app varied geographically, reducing the number of cases in each local authority over the year by up to 30% (Figure 6e).

Overall, our results show that digital contact tracing played an important role in reducing COVID transmission in England and Wales in practice, as was expected in theory. The effect of contact tracing apps can be improved by increasing uptake and engagement, and increasing trust so that users promptly self-isolate and/or take a test when they receive a notification. We conclude that digital contact tracing - a relatively low cost and rapidly available intervention - is a valuable public health measure.