Misconceptions on British tech response to COVID-19

Following our analysis in May of UK's NHSX tracking app, the news that the UK is switching to use Apple & Google's Exposure Tracking APIs, and the recent appearance of contact tracing on iPhones, I'm sharing analysis of what's happened since.

There's been plenty of politics at play in terms of how the British COVID response has been communicated, and there are a number of common misconceptions.

The first misconception is that the Government was committed to pursuing "going it alone". It's true that they announced that they were going to try this, but it's also a statement of fact that they also contracted to build an implementation using Apple and Google's API. This was done May 6th 2020, and the contract can be read here. That's before the Isle of Wight trial, and before any external analysis of the NHSX app.

The second misconception is that the NHSX app was inadequately tested. When the Isle of Wight trial was started, the NHSX open-sourced their app and shared a number of documents about how it was built and tested. Many journalists asked me whether we felt those documents showed an adequate test methodology had been used. The discovery Reincubate made and covered was that the NHSX team had found a novel technique to attempt to address the shortcomings of the Australian contact tracing app, but it wasn't clear how well it would work.

Commentators, expert analysts and the cyber-security firm the BBC retained lacked certainty on everything apart from the fact that there were technical advantages of the Apple / Google API approach, and that the only way to ascertain how effective this technique would be at working with the caveats it had would be to test it.

Of course, the ideal test of a complex system such as this is to roll it out to a reasonable number of users in order to gather real-world data on how well it worked. That's exactly what was done, and it's strange that some commentators conflated the trial in the Isle of Wight with a rollout that wasn't a trial.

Thirdly, there's a misconception that the NHSX app was a waste of money, or led to a delay in the Government rolling out a "working" app. There are two reasons why this is faulty. As I wrote back in May, deploying an app using the Apple or Google API was not a practical option. I shared the following four points:

• iOS 13.5 has not been released, and may not be for some weeks
• Prior to yesterday, the last iOS 13.5 beta had a major security flaw, suggesting heavy lifting is going on in Apple’s engineering teams
• Once it is released, it will likely take months for a majority of iOS users to install it (unusually, the equivalent Android adoption may be more rapid)
• Older iOS devices — such as the iPhone 6 — cannot run iOS 13, and will not be able to use the Apple technique

Since then, not much has changed. iOS 13.5 was released 20th May with the "exposure notification" APIs. Statscounter reported a 2.2% adoption rate for 13.5 by the end of that month, and only 37% for March's iOS 13.4. iPhones sold as recently as 2015 can't run iOS 13, and it'll likely be near the end of 2020 before any reasonable number of users are running 13.5. (As of June 2020, Apple's data shows that 92% of devices run iOS 13, but they exclude devices that are more than 4 years old.)

As a result, given how important adoption is for the technology to work, it might be feasible to explore using Apple's APIs in early 2021. Perhaps if the UK doesn't have an Apple / Google API-based tool within a few moonths, that could be considered a delay. (Singapore is rolling out dedicated hardware devices to its population to address this problem.)

However, in the meantime, the trial on the Isle of Wight has concluded, as have tests on the Apple and Google APIs. The data is unhelpful for both approaches, with the London Assembly discussing distance problems. The BBC noted:

Specifically, the software registered about 75% of nearby Android handsets but only 4% of iPhones. By contrast, the Apple-Google model logged 99% of both Android mobiles and iPhones. But its distance calculations were weaker. In some instances, it could not differentiate between a phone in a user's pocket 1m (3.3ft) away and a phone in a user's hand 3m (9.8ft) away.

The "waste of money" accusation is an unusual one. When a Government is faced with its citizens dying, what price should it put on investing in technological R&D to save lives? Should they have waited (or still be waiting, in practical terms) for Apple's solution to be available? Had they waited for that, and then run into problems with distance tracking and adoption of iOS 13.5, it seems like spending millions of pounds exploring whether another approach could work wouldn't be that bad. As it was, the dual-track approach the UK has run seems like the gold standard — not sitting on its hands, waiting for something that might not work from the tech giants.

Finally, there's a new picture emerging, and that is that technology won't provide a panacea for contact tracing. Neither of the approaches used — the NHSX's, or Apple & Google's — work as well as had been hoped.

The problem seems to be fourfold:

• Adoption (users don't have iOS upgrades or Android software required to use the technology)
• Hardware capability (iPhones from 2015 can't run the app, the average Android device has low quality Bluetooth which won't track distance well)
• Privacy (users are concerned about installing or granting access to the apps — Apple enabling the settings menu for its system was enough to cause consternation in the UK in late June)
• Behaviour (users won't necessarily follow up on COVID reporting or testing requirements for the apps to work — as it is, there are concerns over much simpler social distancing rules being ignored)

Whilst some countries have claimed limited success with their contact tracing apps, data is scant, and that question of whether the technology works is now firmly a politically-led — and not technological — narrative. My view is the best users may get from these technologies is partially accurate exposure notification, as distinct from contact tracing.

It may turn out that neither approach is ultimately effective in tracking and managing the spread of COVID-19, but it's hard to argue against the sense of investing in a number of technical solutions in the hope of doing so.