I previously wrote about covid tracking apps and that what a “tracking app” is varies tremendously by country. Please read that first.
There is another issue that crops up – and that’s the effect of conditional probability (or Bayes’ Theorem). I am still reviewing this to see if I did something wrong (I think I did do some things wrong – see Afterword – the idea is probably sound but the calculations may be wrong) – with reasonable assumptions, phone-based covid-19 contact detection apps seem unlikely to provide a benefit.
Singapore is the only country to have used a Bluetooth-based contact app so far. At the time of this writing, the U.K. is running a trial of BLE-based tracking apps.
Singapore achieved only a 12% participation rate. (Update: as of end of May it was up to 25%, which is still too small. Consequently, Singapore cancelled the smart-phone based contact tracing app).
The probability that a person has the app on their phone is then p=0.12. The probability that a 2nd person has the app on their phone is also p=0.12. The probability that two strangers coming within close range both have the app installed is just .0144 or 1.44% – almost zero. (Update: at the 25% participation rate, this works out to 6.25%, which is still nearly zero).
There is another problem with tracking apps due to Bayes’ Theorem and that the true incidence of Covid-19 in the sample population may be very low.
- Let’s assume that 1 in 500 people are currently carrying Covid-19 and could spread the infection.
- Let’s assume 100% of everyone has the Bluetooth app installed, and 100% of the population has a smart phone, and its on all the time.
- Let’s assume that the app is 90% accurate in finding a true contact, and 10% of the time, it flags a contact we do not need to worry about (a false positive). In reality, I suspect the accuracy is quite a bit less than 90%, but let’s work with the 90% figure for this example.
Out of 500 potential contacts, 10% or 50 might be flagged as false positives and 1 person might be flagged as a true positive. We have now flagged 51 people in the population. (But see Afterword for the problem with this calculation.)
But we know we expect only 1 true positive out of all those sampled – but we now have 51 positive results. We do not know which one of those 51 is the true positive. For any individual in that set, the odds that they are the actual true positive is 1/51 or just 1.96%.
This means, if we make the above assumptions, and further assume that 50% of all smart phone users install the app, giving us a chance to detect 16% of potential contacts – the likelihood we find a true case is 16% x 1.96% or 0.3% (one third of 1%).
Which is indistinguishable from zero. Thus, contact tracing apps based on BLE are unlikely to provide any true benefit.
You can work this out for other values such as 1 in 100 have Covid-19. Then we would have 1 in 100 get a true positive and 10 out of 100 get a false positive. We now have 11 people identified of which only 1 is a true positive or 1/11 or 9.1%. This raises our true detection to about 1.4%.
One way this would be improved is if you had multiple independent contacts with a subject. Bayes’ Theorem tells us how to incorporate additional evidence into our probability. A second contact dramatically increases the probability – instead of a low value we might calculate a p of 0.9 +/- depending on assumptions.
Do NOT take my word for this. My analysis could be wrong.
Afterword
Vertasium (Derek) has an excellent introduction to conditional probability and Bayes’ Theorem.
Issues and Problems With This Analysis
The above analysis assumes in this concept of false positives, that the 50 “false positives” are people who are later flagged as having Covid-19. That is not realistic since only 1 in 500 are true positives (and some percent of those tested are true and some are false), we would have to encounter 50 x 500 people or 25,000 to find that 50 of our “positive” contacts were “false positives” meaning too far away, physically separated or otherwise blocked for us to be concerned with a contact.
We could modify this to the 1 in 100 level, with a 10% false positive rate – and at this level, we would only have to encounter 1,000 people, which seems realistic, especially for those in a big city, using public transport, and generally encountering many people over the course of a few weeks. The calculations would be a little different: we’d have 10 true positives out of 1,000 people and 10 x 10 or 100 false positives.
I think the idea is correct but not sure about my implementation as this is not something I know a lot about.
We also should add another layer in the Bayes network to incorporate the probability of a Covid-19 test being true or false, correctly or incorrectly.
A related problem is that in the UK implementation of a BLE tracking app, end users may flag themselves as having Covid-like symptoms. This means anytime someone gets sick (or if they are just mischievous) they might flag themselves – and then, those a lot of false positives would pop up in a small sample.
