Those of us who have seen Neil Ferguson’s ICL Covid sim model have the same views as this computational epidemiologist:
As Ferguson himself admits, the code was written 13 years ago, to model an influenza pandemic. This raises multiple questions: other than Ferguson’s reputation, what did the British government have at its disposal to assess the model and its implementation? How was the model validated, and what safeguards were implemented to ensure that it was correctly applied? The recent release of an improved version of the source code does not paint a favorable picture. The code is a tangled mess of undocumented steps, with no discernible overall structure. Even experienced developers would have to make a serious effort to understand it.
I’m a virologist, and modelling complex processes is part of my day-to-day work. It’s not uncommon to see long and complex code for predicting the movement of an infection in a population, but tools exist to structure and document code properly. The Imperial College effort suggests an incumbency effect: with their outstanding reputations, the college and Ferguson possessed an authority based solely on their own authority. The code on which they based their predictions would not pass a cursory review by a Ph.D. committee in computational epidemiology.
When people mention “Covid tracking apps” it would be useful to first define what is meant by “Covid tracking app”. There are many approaches in use and many that are proposed. The various methods are remarkably different. When you hear that “Country X used a tracking app and they have fewer cases”, this does not mean they used a tracking app like you have in mind.
Most apps use location data provided by the cellular network itself or on GPS/Wi-Fi position fixes stored on the phone and shared directly with public health authorities. Some use the data for contact tracing, coupled with free Covid-19 testing, while others use location data to enforce strict geo-fenced quarantine procedures that if violated, may result in arrest and imprisonment. Few existing apps use close contact tracing based on Bluetooth.
Contact tracing apps, by themselves, appear to provide little value. As we will see, to be useful there needs to be supporting infrastructure outside the app – such as Korea offering Covid-19 testing to those in close contact. And the app must be installed by nearly all smart phone users (and this will miss about 15% of phones that are not smart phones). Most countries are not using phone-based apps to track location – they are using the phone network to report locations on 100% of phones in use, which is very different than voluntary installation of a tracking app.
Consequently, when you hear someone refer to “contact tracing app”, you need to ask them to define what they mean by “contact tracing app”.
What follows is a review of various “contact tracing” apps used in different countries.
Update – I see one financial analyst thinks “It’s different this time” and there is little risk of inflation. So there is that view too. Deflation is certainly likely in the shorter term due to suppressed demand (retail sales and services fell by over 16% in April).
Why the stock market rise? Probably because of future inflation. The U.S. government is printing money like crazy – calling it an “economic stimulus”.
Chart of the U.S. Money Supply from U.S. Federal Reserve:
How will these trillions in spending be paid for? Inflation.
Inflation devalues the dollar making today’s debt’s cheaper to pay off in the future. Governments have always done this. Will this be what happens this time? Who knows for sure?
Inflation taxes everyone simultaneously by lowering the purchasing power of the dollars they hold.
“The technology is more or less … I wouldn’t say useless,” says Gestur Pálmason, a detective inspector with the Icelandic Police Service who is overseeing contact tracing efforts. “But it’s the integration of the two that gives you results. I would say it [Rakning] has proven useful in a few cases, but it wasn’t a game changer for us.”
He says there have been instances where the data was useful, but that the impact of automated tracing has been exaggerated by people eager to find technological solutions to the pandemic.
Mandatory physical distancing measures, temperature checks and filling out medical history questionnaires prior to airplane flights, possible Covid-19 testing before boarding, limited or non-existent meal and beverage service on airlines, no more free hot breakfasts at hotels, restaurants allowed to use only 25-50% of their seats, mandatory face mask wearing at all times … and higher prices. Airlines can not keep flying idled seats – someone has to pay for it.. Hotels, restaurants and car rental agencies will have to charge more to fewer customers in order to cover their fixed costs.
What does this mean for travel? It means recreational travel will be limited until a vaccine is widely distributed and people have confidence in its effectiveness. Many will choose to avoid the “new normal” hassles of travel during this time.
I will not comment on Covid-19 and only make a few comments on the publicly available source code.
This is not a comment about whether models should be used – or not.
This is not a comment about whether this model’s output is correct – or not (we have no way of knowing either way). Even with the model output being off my very large amounts, we still have no way of knowing.
This is not a comment on whether there should be a lock down – or not.
This is not a comment on whether a lock down is effective – or not.
This is a review of a software project.
The review findings are typical of what is often seen in academic software projects and other “solo contributor” projects (versus modern “production code” projects). The issues that often arise in academic projects are due to the nature of individuals or small groups, not trained in software, tinkering with software code until it grows out of control. This likely occurs in other fields but seldom do such works become major components of public policy.
When software is used for public policy it needs to be publicly reviewed by independent parties. Until the past month, this code had apparently not been reviewed outside of the ICL team.
Models are a valuable tool, when properly used and their limitations are understood. A reasonable model can enable planners to play “what if” scenarios and to adjust input parameters and see what might occur. For example, consider a model for complex manufacturing – we might look at productivity measures, inventory, defect rates, costs of defect repairs, costs of dissatisfied customers, impacts on profits and revenue, supplier issues and so on. If we choose to optimize for profit, then we use our model to find optimal values for each parameter to achieve maximum profit. Or perhaps we optimize for customer satisfaction instead -what happens to our profits if we do that? That is a What-if question. For this purpose the model need not be perfect but at least needs to be “within the ball park”. The key is “within the ball park”. If the model flies off the rails in many cases, it is not a good and accurate model and there is a risk we make seriously wrong decisions.
A model may also be used to compare scenarios. We may not need precise future projections for that – instead, if we say, increase X, our model shows high profits, but in another run, we decrease X and show losses. We may not need to know the exact dollar value – only that one path leads to profit and one leads to losses. In this way, precise projections are not always essential.
This code – placed on GitHub – is apparently a revision released by the University of Edinburgh, based on the original source code by Neil Ferguson of the Imperial College of London. They are said to have asked Microsoft and others to help and clean it up and fix defects. Consequently, this is not the exact same code that Neil Ferguson was using to create is models two months ago, but code that has been since updated by others.
This code is thousands of lines of very old C programming.
First thing I noticed was how so much code has been placed in one gigantic source file – 5,400 lines in a single source file. Ouch.
This explains much:
The “undocumented C” argument comes when the author is the only one working on a project and sees no need to document their work. After all, it’s just me! There are two problems with this thinking: (1) over time, even personal projects like this one, grow in size until they become thousands of lines of code. Years later, our understanding of our own original code may not be as good as we think it is. We forget why we made particular design choices. We forget why we assumed certain conditions or values. Bottom line: over time, we forget. And (2) personal projects like the Covid-19 simulation eventually became the basis of major public policy and others are asked to review, check or modify the code base. No documentation puts the entire model at risk. This is not the right way to do these kinds of software projects, particularly when this is the basis for advising world leaders on major public policies that impact billions of people.
For Immediate Release May 5, 2020 Contact: firstname.lastname@example.orgWASHINGTON – The U.S. Department of Transportation’s Federal Aviation Administration (FAA) today announced the eight companies that will assist the Federal government in establishing requirements for future suppliers of Remote Identification (Remote ID). Remote ID will enable Unmanned Aircraft Systems (UAS), commonly called drones, to provide identification and location information while operating in the nation’s airspace.
The FAA selected the following companies to develop technology requirements for future Remote ID UAS Service Suppliers (USS): Airbus, AirMap, Amazon, Intel, One Sky, Skyward, T-Mobile, and Wing. These companies were selected through a Request for Information process in December 2018.
“The FAA will be able to advance the safe integration of drones into our nation’s airspace from these technology companies’ knowledge and expertise on remote identification,” said U.S. Transportation Secretary Elaine L. Chao.
This initial group will support the FAA in developing technology requirements for other companies to develop applications needed for Remote ID. The applications will provide drone identification and location information to safety and security authorities while in flight.
The technology is being developed simultaneously with the proposed Remote ID rule. Application requirements will be announced when the final rule is published. The FAA will then begin accepting applications for entities to become Remote ID suppliers. The FAA will provide updates when other entities can apply to become qualified Remote ID USS on FAA.gov.
Drones are a fast-growing segment of the transportation sector with nearly 1.5 million drones and 160,000 remote pilots now registered with the FAA. The agency’s ability to develop Remote ID technology simultaneously with the rule enables the FAA to continue to build on a UAS Traffic Management (UTM) system that has demonstrated global leadership through the small UAS rule and the implementation of the Low Altitude Authorization and Notification Capability (LAANC), which automates the application and approval process for most UAS operators to obtain airspace authorizations.
The FAA bungled this press release. The Remote ID Cohort group was something in the works long ago. The FAA, though, issued this press release without clearly explaining that the review of pubic comments is ongoing and that the public will not be ignored. Many of us (just look at social media) saw this press release and read it as saying that public input in the NPRM on Remote ID was being ignored. The FAA really bungled this.
I’ve emailed an inquiry to the FAA to see if they can elaborate on what this is about and what is happening in regards to the public comments. Why does the comment evaluation period need to be done in total secrecy – no information on how it is performed, how many comments have been processed, or any time frame information.
Update: May 8 – FAA has issued a 2nd press release on this
Thanks for the questions we received after yesterday’s press release on the Remote ID Cohort. To clarify, the Cohort is not part of the decision-making process for the proposed Remote ID rule final rule. The Cohort will help the FAA develop technology requirements for other companies to develop applications needed for Remote ID. The comment period on the Remote ID Notice of Proposed Rulemaking closed on March 2, 2020, and the FAA is reviewing the more than 53,000 comments
Reality-based Thinkng on Business, Tech, Energy, Transportation
Privacy & Cookies Policy
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.