We cannot trust current COVID-19 models
By Eran Bendavid, Jay Bhattacharya, and Neeraj Sood
March 21, 2020
Extraordinary measures like shelter-in-place orders and quarantines are justified if unchecked, COVID-19 would kill millions. However, we have not measured the single most important piece of information to know how deadly this virus really is and current estimates could plausibly be off by orders of magnitude. Despite the dire forecasts of many epidemiologists and modelers, we have little reliable information that COVID-19 will kill millions of Americans even without shelter-in-place policies.
The fear of COVID-19 is based on its high estimated case fatality rate — about 2% to 4% of people with confirmed COVID-19 have died. So if 100 million Americans ultimately get COVID-19, an unthinkable 2 million to 4 million may die. We believe that the estimate is deeply flawed. The true fatality rate is the portion of those infected who die, not the deaths from among the identified positive cases. The number of cases is a misleading estimate of the number of infections because of selection bias in testing. Just how biased it is could make the difference between an epidemic with 2 million or 20,000 deaths. If the number of actual infections is much larger than the number of cases - orders of magnitude larger - then the true fatality rate is much lower as well. Is this plausible? The data to date suggest this is not only plausible but likely.
Population samples from China, Italy, Iceland, and the US provide relevant evidence. On or around January 31, countries sent planes to evacuate citizens from Wuhan . When those planes landed, the passengers on those planes were tested for COVID-19 and quarantined. After 14 days, the measured prevalence was 0.9%. If this was the prevalence in the greater Wuhan area on January 31, then, on a population of about 20 million people, greater Wuhan had 178,000 infections, about 30-fold more than the number of cases. Even if some of those infected would eventually turn into cases, the fatality rate was near 0.1%.
Next, the Italian town of Vò in Padua. On March 6, all 3,300 people of Vò were tested, and 90 tested positive, a prevalence of 2.7%. Applying that prevalence to the larger province of Padua (population 955,000) which had 198 reported cases, suggests there were actually 26,000 infections at that time -130-fold the number of reported cases, and a fatality rate of 0.03%.
In Iceland, deCode Genetics is partnering with the government to perform widespread testing. In a sample of nearly 2,000 entirely asymptomatic individuals, researchers estimated disease prevalence of just over 1%. Given that Iceland’s epidemic is weeks behind that in the US (their first case was reported on February 28), is it plausible that the proportion of the US population that has been infected is double, triple, or even ten-fold higher? Yes.
Finally, the best (yet very weak) evidence in the US comes from the National Basketball Association. Between March 11 and March 19, a number of NBA players and teams received testing. By March 19, 10 out of 450 rostered players were positive. Since not everyone was tested, that represents a lower bound on the prevalence of 2.2%. The NBA is not a representative population and maybe contact among players facilitated transmission. But if we extend the data to cities with NBA teams (population 45 million), we get at least 990,000 infections in the US. During those few days, the number of cases reported in the US was 72- to 773-fold lower. These numbers imply a fatality rate from COVID-19 orders of magnitude smaller than it appears.
How can we reconcile these estimates with the epidemiological modelers? First, the test used to identify cases - a PCR-based test - is unable to catch people who were infected and recovered. Second, testing rates were woefully low for a long time and typically reserved for the severely ill. Together, these facts imply that the confirmed cases are likely orders of magnitude less than the true number of infections. Epidemiological modelers have not adequately adapted their estimates for these facts.
The epidemic started in China sometime in November or December 2019. The first confirmed US case was a person who traveled from Wuhan on January 15, 2020, and it is likely that the virus entered before that: a lot of people traveled from Wuhan to the US in December. Existing evidence suggests that the virus is highly transmissible and that the number of infections double roughly every three days. An epidemic seed on January 1 implies that by March 4 about 2 million people in the US were infected. We have had 118 COVID deaths to date from these 2 million infections, a mortality rate of 0.01% (one-tenth of the flu mortality rate of 0.1%). By now, we may have had as many as 8 million Americans infected, 2.5% of the population. If that is true, this should be very reassuring news.
This does not make COVID-19 a non-issue. The daily reports from Italy and across the US show real struggles and overwhelmed health systems. However, dealing with and addressing a 20,000 or 40,000-death epidemic is very different from a 2 million-death epidemic. Given the enormous consequences of decisions around COVID-19 response, getting clear data to guide our decisions now is critical. We do not know the true infection rate in the US. Antibody testing of representative samples to measure disease prevalence (including the recovered) would provide immediate, actionable information.
If epidemic deaths are, as we believe, likely to be much more limited, then measures focused on older populations and hospitals are sensible. We will need to find ways to prevent hospitals from reaching a breaking point treating those COVID-19 patients such as rescheduling of elective procedures, repurposing of hospital resources to care for critically ill patients, and improved triaging. And we will need to focus on reducing risks for older adults and individuals with co-morbidities. A universal quarantine may not be worth the costs, and we should undertake immediate steps to evaluate the empirical basis of the models supporting it.