will_decision.xlsx

	A	B	C
1	Probability to be estimated	Fractional probability	Explanation
2	P(SARS-CoV-2 existed in the wild by 2019) * P(SARS-CoV-2 spilled over from the wild in 2019)	0.015	I have seen various estimates of the probability of a spillover per year from 0.01 to 0.1. These numbers seem are a compound of the probability the virus exists, and the probability it emerges. I have used this framing because I will compare the same for LL (probability it leaks from WIV * probability that WIV could/would make it). The former can be estimated to some degree from phylogenetic or statistical arguments (assuming a ZO of course), while the latter is impacted by population patterns, trade flows, state of economy, etc. For a simple starting point, if we consider pandemics of the 20th century (as both sides do in this debate), we can get very contrasting numbers based on the reference population we use. If we look only at Sarbecoviruses, only at Coronaviruses, include other viral phyla, include bacteria, etc we get differnt basal rates. It also matters what kind of pandemic - do we consider only those that are truly epochal (e.g. H5N1) or everything above 200k+ deaths? At a larger scale our estimates are more precise, but less accurate due to ascertainment bias (e.g. you don't see the random new Hanta virus species that kills one person). I think the probability of spillover/year for Coronaviruses might be as high as 0.25 based on the number of unique Coronaviruses that have emerged in the 2000's alone (e.g. see ZO, debate 1, slide 6). On the other hand, given that none of these were even close to as infectious as SARS-CoV-2, that number if clearly too high. Some evolutionary rate estimates might help here (e.g. divergence of RATG13 was ~40 years ago under belief in ZO), but niether side discusses these numbers in depth. Ultimately, if I think that emergence is a function of human urbanization patterns, economic flows, etc. I should use recent years as estimates. The convenient estimate of Keusch et al. 2022 (ZO, debate 1, slide 6) would be 6 human coronaviruses in 20 years = 30%/year. If I further assume that the probability that these are pandemic-capable is less a function of temprorary patterns and more a function of immune-virus interactions, then I can consider longer time scales. LL suggests 5 flu pandemics since 1920 century with more than 200k deaths (LL, debate 1, slide 21). 5/100 * 3/10 = 0.015/year. This is obviously a grotesque simplication, but seems reasonable (especially given the low level of discussion in this debate) and in line with LL (reference slide 103, LL gives an estimate of 5%/year chance of zoonotic event).
3	P(A leak from WIV would occur in 2019)	0.05	The per year lab leak rate can be estimated in a huge number of different ways. I believe the relevant factors are the goal of the research, the scale of the operation (numbers of infectious particles created/year), the number of types of experiments - especially those at multiple different biosafety levels or where samples are transported out of labs, the laboratory equipment (e.g. hoods, respirators, aerosol minimizing lab equipment), the sterilization infrastructure (on-site vs. offsite, autoclave/EO/H2O2, etc.). I would have liked to have seen more of this discussion, especially from the LL side. They give a final estimate of P(leak)/year=15% (LL, debate 3, slide 103). Debate 1 slides reference a variety of reasons for this (BSL2, mouse work, etc) but I think 15% is too high. Extrapolating from past data would have helped me believe this number a lot more. In the inidial debate, the priors that a leak would occur, given a very infectious agent was created, are estimated by LL in slides 19-24 (debate 1) as approximately 1 in 5 based on prior lab leaks, the fact that WIV likely did experiments in BSL2, and the growth in GOF work. I find the evidence presented in slides 19-24 to be very imprecise (see my extended discussion of these slides in the "meta level considerations" section). Briefly, I find the examples of previous lab leaks cited as some combination of wildly different in scale (e.g. the 1979 anthrax release from an offensive bioweapons facility with 1000L+ fermentors and dryers) or intentionally skewed (e.g. 1977 H1N1 - no comparison of vaccine trial vs. lab leak). These make me skeptical of Rootclaim's committment to objectivity, but at a more specific level, I don't think they support their high estimate of per year lab leak rates. I also think no evidence that improved GOF was relevant (no large scale DNA synthesis, no large scale competition or binding assays, etc.) was presented. The only experiment LL explicitly references is the potential for a FauI site to have been added, and that's relevant closer to the single grad student, single 12 lane gel scale than it is the BGI scale. I think sources with a more reasoanble accounting of lab risk based on the above factors exist, but I felt uncomfortable pulling them because ZO did not present significant evidence on this question. My intuition is that a 0.02 would be a very high rate, and that 0.01 would is closer to a sober analysis of lab leak rates, but I have set it equal to a higher value (0.05) based on the evidence LL has presented and uncertainty around this.
4	P(WIV was carrying out some DEFUSE style research)	1	The DEFUSE proposal represents a specific proposal for coronavirus research drawn from a much larger body of work and potential experiments. Proving that the DEFUSE proposal did not contain an exact experiment that would have created SARS-CoV-2 has limited probative value for three reasons. First, there are many experiments the researchers creating DEFUSE would want to run with the money, but would not include in DEFUSE due to the grant length and explanability constraints (based on my experience in writing RO1's, SBIRs, etc). Second, the research program that generated DEFUSE has many other arms, proposing similar/adjacent experiments. In other words, it seems clear to me that the (collective) research program that generated DEFUSE would be likely to generate other projects/experiments capable of creating SARS-CoV-2. Third, in my training, I was taught to write grants that were "30% completed and 70% to be done". In practice this was a substantial exaggeration, but I think it speaks to the idea that most grant proposals have preliminary steps or POCs for many aims already done. I believe that WIV was certainly doing some research with the goal of creating GOF mutants, and would have planned to continue that work to prepare for a future grant cycle. I estimate a higher percentage here than both LL and ZO (e.g. reference LL, debate 3, slide 103 which ZO concedes/uses in slide 57).
5	P(WIV researcher would have made SARS-CoV-2 if they could)	0.5	Similar to above (without taking into account any of the genomic evidence - that’s handled in the "bayes_factors"), I think it's 50:50 that WIV researchers would have chosen to create something as infectious as SARS-CoV-2 if they could. From the debate, my understanding is that most of the GOF work was to be done at UNC, but given that DEFUSE wasn't funded, and WIV had the samples, I can see ongoing efforts there work towards creating GOF mutants as the basis of new grant applications or publications. I have discounted this from the certainty above because I think it's equally plausible they were going a different direction with the research (e.g. cataloging, sampling, and sequencing rather than constructing).
6	P(WIV had undisclosed and sequenced backbones)	1	Based on working in 4 microbiology labs with large collections of samples from unique and interesting locations, I would be extremely surprised if WIV did not have a large set of samples that had yet to be sequenced. They also probably had many samples that were sequenced, but which weren't deposited on the publicly accessible server (likely just on a single grad student's hard drive).
7	P(WIV had an undisclosed backbone close enough to make SARS-CoV-2)	0.01	This is an area that I think the ZO side won rather cleanly in the debate. The LL side did not present any credible evidence (IMO) of secret backbones with high nucleotide identity to SARS-CoV-2. LL provides an estimate of 50% chance that the '180' undisclosed viruses that DEFUSE had have a 50% chance of being a good enough backbone (LL, debate 3, slide 103). I think this is substantially too high. To provide a brief recap (see the 'backbones' subsection for more details). RATG13 (96.1% similar to SARS-CoV-2, ~80% similar to SARS1) was found in 2013, and it's sequence was disclosed in 2020. There is no evidence WIV made backbone tools for this virus in the 6 years between acquisition and pandemic and it seems extremely improbable that this could have been used as the starting point to make SARS-CoV-2 due to NT divergence (ZO, debate 2, slides 11-24). Viruses discovered later including BANAL-52, BANAL-236 (Yuri: ~3% NT, 1% AA difference, BANAL-52 has 77/78 and 175/180 AA identity in RBM and RBD, respectively) and RmYN02 have some features that are closer, but overall still appear much too dissimliar to be a few years of lab passage away from SARS-CoV-2. Ecohealth published sequences of 200 viruses in August 2020, based on a paper submission date of August 2019. Assuming the LL had not occurred (this predates even the earliest circulating virus claimed by LL in this debate - see LL debate 3 Q&A), they would have no reason to hide any viruses (ZO, debate 2, slides 25-28). Even if the leak had occurred in August, with the researchers somehow monitoring but hoping it wouldn't emerge, a rejected 2018 paper with automatic embargo lift in 2022 would have disclosed these viruses (ZO, debate 2, slides 25-28). I think ZO makes a fairly convincing case that the LL side has made many predictions that WIV is nefariously hiding sequences that have failed to materialize. I think the '180' viruses figure they are describing are the known viruses that DEFUSE had. Now, I believe it's very probable the lab had sequenced viruses it didn't disclose in existing publications because it wanted to retain them for future publications (very different than hiding them). The LL side notes this, and suggests a significant time delay between collection and publishing (LL, debate 3, slide 31). Were the undisclosed viruses potential backbones? This is where more subject matter expertise would have helped from the debaters. As I tried to suggest in debate 2 questioning, there are standard tools used in microbial ecology to estimate how much diversity additional sampling will yield. Intuitive, though perhaps statistically inadmissible rarefaction curves could have given a sense, as perhaps could have phylogenetic diversity measures like NRI/NTI. NRI/NTI might be interesting in that they could suggest if there had been intentional removal of sequences in the trees of what WIV disclosed (but again, total speculation). I proposed a (probably stupid) empirical approach in the debate (calculate tip-to-tip distance matrix of known WIV samples to SARS-CoV-2, fit a distribution to that, calculate how unlikely drawing SARS-CoV-2 would be from that distribution). In leiu of this, on the LL side I am left evaluating extrapolations from DEFUSE proposal numbers, two tweets about later than disclosed sampling trips (LL, debate 3, slides 31-38), and the timing of the WIV sample database going offline (LL, debate 3, slides 117-118). In regards to the DEFUSE proposed sampling locations, the bats that DEFUSE would have sequenced were already well-represented in the previous WIV collections (e.g. see ZO, debate 3, slides 45-46). DEFUSE was going to similar spots as they had previously, which have so far failed to yield anything close to SARS-CoV-2. Obviously they would have wanted to add to the diversity of their collection, but I am weighing that against their directly stated intention, and my guess that they had permits for specific caves. The fact that the WIV database went down, came back up, then went down (LL, debate 3, slides 117, 118) seems to be inconsistent with the timing of the pandemic. I can see one scenario consistent with LL: the researchers knew/suspected they released something in e.g. August, but were monitoring and hoping it didn't cause a problem. This could occur with e.g. a faulty sterilization system that was known about, but where environmental monitoring didn't show leaked sequences of relevance. In this case, they bring the database back up in September, see something that spooks them, and then take it back down. Ultimately, I would probably say 1 in 1000 chance that WIV had an appropriate backbone. I have moderated that 10X in LL's favor because I do find the (later) non-sharing of the database odd. I think there are many reasons they would not share this database unrelated to GOF work, but it is somewhat suspicious.
8	P(WIV made >10 fully functional GOF mutants)	0.1	To separate these probabilities from all the others (e.g. not to double count against LL my strong belief that WIV did not have a close backbone), I am assuming that they have all the backbones they need. I think that ZO wins that the size of the working group at WIV was limited, and they probably didn't have much money to work with. The tools needed would be reasonably cheap (labor + sequencing + Gibson assembly) for small scale operations, but for making a large number of mutants this would require automation. LL side discusses the potential for using RFLP analysis (or just perhaps a quick restriction digest) on the FauI site found in the PRRAR FCS. If this is true, it would IMO point to a small scale operation where one grad student is running 10-50 lane gels, not making 1000's of different constructs (you'd use sequencing or some colormetric read out for that I would think). This scale about in line with the 8x 2017 WIV1 chimeras produced (ZO, debate 2, slide 7). I think the probability of a leak scales with the size of this operation, and LL didn't convince me it was particularly large. Thus, I am assuming 1-2 grad students making a new backbone system and putting/designing the different mutations. Obviously, 1 GOF construct would have been enough for a leak, but I had to pick some way to discount for resource/person hour constraints and settled on this.
9
10
11	Ratio or quantity to be estimated		Explanation
12	P0(LL) / P0(ZO)		The ultimate starting quantity I want - the prior odds of lab leak vs. zoonotic origin.
13	P0(LL) / P0(ZO) = [P(lab leak in 2019) * P(lab made by 2019)] / [P(natural leak 2019) * P(nature made by 2019)]		I want to evaluate the evidence that a lab leak would could occur and multiply by the probability that there was something to leak. Although it's a somewhat awkward framing to compare this to the probability of a natural creation and a natural leak, I think this is the only way to incorporate uncertainty about whether or not WIV was actually doing the activities that would have created a pandemic-capable virus.
14
15	P(lab leak in 2019) = 0.05		From above, P(A leak from WIV would occur in 2019). I am intending this as P(A leak from WIV would occur in 2019 \| WIV has a pandemic capable virus to leak).
16	P(lab made by 2019) = product of		Reference all the data above for explanations of each value.
17	P(WIV was carrying out some DEFUSE style research)	1
18	P(WIV researcher would have made SARS-CoV-2 if they could)	0.5
19	P(WIV had undisclosed and sequenced backbones)	1
20	P(WIV had an undisclosed backbone with NT similarity > 98%)	0.01
21	P(WIV made >10 fully functional GOF mutants)	0.1
22	P(lab made by 2019) = 5.00E-04		This is my starting probability that I think WIV could/would have made SARS-CoV-2 by 2019.
23
24	P(natural leak 2019) * P(nature made by 2019) = 0.015		From above, I want to estimate the probability that nature made this virus P(SARS-CoV-2 existed in the wild by 2019) * P(SARS-CoV-2 spilled over from the wild in 2019)
25
26	P0(LL) / P0(ZO) = [P(lab leak in 2019) * P(lab made by 2019)] / [P(natural leak 2019) * P(nature made by 2019)] = 5E-4*0.05/0.015
27	P0(LL) / P0(ZO) = 1.7E-3		My final estimate of the starting probability. I believe that lab leak is 1/600 chance compared to natural origin. Note this appears more ZO favored than ZO's own starting point (ZO, debate 3, slide 58) but it incorporates data that ZO uses to apply approximately a ~1E6 factor against LL. I believe this is a reasonable answer (I did not post-hoc adjust this) that is within the 1:100 - 1:1000 ranges that other analyses cited in this debate start with.
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100