taleb on popperian science.pdf

REFLECTIONS ON THE COMMENTS ABOUT OUR NONNAÏVE PRECAUTIONARY PRINCIPLE

To repeat, the gist of our paper is decision-making under opacity, fat tails, asymmetries, and systemic effects – in other words, the Fourth Quadrant. Clearly, genetic modification is a subplot (it was not discussed in the first version), but GMOs have a peculiar illustrative role be- cause they multiply, have systemic not just idiosyncratic risks, and opacity about the interractions is compounded under the curse of dimensionality.

The next version of the draft, aside from "tighten- ing" the discussion in places where some links are not explicit, will address commentary that we have been collecting for 8 months. We are incorporating some notions on tail risk from the insurance industry and hydrology (it is crucial that insurance is very careful about eliminating the tail risk exposure to GMOs). The current document was written for a nonmathematical but rigorous audience and we will keep it that way. But to rigorously "debunk" our paper one needs a bit of work to show that the precautionary principle as we defined it and narrowed it down doesn’t apply to a particular domain or in general because either:

Category 1 arguments (that we call rational):

•

absence of evidence in that domain can be deemed equivalent in effect to evidence of absence because of thin tails and/or no systemic effects,

•

we should not have precaution in that domain and provide logical reasons for it (in other words the exposure falls outside the fourth quadrant). Category 2 arguments (reflecting flawed reasoning):

•

fat tails don’t exist,

•

we should never have precaution,

•

tail risk analysis is subsumed into conventional statistical testing.

Sophisty

It is hard to believe, but so far most comments about our works fall into Category 2, of the type of sophistry: "we need to take risks otherwise we would have no progress" which leads into the absurdity of "let us fly any airplane" or "smoking is anti-progress" not realizing that only a very small minority of the "new" manages to survive, a logical fallacy of explaining the consequent. It assumes that the statement "most of our improvements comes from new technologies" is equivalent to "all new technologies are improvements" when less than < .1% constitutes so –99.9% of technologies fail. One has to be remarkably blind to the failure rate to make such flaw of reasoning. Actually our nonnaive Precautionary Principle is precisely to allow a broader class of risk; had mankind not exercised precaution (in combination with reduced- left tail risk taking) we would not be here today.

Some people invoke the "naturalistic fallacy" under a logic that would lead to blank rejection of track records,

hence statistical significance (incidentally this is a mis- representation of the naturalistic fallacy as it applies only to the moral domain not the statistical one.)

Another sophistry is invoking "scientific consensus" without explaining why, as we show in Appendix A, there was "scientific consensus" on transfat, smoking, carbohydrates, etc. Given the transitory nature of such consensus on positive matters it becomes unreliable for the risk domain as we can’t blow up the planet on something that is transitory. (We subscribe to scientific consensus about the climate principally because of the asymmetry, hence under a precautionary argument).

The death of Popperian Science We are witnessing the death of Popperian science owing to science journalists, with their focus on "evidence", not realizing that science doesn’t establish positive "ev- idence", but largely negative one. If scientific consensus were gospel, we would still be stuck in the dark ages (imagine someone using against Einstein the argument of "this goes against consensus"). Science journalists are reversing the logic of things, which is why we may be entering a new age of ignorance. Arguing for infallibility (opposite of Popperian fallibilism) brings us back to councils of cardinals and ayatollahs deciding on truth. Journalists make a fallacy of aggregation: science is not an aggregation of scientists, but the upper bound of constantly emerging insights. In fact it is the same as with markets, where going with consensus makes you poor, since markets are driven by a minority of winners. (Note the miracle of Popperianism is that statistical theory is fully compatible with Popperian falsification as we never really "accept", just "fail to reject").

From Popper to Fragility One final remark about risk. In the Incerto I took Popper one step further by moving from asymmetry of error to asymmetry of impact, namely fragility. I am showing that in some domains, fallibilism isn’t about truth but needs to be compounded by "impact" in the tail, when survival is at stake. In other words, left-tail risk analysis supersedes science and statistical science–if we were to use a standard scientific confidence interval of 95% band around plane safety, almost all frequent fliers would be extinct.

EXTREME RISK INITIATIVE —NYU SCHOOL OF ENGINEERING WORKING PAPER SERIES 2

RESPONSE TO SPECIFIC AUTHORS We are obligated to respond to scholarly comments about the first draft of our Precautionary Principle paper, provided these are valid and technical (even those from blogs).

The point is that a paper, particularly when in draft form, is only vulnerable to comments, both from good and bad faith, —i.e, does not improve from them — if there is a central flaw, not a missing footnote, a typo, a poorly phrased sentence, an incomplete argument, etc.

Trevor Charles Response to Trevor Charles is here.

Adam Merberg’s Errors in his discussion of the PP Merber’s comments.

Unfortunately, we find that the (blog) comments of Adam Merberg include a concentration of elementary mistakes The point is that he used amateur-sophist vagueness in his accusations of "mathiness", claiming that the math had nothing to do with the application, was not "precise" enough1, or "hides critical errors." But when he tried to be concrete, he made critical errors in every error he detected.

Evidence of Bad Faith Had Merberg found mistakes in our paper and contacted us, or asked for clarifications, one would have assumed good faith. But what Merberg did was start from the position that "this paper is bullshit ("mathiness"), and here are two mistakes to prove it" —and given that the mistakes were his own mistakes, he is looking now for more mistakes. It is based on that that I disagree with my co-authors who want a lengthy point by point "discussion" of what he wrote.

Error in interpretation of the subexponential class Merberg makes a mathematical mistake in saying that we made a mathematical mistake in our definition of exponential class: with time the sum gets larger, but with time the maximum is ALSO likely to be bigger. In other words, the distribution of the ratio is time invariant; what time does is reveal statistical properties.

We wondered how someone can make an elementary mistake, one that would flunk an undergrad.

Error in the discussion of fragility Merberg thought that fragility as we described it was "unit" dependent and claiming a detection of a flaw in our paper offered a false counterexample.

His error is as follows. For instance assume the dis- tribution of the "magnitude" of earthquakes follows a

1. You can always say math is not "precise" about any quant paper, and even for proofs in pure math, but it can be indicative of harassment by referee. The benchmark journals use outside of math is to reject anything that has "fluff", i.e. vapid formalism.

power law with f(x) = α(x + 1)−α−1 ,x ≥ 0. Now consider his transformation of the energy (as given in his example), y = h(x) = 32x (actually he may be using y = h(x) = 32x−1 but there is no difference in his error). Hence the distribution of y, g(y) It means the distribution of y is =

even α logα(32) more log(32y)−α−1

y

fat-tailed. .

Now consider φ the harm function. The proper way is to compare

φ(x)f(x)dx

to

φ(h−1(y))g(y)dy = φ(h−1(y))f(h−1(y))/h (h−1(y))dy

not conflate as Merberg did f(x) and g(x) (or f(x) and g(y)). (This is not abstract math. In practical terms, the coffee cup will more likely break upon a magnitude 6 event (or beyond) or, equivalently, an event of energy bigger than 632. And this will happen whatever the ratio between the frequency of earthquakes of magnitude 1 and those of magnitude ≥ 6.)

There are many other severe errors, at all levels, especially some embarrassing logical flaws.

Merberg’s replies with additional mistakes Merberg still convinced that he did nothing wrong2 went on a tirade about the sum of r.v.s X

1

,X

2

,...,X

n

where he is interested in the behavior of the unconditional sum not for tail events, the CDF F(x)=1 − (x + 1)−α. But he mysteriously selected x ≥ 1 not x ≥ 0 This x ≥ 1 in his exposition does not have a meaning since observations in the sum of between 0 and 1 are strangely and artificially excluded. So to compute the unconditional sum he is conditioning on a non-tail threshold... I am looking at these during Aramaic-Syriac summer school (in which I am not allowed to do probability) so I can’t get deep into what Merberg is trying to say to save face. So I wonder why he keeps nitpicking with errors?

Good and bad faith Also before he started his brand of academic charac- ter assassination via sophistry, Merberg failed to see at the end a section "APPENDIX C MATHEMATICAL DERIVATIONS OF FRAGILITY", where his questions

2. We now see the confusion as Adam Merberg took the sentence out of its context and divorced it of the previous three paragraphs and the mathematical derivations. It is clear our statement is "for large enough values of x" and it is not necessary to repeat the condition in every sentence.

Just to be clear about what we mean in context about the dominance of the max value for tail events: conditional on a large deviation, it is more likely to be attributable to a large single deviation than to a sum. The previous paragraph said so explicitly: "Thus, every time the sum exceeds x, for large enough values of x, the value of the sum is due to either one or the other exceeding x-the maximum over the two variables-and the other of them contributes negligibly." And the section explicitly declares that we are concerned with tail events only: "Our interest is in distinguishing between cases where tail events dominate impacts, as a formal definition of the boundary between the categories of distributions to be considered as Mediocristan and Extremistan."

EXTREME RISK INITIATIVE —NYU SCHOOL OF ENGINEERING WORKING PAPER SERIES 3

are (or could be) answered (and can be expected to be answered since that is the title of the section)–nor did he wonder why they are not answered there. A scholar in good faith waits to see what is in the referenced text (or asks for it) before starting his character assassination. And the text has been on the web for a year in the version on my site, removed from arXiv as we had too many appendices in text.

We wrote:

Our analysis is designed to characterize the response of a system to a distribution of envi- ronmental perturbations.

In order to describe the system response we assume a single dimensional measure of the structural integrity of the system, s. The dam-

age, dissolution or destruction of the system is measured by the deviation from a reference value, described by negative values of s of in- creasing magnitude. No assumption is made of the existence of corresponding positive values of improvement in structure, though such may exist.

We note that this mathematical framework enables physical system damage to be mapped onto a single dimension, as is done in market pricing of value, and thus we adopt for fragility the terminology, “vega,” of price sensitivity to uncertainty associated with derivatives con- tracts.