EA - Mistakes in the moral mathematics of existential risk (Part 2: Ignoring background risk) - Reflective altruism by BrownHairedEevee

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Mistakes in the moral mathematics of existential risk (Part 2: Ignoring background risk) - Reflective altruism, published by BrownHairedEevee on July 3, 2023 on The Effective Altruism Forum.This is the second part of "Mistakes in the moral mathematics of existential risk", a series of blog posts by David Thorstad that aims to identify ways in which estimates of the value of reducing existential risk have been inflated. I've made this linkpost part of a sequence.In the decades to come, advanced bioweapons could threaten human existence. Although the probability of human extinction from bioweapons may be low, the expected value of reducing the risk could still be large, since such risks jeopardize the existence of all future generations. We provide an overview of biotechnological extinction risk, make some rough initial estimates for how severe the risks might be, and compare the cost-effectiveness of reducing these extinction-level risks with existing biosecurity work. We find that reducing human extinction risk can be more cost-effective than reducing smaller-scale risks, even when using conservative estimates. This suggests that the risks are not low enough to ignore and that more ought to be done to prevent the worst-case scenarios.Millett and Snyder-Beattie, “Existential risk and cost-effective biosecurity”1. IntroductionThis is Part 2 of a series based on my paper “Mistakes in the moral mathematics of existential risk”.Part 1 introduced the series and discussed the first mistake: focusing on cumulative rather than per-unit risk. We saw how bringing the focus back to per-unit rather than cumulative risk was enough to change a claimed 'small’ risk reduction of one millionth of one percent into an astronomically large reduction that would drive risk to almost one in a million per century.Today, I want to focus on a second mistake: ignoring background risk. The importance of modeling background risk is one way to interpret the main lesson of my paper and blog series “Existential risk pessimism and the time of perils.” Indeed, in Part 7 of that series, I suggested just this interpretation.It turns out that blogging is sometimes a good way to write a paper. Today, I want to expand my discussion in Part 7 of the existential risk pessimism series to clarify the second mistake (ignoring background risk) and to show how it interacts with the first mistake (focusing on cumulative risk) in a leading discussion of cost-effective biosecurity.Some elements of this discussion are lifted verbatim from my earlier post. In my defense, I remind my readers that I am lazy.2. Snyder-Beattie and Millett on cost-effective biosecurityAndrew Snyder-Beattie holds a DPhil in Zoology from Oxford, and works as a Senior Program Officer at Open Philanthropy. Snyder-Beattie is widely considered to be among the very most influential voices on biosecurity within the effective altruist community.Piers Millett is a Senior Research Fellow at the Future of Humanity Institute. Millett holds advanced degrees in science policy, research methodology and international security, and has extensive industry experience in biosecurity.Millett and Snyder-Beattie’s paper, “Existential risk and cost-effective biosecurity”, is among the most-cited papers on biosecurity written by effective altruists. The paper argues that even very small reductions in existential risks in the biosecurity sector (henceforth, 'biorisks’) are cost-effective by standard metrics.Millett and Snyder-Beattie estimate the cost-effectiveness of an intervention as C/(NLR), where:C is the cost of the intervention.N is “the number of biothreats we expect to occur in 1 century”.L is “the number of life-years lost in such an event”.R is “the reduction in risk [in this century only] achieved by spending . C”.Millett and Snyder Be...