EA - Book summary: 'Why Intelligence Fails' by Robert Jervis by Ben Stewart

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Book summary: 'Why Intelligence Fails' by Robert Jervis, published by Ben Stewart on June 20, 2023 on The Effective Altruism Forum.Here’s a summary of ‘Why Intelligence Fails’ by the political scientist Robert Jervis. It’s a book analysing two cases where the U.S. intelligence community ‘failed’: being slow to foresee the 1979 Iranian Revolution, and the overconfident and false assessment that Saddam Hussein had weapons of mass destruction in 2003.I’m interested in summarising more books that contain valuable insights but are outside the typical EA canon. If you’d like more of this or have a book to suggest, let me know.Key takeawaysGood intelligence generally requires the relevant agency and country office to prioritise the topic and direct scarce resources to it. Good intelligence in a foreign country requires a dedicated diplomatic and covert collection corps with language skills and contextual knowledge.Intelligence analysis can be deficient in critical review, external expertise, and social-scientific methodology. Access to classified information only generates useful insight for some phenomena.Priors can be critical in determining interpretation within intelligence, and they can often go unchallenged.Political pressure can have a significant effect on analysis, but is hard to pin down.If the justification of an intelligence conclusion is unpublished, you can still interrogate it by asking:whether the topic would have been given sufficient priority and resources by the relevant intelligence organisationwhether classified information, if available, would be likely to yield insightwhether pre-existing beliefs are likely to bias analysiswhether political pressures could significantly affect analysisSome correctives to intelligence failures which may be useful to EA:demand sharp, explicit, and well-tracked predictionsdemand early warning indicators, and notice when beliefs can only be disproven at a late stageconsider negative indicators - 'dogs that don't bark', i.e. things that the view implies should not happenuse critical engagement by peers and external experts, especially by challenging fundamental beliefs that influence what seems plausible and provide alternative hypotheses and interpretationsuse red-teams, pre-mortems, and post-mortems.Overall, I’ve found the book to somewhat demystify intelligence analysis. You should contextualise a piece of analysis with respect to the psychology and resources involved, including whether classified information would be of significant benefit. I have become more sceptical of intelligence, but the methodology of focusing on two known failures - selecting on the dependent variable - mean that I hesitate to become too pessimistic about intelligence as a whole and as it functions today.Why it’s relevant to EAThe most direct application of this topic is to the improvement of institutional decision-making, but there is value for any cause area that depends on conducting or interpreting analysis of state and non-state adversaries, such as in biosecurity, nuclear war, or great power conflict.This topic may also contribute to the reader's sense of when and how much one should defer to the outputs of intelligence communities. Deference is motivated by their access to classified information and presumed analytic capability. However, Tetlock’s ‘Expert Political Judgment’ cast doubt on the value of classified information for improving prediction compared to generalist members of the public.Finally, assessments of the IC’s epistemic practices might offer lessons for how an intellectual community should grapple with information hazards, both intellectually and socially. More broadly, the IC is an example of a group pursuing complex, decision-relevant analysis in a high-uncertainty environment. Their successes and ...