EA - How can we improve Infohazard Governance in EA Biosecurity? by Nadia Montazeri

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How can we improve Infohazard Governance in EA Biosecurity?, published by Nadia Montazeri on August 5, 2023 on The Effective Altruism Forum.Or: "Why EA biosecurity epistemics are whack"The effective altruism (EA) biosecurity community focuses on reducing the risks associated with global biological catastrophes (GCBRs). This includes preparing for pandemics, improving global surveillance, and developing technologies to mitigate the risks of engineered pathogens. While the work of this community is important, there are significant challenges to developing good epistemics, or practices for acquiring and evaluating knowledge, in this area.One major challenge is the issue of infohazards. Infohazards are ideas or information that, if widely disseminated, could cause harm. In the context of biosecurity, this could mean that knowledge of specific pathogens or their capabilities could be used to create bioweapons. As a result, members of the EA biosecurity community are often cautious about sharing information, particularly in online forums where it could be easily disseminated.The issue of infohazards is not straightforward. Even senior biosecurity professionals may have different thresholds for what they consider to be an infohazard. This lack of consensus can make it difficult for junior members to learn what is appropriate to share and discuss. Furthermore, it can be challenging for senior members to provide feedback on the appropriateness of specific information without risking further harm if that information is disseminated to a wider audience. At the moment, all EA biosecurity community-building efforts are essentially gate-kept by Open Phil, whose staff are particularly cautious about infohazards, even compared to experts in the field at the Center for Health Security. Open Phil staff time is chronically scarce, making it impossible to copy and critique their heuristics on infohazards, threat models, and big-picture biosecurity strategy from 1:1 conversations.Challenges for cause and intervention prioritisationThese challenges can lead to a lack of good epistemics within the EA biosecurity community, as well as a deference culture where junior members defer to senior members without fully understanding the reasoning behind their decisions. This can result in a failure to adequately assess the risks associated with GCBRs and make well-informed decisions.The lack of open discourse on biosecurity risks in the EA community is particularly concerning when compared to the thriving online discourse on AI alignment, another core area of longtermism for the EA movement. While there are legitimate reasons for being cautious about sharing information related to biosecurity, this caution may lead to a lack of knowledge sharing and limited opportunities for junior members of the community to learn from experienced members.In the words of a biosecurity researcher who commented on this draft:"Because of this lack of this discussion, it seems that some junior biosecurity EAs fixate on the "gospel of EA biosecurity interventions" - the small number of ideas seen as approved, good, and safe to think about. These ideas seem to take up most of the mind space for many junior folks thinking about what to do in biosecurity. I've been asked "So, you're working in biosecurity, are you going to do PPE or UVC?" one too many times. There are many other interesting defence-dominant interventions, and I get the sense that even some experienced folks are reluctant to explore this landscape."Another example is the difficulty of comparing biorisk and AI risk without engaging in potentially infohazardous concrete threat models. While both are considered core cause areas of longtermism, it is challenging to determine how to prioritise these risks without evaluating the likelihood of a catast...