EA - AI Incident Sharing - Best practices from other fields and a comprehensive list of existing platforms by stepanlos

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Incident Sharing - Best practices from other fields and a comprehensive list of existing platforms, published by stepanlos on June 29, 2023 on The Effective Altruism Forum.Purpose of this post: The purpose of this post is three-fold: 1) highlight the importance of incident sharing and share best practices from adjacent fields to AI safety 2) collect tentative and existing ideas of implementing a widely used AI incident database and 3) serve as a comprehensive list of existing AI incident databases as of June 2023.Epistemic status: I have spent around 25+ hours researching this topic and this list is by no means meant to be exhaustive. It should give the reader an idea of relevant adjacent fields where incident databases are common practice and should highlight some of the more widely used AI incident databases which exist to date. Please feel encouraged to comment any relevant ideas or databases that I have missed, I will periodically update the list if I find anything new.Motivation for AI Incident DatabasesSharing incidents, near misses and best practices in AI development decreases the likelihood of future malfunctions and large-scale risk. To mitigate risks from AI systems, it is vital to understand the causes and effects of their failures. Many AI governance organizations, including FLI and CSET, recommend creating a detailed database of AI incidents to enable information-sharing between developers, government and the public. Generally, information-sharing between different stakeholders 1) enables quicker identification of security issues and 2) boosts risk-mitigation by helping companies take appropriate actions against vulnerabilities.Best practices from other fieldsNational Transportation Safety Board (NTSB) publishes and maintains a database of aviation accidents, including detailed reports evaluating technological and environmental factors as well as potential human errors causing the incident. The reports include descriptions of the aircraft, how it was operated by the flight crew, environmental conditions, consequences of event, probable cause of accident, etc. The meticulous record-keeping and best-practices recommendations are one of the key factors behind the steady decline in yearly aviation accidents, making air travel one of the safest form of travel.National Highway Traffic Safety Administration (NHTSA) maintains a comprehensive database recording the number of crashes and fatal injuries caused by automobile and motor vehicle traffic, detailing information about the incidents such as specific driver behavior, atmospheric conditions, light conditions or road-type. NHTSA also enforces safety standards for manufacturing and deploying vehicle parts and equipment.Common Vulnerabilities and Exposure (CVE) is a cross-sector public database recording specific vulnerabilities and exposures in information-security systems, maintained by Mitre Corporation. If a vulnerability is reported, it is examined by a CVE Numbering Authority (CNA) and entered into the database with a description and the identification of the information-security system and all its versions that it applies to.Information Sharing and Analysis Centers (ISAC). ISACs are entities established by important stakeholders in critical infrastructure sectors which are responsible for collecting and sharing: 1) actionable information about physical and cyber threats 2) sharing best threat-mitigation practices. ISACs have 24/7 threat warning and incident reporting services, providing relevant and prompt information to actors in various sectors including automotive, chemical, gas utility or healthcare.National Council of Information Sharing and Analysis Centers (NCI) is a cross-sector forum designated for sharing and integrating information among sector-based ISACs (Information Sharing an...