EA - Policy ideas for mitigating AI risk by Thomas Larsen

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Policy ideas for mitigating AI risk, published by Thomas Larsen on September 16, 2023 on The Effective Altruism Forum.Note: This post contains personal opinions that don't necessarily match the views of others at CAIP.Executive SummaryAdvanced AI has the potential to cause an existential catastrophe. In this essay, I outline some policy ideas which could help mitigate this risk. Importantly, even though I focus on catastrophic risk here, there are many other reasons to ensure responsible AI development.I am not advocating for a pause right now. If we had a pause, I think it would only be useful insofar as we use the pause to implement governance structures that mitigate risk after the pause has ended.This essay outlines the important elements I think a good governance structure would include: visibility into AI development, and brakes that the government could use to stop dangerous AIs from being built.First, I'll summarize some claims about the strategic landscape. Then, I'll present a cursory overview of proposals I like for US domestic AI regulation. Finally, I'll talk about a potential future global coordination framework, and the relationship between this and a pause.The Strategic LandscapeClaim 1: There's a significant chance that AI aligment is difficult.There is no scientific consensus on the difficulty of AI alignment. Chris Olah from Anthropic tweeted the following, simplified picture:~40% of their estimate is on AI safety being harder than Apollo, which took around 1 million person-years. Given that less than a thousand people are working on AI safety, this viewpoint would seem to imply that there's a significant chance that we are far from being ready to build powerful AI safely.Given just Anthropic's alleged views, I think it makes sense to be ready to stop AI development. My personal views are more pessimistic than Anthropic's.Claim 2: In the absence of powerful aligned AIs, we need to prevent catastrophe-capable AI systems from being built.Given developers are not on track to align AI before it becomes catastrophically dangerous, we need the ability to slow down or stop before AI is catastrophically dangerous.There are several ways to do this.I think the best one involves building up the government's capacity to safeguard AI development. Set up government mechanisms to monitor and mitigate catastrophic AI risk, and empower them to institute a national moratorium on advancing AI if it gets too dangerous. (Eventually, the government could transition this into an international moratorium, while coordinating internationally to solve AI safety before that moratorium becomes infeasible to maintain. I describe this later.)Some others think it's better to try to build aligned AIs that defend against AI catastrophes. For example, you can imagine building defensive AIs that identify and stop emerging rogue AIs. To me, the main problem with this plan is that it assumes we will have the ability to align the defensive AI systems.Claim 3: There's a significant (>20%) chance AI will be capable enough to cause catastrophe by 2030.AI timelines have been discussed thoroughly elsewhere, so I'll only briefly note a few pieces of evidence for this claim I find compelling:Current trends in AI. Qualitatively, I think another jump of the size from GPT-2 to GPT-4 could get us to catastrophe-capable AI systems.Effective compute arguments, such as Ajeya Cotra's Bioanchors report. Hardware scaling, continued algorithmic improvement, investment hype are all continuing strongly, leading to a 10x/year increase of effective compute used to train the best AI system. Given the current rates of progress, I expect another factor of a million increase in effective compute by 2030.Some experts think powerful AI is coming soon, both inside and outside of frontier labs. ...