EA - What's in a Pause? by Davidmanheim

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What's in a Pause?, published by Davidmanheim on September 17, 2023 on The Effective Altruism Forum.This post is part of AI Pause Debate Week. Please see this sequence for other posts in the debate.An AI Moratorium of some sort has been discussed, but details matter - it's not particularly meaningful to agree or disagree with a policy that has no details. A discussion requires concrete claims.To start, I see three key questions, namely:What does a moratorium include?When and how would a pause work?What are the concrete steps forward?Before answering those, I want to provide a very short introduction and propose what is in or out of bounds for a discussion.There seems to be a strong consensus that future artificial intelligence could be very bad. There is quite a significant uncertainty and dispute about many of the details - how bad it could be, and when the different risks materialize. Pausing or stopping AI progress is anywhere from completely unreasonable to obviously necessary, depending on those risks, and the difficulty of avoiding them - but eliminating those uncertainties is a different discussion, and for now, I think we should agree to take the disputes and uncertainties about the risks as a given. We will need to debate and make decisions under uncertainty. So the question of whether to stop and how to do so depends on the details of the proposal - but these seem absent from most of the discussion. For that reason, I want to lay out a few of the places where I think these need clarification, including not just what a moratorium would include and exclude, but also concrete next steps to getting there.Getting to a final proposal means facing a few uncomfortable policy constraints that I'd also like to suggest be agreed on for this discussion. An immediate, temporary pause isn't currently possible to monitor, much less enforce, even if it were likely that some or most parties would agree. Similarly, a single company or country announcing a unilateral halt to building advanced models is not credible without assurances, and is likely both ineffective at addressing the broader race dynamics, and differentially advantages the least responsible actors. For these reasons, the type of moratorium I think worth discussing is a multilateral agreement centered on countries and international corporations, one which addresses both current and unclear future risks. But, as I will conclude, much needs to happen more rapidly than that - international oversight should not be an excuse for inaction.What Does a Moratorium Include?There is at least widespread agreement on many things that aren't and wouldn't be included. Current systems aren't going to be withdrawn - any ban would be targeted to systems more dangerous than those that exist. We're not talking about banning academic research using current models, and no ban would stop research to make future systems safer, assuming that the research itself does not involve building dangerous systems. Similarly, systems that reasonably demonstrate that they are low risk would be allowed, though how that safety is shown is unclear.Next, there are certain parts of the proposal that are contentious, but not all of it. Most critics of a moratorium agree that we should not and can't afford to build dangerous systems - they simply disagree where the line belongs. Should we allow arbitrary plugins? Should we ban open-sourcing models? When do we need to stop? The answers are debated. And while these all seem worrying to me, the debate makes sense - there are many irreducible uncertainties, we have a global community with differing views, and actual diplomatic solutions will require people who disagree to come to some agreement.As should be clear from my views on the need to negotiate answers, I'm not planning to dictate exactl...