EA - Anonymous advice: If you want to reduce AI risk, should you take roles that advance AI capabilities? by Benjamin Hilton

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Anonymous advice: If you want to reduce AI risk, should you take roles that advance AI capabilities?, published by Benjamin Hilton on October 11, 2022 on The Effective Altruism Forum. This is a cross-post of an article from the 80,000 Hours website with only minor changes for this audience. See the original here.We’ve argued that preventing an AI-related catastrophe may be the world’s most pressing problem, and that while progress in AI over the next few decades could have enormous benefits, it could also pose severe, possibly existential risks. As a result, we think that working on some technical AI research — research related to AI safety — may be a particularly high-impact career path. But there are many ways of approaching this path that involve researching or otherwise advancing AI capabilities — meaning making AI systems better at some specific skills — rather than only doing things that are purely in the domain of safety. In short, this is because: Capabilities work and some forms of safety work are intertwined. Many available ways of learning enough about AI to contribute to safety are via capabilities-enhancing roles. So if you want to help prevent an AI-related catastrophe, should you be open to roles that also advance AI capabilities, or steer clear of them? We think this is a hard question! Capabilities-enhancing roles could be beneficial or harmful. For any role, there are a range of considerations — and reasonable people disagree on whether, and in what cases, the risks outweigh the benefits. So we asked the 22 people we thought would be most informed about this issue — and who we knew had a range of views — to write a summary of their takes on the question. We received 10 really interesting responses, and think that these are likely a reasonable representation of the range of views held by the broader set of people. We hope that these responses will help inform people making difficult decisions about working in roles that might advance AI capabilities. The following are written by people whose work we respect and who would like to remain anonymous. These quotes don’t represent the views of 80,000 Hours, and in some cases, individual pieces of advice may explicitly contradict our own. Nonetheless, we think it’s valuable to showcase the range of views on difficult topics where reasonable people might disagree. We’ve included the responses from these 10 experts in full. We’ve only made minor edits for clarity and ease of reading. Advice from 10 anonymous experts Expert 1 Right now I wish AI labs would slow down on the margin, but I don’t think it’s obvious that capabilities work is net negative and reasonable people can disagree on this point. What I say below would probably change if I were highly confident it’s extremely bad to advance capabilities (especially if I believed that advancing capabilities by a month is much more bad than advancing alignment research by a month is good). With that said: AI labs are a great place to pick up good skills, especially for people doing technical roles (ML engineer, ML researcher) at those labs. If you’re early career and can get into one and think it’d be more interesting and a better personal fit for you than other roles you’re considering, you should probably go for it — on a pretty wide distribution of views (including those that think capabilities enhancement is pretty net negative), the investment in your human capital probably creates more good than your contribution to capabilities (at a junior level) creates harm. It’s worth specifically asking about working on the safety teams of capabilities organizations and not assuming you have to choose between a pure capabilities role and no role; this won’t always work but I’d expect a reasonable fraction of the time you can tilt your role toward more safety projec...