EA - New career review: AI safety technical research by Benjamin Hilton

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New career review: AI safety technical research, published by Benjamin Hilton on July 18, 2023 on The Effective Altruism Forum.Note: this post is a (minorly) edited version of a new 80,000 Hours career review.Progress in AI - while it could be hugely beneficial - comes with significant risks. Risks that we've argued could be existential.But these risks can be tackled.With further progress in AI safety, we have an opportunity to develop AI for good: systems that are safe, ethical, and beneficial for everyone.This article explains how you can help.SummaryArtificial intelligence will have transformative effects on society over the coming decades, and could bring huge benefits - but we also think there's a substantial risk. One promising way to reduce the chances of an AI-related catastrophe is to find technical solutions that could allow us to prevent AI systems from carrying out dangerous behaviour.ProsOpportunity to make a significant contribution to a hugely important area of researchIntellectually challenging and interesting workThe area has a strong need for skilled researchers and engineers, and is highly neglected overallConsDue to a shortage of managers, it's difficult to get jobs and might take you some time to build the required career capital and expertiseYou need a strong quantitative backgroundIt might be very difficult to find solutionsThere's a real risk of doing harmKey facts on fitYou'll need a quantitative background and should probably enjoy programming. If you've never tried programming, you may be a good fit if you can break problems down into logical parts, generate and test hypotheses, possess a willingness to try out many different solutions, and have high attention to detail.If you already:Are a strong software engineer, you could apply for empirical research contributor roles right now (even if you don't have a machine learning background, although that helps)Could get into a top 10 machine learning PhD, that would put you on track to become a research leadHave a very strong maths or theoretical computer science background, you'll probably be a good fit for theoretical alignment researchRecommendedIf you are well suited to this career, it may be the best way for you to have a social impact.Thanks to Adam Gleave, Jacob Hilton and Rohin Shah for reviewing this article. And thanks to Charlie Rogers-Smith for his help, and his article on the topic - How to pursue a career in technical AI alignment.Why AI safety technical research is high impactAs we've argued, in the next few decades, we might see the development of hugely powerful machine learning systems with the potential to transform society. This transformation could bring huge benefits - but only if we avoid the risks.We think that the worst-case risks from AI systems arise in large part because AI systems could be misaligned - that is, they will aim to do things that we don't want them to do. In particular, we think they could be misaligned in such a way that they develop (and execute) plans that pose risks to humanity's ability to influence the world, even when we don't want that influence to be lost.We think this means that these future systems pose an existential threat to civilisation.Even if we find a way to avoid this power-seeking behaviour, there are still substantial risks - such as misuse by governments or other actors - which could be existential threats in themselves.There are many ways in which we could go about reducing the risks that these systems might pose. But one of the most promising may be researching technical solutions that prevent unwanted behaviour - including misaligned behaviour - from AI systems. (Finding a technical way to prevent misalignment in particular is known as the alignment problem.)In the past few years, we've seen more o...