EA - Hiring Retrospective: ERA Fellowship 2023 by Oscar Delaney

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Hiring Retrospective: ERA Fellowship 2023, published by Oscar Delaney on August 5, 2023 on The Effective Altruism Forum.SummaryWe hired 31 people for our current Summer Research Fellowship, out of 631 applicationsThe applications were of impressive quality, so we hired more people than expectedWe think we made several mistakes, and should:Communicate more clearly what level of seniority/experience we want in applicantsHave a slightly shorter initial application, but be upfront that it is a long formAssign one person to evaluate all applicants for one particular question, rather than marking all questions on one application togetherHiring round processesThe Existential Risk Alliance (ERA) is a non-profit project equipping young researchers with the skills and knowledge needed to tackle existential risks. We achieve this by running a Summer Research Fellowship program where participants do an independent research project, supported by an ERA research manager, and an external subject-matter expert mentor.Promotion and OutreachWe tried quite hard to promote the Fellowship. Personal connections and various EA community sources were the most common referral source for applications:Initial applicationOur initial application form consisted of submitting a CV (which we put minimal weight on) and answering various open-ended questions. Some questions on motivation, reasoning ability, and previous experience were the same across all cause areas, and we also asked some cause area-specific subject matter questions. Our application form was open for 22 days, and we received a total of 631 applications from 556 unique applicants (some people applied to multiple cause areas). There was significant variation in the number of applications in each cause area: AI Gov = 167, AI Tech = 127, Climate = 100, Biosecurity = 96, Misc & Meta = 86, Nuclear = 51.People tended to apply late in the application period, with more than half of applications arriving within three days of the deadline.The majority of applicants were male:And the UK and US were by far the most common countries of residence for applicants:InterviewsAfter assessing the written applications, we invited 80 applicants (13%) for an interview as the second part of the recruitment process. Interviews were conducted by the research manager of the cause area the person was applying to, and lasted around thirty minutes. We used structured interviews, where each cause area had a standard list of questions that all interviewees were asked, to try to maximize comparability between applicants and improve fairness. Interview questions sought to gauge people's cause-area-specific knowledge, and ability to reason clearly responding to unseen questions. Even though only the people who did best on the initial application were invited to interviews, there was some positive correlation between initial application and interview scores. If this correlation was very strong, that would be some reason not to do an interview at all and just select people based on their application.This was not the case: the interview changed our ordering of applicants considerably.Composition of the final cohortWe had initially projected to fill approximately 20 fellowship spots, with an expected 3-4 candidates per cause area. We aimed to interview at least three times the number of candidates as the positions we planned to offer, to improve our chances of selecting optimal candidates.Because of the large number of excellent applications, we decided to open our fellowship to remote candidates that we couldn't host otherwise. Within cause areas, we selected candidates roughly based on a weighted average of their initial application, and interview (with a 5:3 ratio of weights for application:interview, given the application was far longer than the...