EA - Can we trust wellbeing surveys? A pilot study of comparability, linearity, and neutrality by Conrad S

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Can we trust wellbeing surveys? A pilot study of comparability, linearity, and neutrality, published by Conrad S on March 17, 2023 on The Effective Altruism Forum.Note: This post only contains Sections 1 and 2 of the report. For the full detail of our survey and pilot results, please see the full report on our website.SummarySubjective wellbeing (SWB) data, such as answers to life satisfaction questions, are important for decision-making by philanthropists and governments. Such data are currently used with two important assumptions:Reports are comparable between persons (e.g., my 6/10 means the same as your 6/10)Reports are linear in the underlying feelings (e.g., going from 4/10 to 5/10 represents the same size change as going from 8/10 to 9/10).Fortunately, these two assumptions are sufficient for analyses that only involve the quality of people’s lives. However, if we want to perform analyses that involve trade-offs between improving quality and quantity of life, we also need knowledge of the neutral point, the point on a wellbeing scale that is equivalent to non-existence.Unfortunately, evidence on all three questions is critically scarce. We propose to collect additional surveys to fill this gap.Our aim with this report is two-fold. First, we give an outline of the questions we plan to field and the underlying reasoning that led to them. Second, we present results from an initial pilot study (n = 128):Unfortunately, this small sample size does not allow us to provide clear estimates of the comparability of wellbeing reports.However, across several question modalities, we do find tentative evidence in favour of approximate linearity.With respect to neutrality, we assess at what point on a 0-10 scale respondents say that they are 'neither satisfied nor dissatisfied' (mean response is 5.3/10). We also probe at what point on a life satisfaction scale respondents report to be indifferent between being alive and being dead (mean response is 1.3/10). Implications and limitations of these findings concerning neutrality are discussed in Section 6.2.In general, the findings from our pilot study should only be seen as being indicative of the general feasibility of this project. They do not provide definitive answers.In the hopes of fielding an improved version of our survey with a much larger sample and a pre-registered analysis plan, we welcome feedback and suggestions on our current survey design.Here are some key questions that we hope to receive feedback on:Are there missing questions that could be included in this survey (or an additional survey) that would inform important topics in SWB research? Are there any questions or proposed analyses you find redundant?Do you see any critical flaws in the analyses we propose? Are there additional analyses we should be considering?Would these data and analyses actually reassure you about the comparability, linearity, and neutrality of subjective wellbeing data? If not, what sorts of data and analyses would reassure you?What are some good places for us to look for funding for this research?Of course, any other feedback that goes beyond these questions is welcome, too. Feedback can be sent to [email protected] or to [email protected] report proceeds as follows:In Section 1, we describe the challenges for the use of self-reported subjective wellbeing data, focusing on the issues of comparability, linearity, and neutrality. We highlight the implications of these three assumptions for decision-making about effective interventions.In Section 2, we describe the general methodology of the survey.For the following sections, see the full report on our website.In Section 3, we discuss responses to the core life satisfaction question.In Sections 4, 5, and 6, we describe how we will assess co...