EA - Karma overrates some topics; resulting issues and potential solutions by Lizka

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Karma overrates some topics; resulting issues and potential solutions, published by Lizka on January 30, 2023 on The Effective Altruism Forum.TL;DR: Karma overrates “lowest-common-denominator” posts that interest a large fraction of the community, leading to some issues. We list some potential solutions at the bottom.Please see the disclaimer at the bottom of the post.Posts that interest everyone — or discussions where everyone has an opinion — tend to get a lot more Forum karma (and attention) than niche posts.These posts tend to beabout the EA communityaccessible to everyone, or on topics where everyone has an opinionWhy does this happen?There are different groups with different niche interests, but an overlapping interest in the EA community:When a post about the EA community is published, many people might have opinions, and many people feel that they can vote on the post. Most people upvote, so more people voting usually means that a post will get higher karma.Similarly, if the topic of the post is something that doesn’t require particular expertise to have an opinion about, lots of people feel like they can weigh in. You can think of these as “lowest-common-denominator posts.” This is related to bike-shedding.This leads to some issuesThis misleads people about what the Forum — and the EA community — cares about10 of the 10 highest karma posts from 2022 were community posts, even though less than ⅓ of total karma went to community posts.When someone is trying to evaluate the quality of the Forum, they often go to the list of top posts and evaluate those. This seems like a very reasonable thing to do, but it's actually giving a very skewed picture of what happens on the Forum.Because discussions about the community seem to be so highly valued by Forum readers, people might accidentally start to value community-oriented topics more themselves, and drift away from real-world issuesImagine an author posting about some issue with RCTs that’s relevant to their work — they’ll get a bit of engagement, some appreciation, and maybe some questions. Then they write a quick post about the font on the Forum — suddenly everyone has an opinion and they get loads of karma. Unconsciously, they might view this as an indicator that the community values the second post more than the first. If this happens repeatedly or they see this happening, they might shift towards that view themselves if they defer even a bit to the community’s view.Now imagine this happening on the scale of the thousands of people who use the Forum; these small updates add up.This directs even more attention to community-oriented, low-barrier topics, and away from niche topics and topics that are more complex, which might be more valuable to discussKarma is used for sorting the Frontpage: higher-rated posts stay on the Frontpage for longer. This is useful, as it tends to hide the most irrelevant posts, and generally boosts higher quality content — more people see the better posts.But because posts that hit the middle sections in the Venn diagrams above get more karma, they tend to stick around for longer, which then gets them more karma, etc.(We didn't try to make this list of issues as exhaustive as possible.)Note that karma is not perfect even within a much more specific topic — pretty random factors can affect a Forum post’s karma, and readers aren’t always great at voting, but that is a separate issue. (We might write a post about it later.)Solutions we’re considering or exploringCreate something like a subforum or separate tab for “community opinion” posts, and filter them out from the Frontpage by defaultOr otherwise move in this directionRename “Top” sorting to more clearly indicate what karma actually measuresWe tend to have a somewhat higher bar for sharing “community” posts ...