EA - Using Points to Rate Different Kinds of Evidence by Ozzie Gooen

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Using Points to Rate Different Kinds of Evidence, published by Ozzie Gooen on August 26, 2023 on The Effective Altruism Forum.Epistemic Status: Briefly written. The specific equation here captures my quick intuition - this is meant primarily as a demonstration.There's a lot of discussion on the EA Forum and LessWrong about epistemics, evidence, and updating.I don't know of many attempts at formalizing our thinking here into concrete tables or equations. Here is one (very rough and simplistic) attempt. I'd be excited to see much better versions.EquationInitial PointsScientific Evidence20 - A simple math proof proves X8 - A published scientific study in Economics supporting X6 - A published scientific study in Psychology supporting XMarket Prediction14 - Popular stock markets strongly suggest X11 - Prediction markets claim X, with 20 equivalent hours of research10 - A poll shows that 90% of LessWrong believe X6 - Prediction markets claim X, with one equivalent hour of researchExpert Opinion8 - An esteemed academic believes X, where it's directly in their line of work6 - The author has strong emotions about XReasoning6 - There's a (20-100 node) numeric model that shows X5 - A reasonable analogy between X and something clearly good/bad4 - A long-standing proverbPersonal Accounts5 - The author claims a long personal history that demonstrates X3 - Someone in the world has strong emotions about X2 - A clever remark, meme, or tweet2.3 - An insanely clever, meme, or tweet0 - Believing X is claimed to be personally beneficialTradition / Use12 - Top businesses act as if X8 - A long-standing social tradition about X5 - A single statistic about XPoint ModifiersIs this similar to existing evidence?Subtract the similarity from the extra amount of evidence. This likely will remove most of the evidence value.Is it convenient for the source to believe or say X?-10% to -90%Is there a lot of money or effort put behind spreading this evidence? For example, as an advertising campaign? +5% to +40%How credible is the author or source?-100% to +30%Do we suspect the source is goodharting on this scale?-20%Points, In PracticeEvidence Points, as outlined, are not trying to mimic mathematical bits of information or another clean existing unit. I attempted to find a compromise between accuracy and ease of use.MetaUsing an Equations for DiscussionThe equation above is rough, but at least it's (somewhat) precise and upfront. This represents much information, and any part can easily be argued against.I think such explicitness could help with epistemic conversations.Compare:"Smart people should generally use their inside view, instead of the outside view" vs. "My recommended points scores for inside-view style evidence, and my point scores for outside-view style evidence, are all listed below.""Using many arguments is better than one big argument" vs. "I've adjusted my point table function to produce higher values when multiple types of evidence are provided. You can see it returns values 30% higher than what others have provided for certain scenarios.""It's really important to respect top [intellectuals|scientists|EAs]" vs. "My score for respected [intellectuals|scientists|EAs] is 2 points higher than the current accepted average.""Chesterton's Fence is something to pay a lot of attention to" vs. "See my score table the points from various kinds of traditional practices."In a better world, different academic schools of thought could have their own neatly listed tables/functions. In an even better world, all of these functions would be forecasts of future evaluators.PresumptionsThis sort of point system makes some presumptions that might be worth flagging. Primarily, it claims that even really poor evidence is evidence.I often see people throwing ou...