Law of No Evidence

Reminder: Any claim that there is "no evidence" of something is evidence of bullshit.

This is a reference post for the Law of No Evidence.

Scott Alexander did us all a public service this week with his post The Phrase “No Evidence” Is a Red Flag for Bad Science Communication. If you have not yet read it I recommend doing so, and it is an excellent link to have handy going forward, and especially to have handy when going through studies about the severity of Omicron.

As useful as it is, he is being too kind. Not only is this ‘bad science communication’ it is also ‘not how this works, not how any of this works’ where ‘this’ is knowledge or actual science (as opposed to the brand the scientism of Science(TM)) and most importantly it is also evidence of bullshit, as per my proposed Law of No Evidence:

Law of No Evidence: Any claim that there is “no evidence” of something is evidence of bullshit.

No evidence should be fully up there with “government denial” or “I didn’t do it, no one saw me do it, there’s no way they can prove anything.” If there was indeed no evidence, there’d be no need to claim there was no evidence. This is usually a move to categorize the evidence as illegitimate and irrelevant because it doesn’t fit today’s preferred form of scientism.

The context that led to the law’s formulation was people saying there was “no evidence” that the suspension of the J&J vaccine led to increased vaccine hesitancy, which was over-the-top levels of obvious nonsense, as I was constantly dealing with people’s concerns about that, and there was a huge dip on the vaccination chart at exactly the right time.

The context now is that there have been a lot of assessments that there is ‘no evidence’ that Omicron is less severe than Delta, often based on a particular data point not providing such evidence, which is then often flipped around to be a claim that Omicron definitely isn’t less severe than Delta, and that everyone speculating otherwise is irresponsible. Which is obvious nonsense, we clearly have plenty of evidence in lots of different directions and the whole thing is complicated and difficult and it will be a while before we can draw definite conclusions either way.

Saying there is ‘no evidence’ of something isn’t not lazy or bad science reporting (or other talk). It is definitely both of those, but that is not what it centrally is. No evidence is a magic phrase used to intentionally manipulate understanding by using a motte and bailey between ‘this is false’ and statements of the form ‘this has not been proven using properly peer reviewed randomized controlled trials with p less than 0.05.’ It makes one sound Responsible and Scientific in contrast to those who update their beliefs based on the information they acquire, no matter the source.

It purports to treat evidence the way it would be treated in a court of criminal law, where only some facts are ‘admissible’ and the defendant is to be considered innocent until proven guilty using only those facts. Other facts don’t count. In some cases, we even throw out things we know because those who discovered the facts in question were bad actors, and the information is ‘fruit of the poisoned tree.’ This is all a highly reasonable procedure when one is worried about the state attempting to imprison citizens and abusing its powers to scapegoat people, either by mistake or intentionally, and you would rather ten guilty men go free than put one innocent man in prison. In that context, when deciding whether to deny someone their freedom, I strongly feel we should keep using it.

Yet the detective often knows who did it long before they have enough formal evidence for an arrest, and should act accordingly, because they are a person who is allowed to know things and use Bayes Rule. And if the court finds the defendant not guilty, but you know things the court didn’t know, that doesn’t mean that your knowledge vanishes.

In the context of deciding how to handle a pandemic under uncertainty, or trying to model the world in the course of everyday life to make decisions, using the standards and sets of procedures of a criminal court is obvious nonsense. That goes double given those with contextual power get to choose who counts as the prosecution and who counts as the defendant, so whatever statement they dislike today requires this level of proof, and whatever they feel like asserting today is the default.

This is not an ‘honest’ mistake. This is a systematic anti-epistemic superweapon engineered to control what people are allowed and not allowed to think based on social power, in direct opposition to any and all attempts to actually understand and model the world and know things based on one’s information. Anyone wielding it should be treated accordingly.

Scott’s post eventually does point out that ‘no evidence’ is not how any of this ‘figure things out’ thing works. After pointing out how horrible and misleading it is that we say both “there is no evidence 450,000 people died of vaccine complications” (yes, the original said no evidence of 45,000 deaths, which is also true the way they are using the phrase, but I added another zero to be illustrative, because if the claim about 45,000 deaths is true than so is my claim! There’s even more no evidence for that!) and also “there is no evidence parachute use prevents death when falling from planes” Scott gets to the real issue here, which is that knowledge is Bayesian.

I challenge anyone to come up with a definition of "no evidence" that wouldn't be misleading in at least one of the above examples. If you can't do it, I think that's because the folk concept of "no evidence" doesn't match how real truth-seeking works. Real truth-seeking is Bayesian. You start with a prior for how unlikely something is. Then you update the prior as you gather evidence. If you gather a lot of strong evidence, maybe you update the prior to somewhere very far away from where you started, like that some really implausible thing is nevertheless true. Or that some dogma you held unquestioningly is in fact false. If you gather only a little evidence, you mostly stay where you started.

I'm not saying this process is easy or even that I'm very good at it. I'm just saying that once you understand the process, it no longer makes sense to say "no evidence" as a synonym for “false”.

I would once again go much farther, on multiple fronts.

I’d say that it never ‘made sense’ to use ‘no evidence’ as a synonym for ‘false’ and that this is not a word choice that is made in good faith. If someone uses ‘no evidence’ as a synonym for false, as opposed to a synonym for ‘you’re not allowed to claim that’ then this is not merely evidence of bullshit. It is intentionally and knowingly ‘saying that which is not.’ It is evidence of enemy action.

I’d also assert that Scott Alexander is indeed very good at Bayesian updating. Far better than most of us. He’s saying he’s not very good because he’s comparing himself to a super high standard, a procedure which I mostly approve of for those who can psychologically handle it, but which in context is misleading. Even for those of us who have not done a bunch of explicit deliberate practice with it, you, yes you, are also very good at Bayesian updating. Not perfect, no. As Elon Musk reminded us this week, there’s tons of cognitive biases out there. Doing it exactly right is super hard. But any instinctive or reasonable attempt at approximation of this is much better than resorting to frequentism, and it is what you are doing all the time, all day, automatically, or else you would be unable to successfully put on pants.

After all, there’s ‘no evidence’ you know how. Someone really ought to do a study.