Sagan’s “extraordinary claims require extraordinary evidence” is nonsense. Here’s why

[Rramjet]

I have distilled the following discussion from another thread because I feel it is important enough to deserve a thread of its own.

…Basically, it's a case of "extraordinary claims require extraordinary evidence".

…The particular phrase quoted above is perhaps one of the most frequently promulgated pieces of woo within skeptical discussion. Precisely what is "extraordinary evidence"? Evidence is evidence. You either have it or you do not. Until anyone can actually define "extraordinary evidence" - and critically, provide an example of it - the phrase will remain nonsensical.

Of course the phrase has a certain emotional appeal, but such an appeal has no place in critical thinking, logic, or scientific discussion.

I consider Extraordinary Evidence to be either conclusive irrefutable evidence(such as finding an asteroid or an alien) or series of evidence that is superior to the evidence against the claim.

But I agree. I'd be happy to start with just good evidence.

Yes, you’re getting close, but strictly speaking there is also no such thing as “good” (or “bad”) evidence either. Evidence is evidence. You either have it or you do not. The preceding adjectives (“extraordinary”, “good”, etc), especially without context, are subjective and emotive value judgements that should have no place in true sceptical debate (with the implied constraints of conforming to the strict standards of critical thinking, logic and science).

I heard Richard Dawkins say it on TV a couple of evenings ago, in a programme about woo.

It's by Carl Sagan.

“Sagan is also widely regarded as a freethinker or skeptic; one of his most famous quotations, in Cosmos, was, "Extraordinary claims require extraordinary evidence."[40] This was based on a nearly identical statement by fellow founder of the Committee for the Scientific Investigation of Claims of the Paranormal, Marcello Truzzi, "Extraordinary claims require extraordinary proof."[41] This idea originated with Pierre-Simon Laplace (1749–1827), a French mathematician and astronomer who said, "The weight of evidence for an extraordinary claim must be proportioned to its strangeness."[42]” (http://en.wikipedia.org/wiki/Carl_Sagan)

I was in good company, then, with both Dawkins and Sagan saying it. Rramjet's view that it's "woo" looks like nonsense.

Like a game of Chinese whispers, the true meaning of the message becomes distorted over time to end in complete nonsense. That is, LaPlace’s sensible “weight of evidence” becomes in Sagan’s hands the nonsensical phrase “extraordinary evidence”.

…and no matter the “eminence” of the person stating it, it remains nonsensical.

We should thus, as ever, be forewarned about simply accepting without critical thought received “wisdom”. All is often rarely what it seems.

What "extraordinary evidence" means to me is evidence sufficient in quantity/quality to overcome what makes it an extraordinary claim (which is the evidence against it).

Regular evidence can count, but a lot of that (= an "extraordinary" amount) would be needed as appropriate to counter the evidence against it.

Yes, you are describing Laplace’s “weight of evidence”, but critically, an “extraordinary amount” of evidence does not make the evidence itself “extraordinary”.

I think that it is a mistake to repeat this quote ("extraordinary claims require extraordinary evidence")' because the people at which it is aimed do not understand what it means.
Linda

The “people at which it is aimed” do not understand it in equal proportion to the people who state it – because it is nonsense dressed up as wisdom.

It sounds like sScientists are cheating by raising the bar only for those claims they are biased against.

Corrected that for you… (but ‘scientists” can be forgiven, for they simply do not understand what they say).

[And since that isn't the case, all we have managed to do with this catchy-sounding phrase is to embroil ourselves in attempts to defend ourselves against people like Rodney, who quite correctly are able to recognize that we should not make a distinction a priori between ordinary and extraordinary claims.

Precisely.

I believe therefore, that when anyone who considers themselves to be a true sceptic is tempted to refer back to Sagan’s nonsensical statement containing the phrase “extraordinary evidence”, they should actually refer back to LaPlace and quote his statement (eg here: http://www.todayinsci.com/L/Laplace_PierreSimon/LaPlacePierreSimon-Quotations.htm) containing his eminently accurate and sensible phrase "weight of evidence”.

Of course one can paraphrase LaPlace if one considers his language too archaic for a modern audience. One can also include Sagan in such a paraphrase to get the best of both worlds! Thus:

To paraphrase both Sagan and LaPlace: Extraordinary claims require a greater weight of evidence.

Now we can argue over whether that contention itself actually has rational meaning…

As fls points out, the adjective “extraordinary” in “extraordinary claims” is itself a post hoc rationalisation.

Surely a claim is made and supporting or contrary evidence is put forward by the debating parties.

How can we know a priori that a claim is “extraordinary? After examination of the “weight of evidence”, what if it turns out to be a true claim? Surely then the claim becomes, by definition, “ordinary”. The point is, one simply cannot make the a priori value judgement.

…and of course if Sagan’s statement ( "extraordinary claims require extraordinary evidence"”) was not already shown to be nonsense, this puts the final nail in the coffin.

Quite simply, the veracity of any claim must be assessed according to the “weight of evidence”. It is only after such an assessment that we might consider using adjectives to describe either the claim or the evidence – and that only if we no longer consider ourselves to be true sceptics!

 

[Resume]

Somewhere in that wall of straw, you typed "evidence is evidence." I quite agree.

When some credible evidence is offered for, say, alien spacecraft visiting our blue planet, one is helpless but to examine it. Credible, able to withstand scrutiny; not extraordinary, but credible. Not photos of tossed hubcaps. Not blimps. No special pleading for that evidence either.

 

[tsig]

I have distilled the following discussion from another thread because I feel it is important enough to deserve a thread of its own.






Yes, you’re getting close, but strictly speaking there is also no such thing as “good” (or “bad”) evidence either. Evidence is evidence. You either have it or you do not. The preceding adjectives (“extraordinary”, “good”, etc), especially without context, are subjective and emotive value judgements that should have no place in true sceptical debate (with the implied constraints of conforming to the strict standards of critical thinking, logic and science).



“Sagan is also widely regarded as a freethinker or skeptic; one of his most famous quotations, in Cosmos, was, "Extraordinary claims require extraordinary evidence."[40] This was based on a nearly identical statement by fellow founder of the Committee for the Scientific Investigation of Claims of the Paranormal, Marcello Truzzi, "Extraordinary claims require extraordinary proof."[41] This idea originated with Pierre-Simon Laplace (1749–1827), a French mathematician and astronomer who said, "The weight of evidence for an extraordinary claim must be proportioned to its strangeness."[42]” (http://en.wikipedia.org/wiki/Carl_Sagan)


Like a game of Chinese whispers, the true meaning of the message becomes distorted over time to end in complete nonsense. That is, LaPlace’s sensible “weight of evidence” becomes in Sagan’s hands the nonsensical phrase “extraordinary evidence”.

…and no matter the “eminence” of the person stating it, it remains nonsensical.

We should thus, as ever, be forewarned about simply accepting without critical thought received “wisdom”. All is often rarely what it seems.


Yes, you are describing Laplace’s “weight of evidence”, but critically, an “extraordinary amount” of evidence does not make the evidence itself “extraordinary”.


The “people at which it is aimed” do not understand it in equal proportion to the people who state it – because it is nonsense dressed up as wisdom.


Corrected that for you… (but ‘scientists” can be forgiven, for they simply do not understand what they say).


Precisely.

I believe therefore, that when anyone who considers themselves to be a true sceptic is tempted to refer back to Sagan’s nonsensical statement containing the phrase “extraordinary evidence”, they should actually refer back to LaPlace and quote his statement (eg here: http://www.todayinsci.com/L/Laplace_PierreSimon/LaPlacePierreSimon-Quotations.htm) containing his eminently accurate and sensible phrase "weight of evidence”.

Of course one can paraphrase LaPlace if one considers his language too archaic for a modern audience. One can also include Sagan in such a paraphrase to get the best of both worlds! Thus:

To paraphrase both Sagan and LaPlace: Extraordinary claims require a greater weight of evidence.

Now we can argue over whether that contention itself actually has rational meaning…

As fls points out, the adjective “extraordinary” in “extraordinary claims” is itself a post hoc rationalisation.

Surely a claim is made and supporting or contrary evidence is put forward by the debating parties.

How can we know a priori that a claim is “extraordinary? After examination of the “weight of evidence”, what if it turns out to be a true claim? Surely then the claim becomes, by definition, “ordinary”. The point is, one simply cannot make the a priori value judgement.

…and of course if Sagan’s statement ( "extraordinary claims require extraordinary evidence"”) was not already shown to be nonsense, this puts the final nail in the coffin.

Quite simply, the veracity of any claim must be assessed according to the “weight of evidence”. It is only after such an assessment that we might consider using adjectives to describe either the claim or the evidence – and that only if we no longer consider ourselves to be true sceptics!

You're trying to get anecdotes in as evidence aren't you?

 

[William Parcher]

Evidence is evidence.

Counterfeiters want you to think that money is money.

 

[Resume]

You're trying to get anecdotes in as evidence aren't you?

I'm fairly sure that's the point of all this.

 

[ThunderChunky]

Ordinary claim: This is a cup of coffee.
Extraordinary claim: This is a magical cup of coffee.

I am going to require different levels of evidence to believe these two claims. The ordinary claim I may believe just on the look and smell alone. The extraordinary claim will require more evidence--extraordinary evidence. I will need to observe evidence of the extraordinary nature of the claim. Just looking and smelling like coffee is not enough.

Ordinary claim: I went to taco bell for lunch.
Extraordinary claim: I ate lunch on the moon.

I will believe the first claim in most cases just on someones word because it's quite ordinary. The second claim will require more evidence.

 

[Slimething]

As most of the posters above, I believe you're misinterpreting Sagan's line. I believe, but cannot prove, that he meant that an extraordinary claim must be matched with specific, irrefutable evidence that also matches the quality of the claim.

For example, someone can say "I can cure your hunger." and offers you a sandwich. That's pretty good evidence. However, if the claim was "I can solve world hunger", the sandwich doesn't quite cut it.

 

[John Jones]

Nice try.

Better luck next time.

 

[joobz]

I have distilled the following discussion from another thread because I feel it is important enough to deserve a thread of its own.


Yes, you’re getting close, but strictly speaking there is also no such thing as “good” (or “bad”) evidence either. Evidence is evidence. You either have it or you do not. The preceding adjectives (“extraordinary”, “good”, etc), especially without context, are subjective and emotive value judgements that should have no place in true sceptical debate (with the implied constraints of conforming to the strict standards of critical thinking, logic and science).



“Sagan is also widely regarded as a freethinker or skeptic; one of his most famous quotations, in Cosmos, was, "Extraordinary claims require extraordinary evidence."[40]
[snip]
Quite simply, the veracity of any claim must be assessed according to the “weight of evidence”. It is only after such an assessment that we might consider using adjectives to describe either the claim or the evidence – and that only if we no longer consider ourselves to be true sceptics!

Here's an example of a pendantry gone wrong.
"Extraordinary claims require extraordinary evidence." =
"Extraordinary claims require more evidence than non-extraordinary claims".

 

[RoboTimbo]

Santa Claus lives at the North Pole and delivers presents to all the good little boys and girls on Christmas eve.

Rramjet, is that an ordinary claim or is it extraordinary?

 

[fromdownunder]

*sigh*, Roger,

1. I had bacon and eggs for breakfast
2, I had bacon and eggs for breakfast at The Restaraunt at the End of The Universe.

I could show you a photo of a plate of bacon and eggs, with me next to it, or even shovelling a bit of bacon into my mouth. I could even photoshop a picture of an alien standing next to me while shovelling a bit of bacon into my mouth

Would you accept Statment 2 as accurate in either of these examples? Which one is the more likely statement, and why do you think (if you do) that I need more evidence for statement 2 than statement 1.

Norm

 

[tsig]

*sigh*, Roger,

1. I had bacon and eggs for breakfast
2, I had bacon and eggs for breakfast at The Restaraunt at the End of The Universe.

I could show you a photo of a plate of bacon and eggs, with me next to it, or even shovelling a bit of bacon into my mouth. I could even photoshop a picture of an alien standing next to me while shovelling a bit of bacon into my mouth

Would you accept Statment 2 as accurate in either of these examples? Which one is the more likely statement, and why do you think (if you do) that I need more evidence for statement 2 than statement 1.

Norm

If you told him you were abducted by aliens and then taken to the Restaurant then he'd go with 2.

 

[John Jones]

I have distilled the following discussion from another thread because I feel it is important enough to deserve a thread of its own.






Yes, you’re getting close, but strictly speaking there is also no such thing as “good” (or “bad”) evidence either. Evidence is evidence. You either have it or you do not. The preceding adjectives (“extraordinary”, “good”, etc), especially without context, are subjective and emotive value judgements that should have no place in true sceptical debate (with the implied constraints of conforming to the strict standards of critical thinking, logic and science).



“Sagan is also widely regarded as a freethinker or skeptic; one of his most famous quotations, in Cosmos, was, "Extraordinary claims require extraordinary evidence."[40] This was based on a nearly identical statement by fellow founder of the Committee for the Scientific Investigation of Claims of the Paranormal, Marcello Truzzi, "Extraordinary claims require extraordinary proof."[41] This idea originated with Pierre-Simon Laplace (1749–1827), a French mathematician and astronomer who said, "The weight of evidence for an extraordinary claim must be proportioned to its strangeness."[42]” (http://en.wikipedia.org/wiki/Carl_Sagan)


Like a game of Chinese whispers, the true meaning of the message becomes distorted over time to end in complete nonsense. That is, LaPlace’s sensible “weight of evidence” becomes in Sagan’s hands the nonsensical phrase “extraordinary evidence”.

…and no matter the “eminence” of the person stating it, it remains nonsensical.

We should thus, as ever, be forewarned about simply accepting without critical thought received “wisdom”. All is often rarely what it seems.


Yes, you are describing Laplace’s “weight of evidence”, but critically, an “extraordinary amount” of evidence does not make the evidence itself “extraordinary”.


The “people at which it is aimed” do not understand it in equal proportion to the people who state it – because it is nonsense dressed up as wisdom.


Corrected that for you… (but ‘scientists” can be forgiven, for they simply do not understand what they say).


Precisely.

I believe therefore, that when anyone who considers themselves to be a true sceptic is tempted to refer back to Sagan’s nonsensical statement containing the phrase “extraordinary evidence”, they should actually refer back to LaPlace and quote his statement (eg here: http://www.todayinsci.com/L/Laplace_PierreSimon/LaPlacePierreSimon-Quotations.htm) containing his eminently accurate and sensible phrase "weight of evidence”.

Of course one can paraphrase LaPlace if one considers his language too archaic for a modern audience. One can also include Sagan in such a paraphrase to get the best of both worlds! Thus:

To paraphrase both Sagan and LaPlace: Extraordinary claims require a greater weight of evidence.

Now we can argue over whether that contention itself actually has rational meaning…

As fls points out, the adjective “extraordinary” in “extraordinary claims” is itself a post hoc rationalisation.

Surely a claim is made and supporting or contrary evidence is put forward by the debating parties.

How can we know a priori that a claim is “extraordinary? After examination of the “weight of evidence”, what if it turns out to be a true claim? Surely then the claim becomes, by definition, “ordinary”. The point is, one simply cannot make the a priori value judgement.

…and of course if Sagan’s statement ( "extraordinary claims require extraordinary evidence"”) was not already shown to be nonsense, this puts the final nail in the coffin.

Quite simply, the veracity of any claim must be assessed according to the “weight of evidence”. It is only after such an assessment that we might consider using adjectives to describe either the claim or the evidence – and that only if we no longer consider ourselves to be true sceptics!

Can you be more verbose/specific?

 

[Fnord]

Here's an example of a pendantry gone wrong.
"Extraordinary claims require extraordinary evidence." =
"Extraordinary claims require more evidence than non-extraordinary claims".

Let me give it a try:

1a. More than <something> is extra-<something>.
1b. More than ordinary is extra-ordinary.

2a. Ordinary evidence is sufficient only for ordinary claims.
2b. Extra-ordinary evidence is sufficient only for extra-ordinary claims

::a. Ordinary evidence is required for ordinary claims.
::b. Extra-ordinary evidence is required for extra-ordinary claims.

I know that this must violate some rules regarding syllogistic reasoning, and may run afoul of equivocation, as well. But there may be another reader who is more skilled in this type of thinking to present the idea in a way that is both clear and concise to even the most deluded among us.

For the record, I also think that the OP is trying to make a case for allowing unsupported anecdotes {fantasies, ideas, legends, myths, opinions, rumors, et cetera} to be admissable as evidence.

 

[Patricio Elicer]

How can we know a priori that a claim is “extraordinary? After examination of the “weight of evidence”, what if it turns out to be a true claim? Surely then the claim becomes, by definition, “ordinary”. The point is, one simply cannot make the a priori value judgement.

We cannot know that a claim is extraordinary, but we can make a judgement that it is.

We can judge a claim to be extraordinary when, if true, it would revolutionize our knowledge of the natural world and/or oblige us to modify the laws of physics that have been experimentaly corroborated over and over again for centuries.

So, please answer this: if I say that I can move physical objects solely with the power of my mind, is it ordinary or extraordinary claim?

 

[Rramjet]

Ordinary claim: This is a cup of coffee.
Extraordinary claim: This is a magical cup of coffee.

What makes “This is a magical cup of coffee” an extraordinary claim? Could it be perchance that you are applying a post hoc rationalisation according to a particular belief of yours (based on examination of the evidence of course…) that “There are no such things as magical cups of coffee”?

However, when a claim is made, how can you a priori decide if the claim is “extraordinary” or not. Of course you cannot – you have to wait until after the evidence is examined to make that judgement call. That is why a general statement that requires “extraordinary” claims” to be treated somehow differently than “ordinary” claims is a nonsense. One simply cannot do that in science. All claims must be a priori treated equally according to the strict protocols of critical thinking, logic and the scientific method. Until the evidence is in, one simply cannot know what adjective might be applicable.

I am going to require different levels of evidence to believe these two claims.

But that is just the point. You are saying here that you actually require a greater weight of evidence to convince you of … what precisely… that your a priori belief is incorrect? But that is only your particular belief system and it may be completely misguided (ie; not generalisable). That is why, in science (as in critical thinking and logic) such a priori belief systems are carefully guarded against. Generalisability not only applies post hoc but also a priori (so they cannot be held hostage to personal belief systems).

The ordinary claim I may believe just on the look and smell alone.

Looks (and smells) can be deceiving.

The extraordinary claim will require more evidence--extraordinary evidence.

More evidence according to your own particular belief system maybe (that is your subjective opinion), but how does more evidence equate to “extraordinary” evidence. “More” does not equal “extraordinary” by any accepted definition.

I will need to observe evidence of the extraordinary nature of the claim. Just looking and smelling like coffee is not enough.

Now you are getting closer to it. But still you are second guessing the outcome.

Ordinary claim: I went to taco bell for lunch.
Extraordinary claim: I ate lunch on the moon.

I will believe the first claim in most cases just on someones word because it's quite ordinary. The second claim will require more evidence.

No, the second claim merely requires evidence!

Moreover, you would not seek evidence for the first claim simply because it does not particularly matter to you one way or other. However, if your life or death hinged on the claim “I went to taco bell for lunch.” (such as a murder was committed at KFC during lunch, for which your are being blamed) then you would be required to supply evidence.

Moreover, to you, the claim that you were the murderer in KFC is an extraordinary claim – presumably because you know the evidence that you could not have been the murderer (a post hoc rationalisation in other words), but to the CSI police, who deal with these things all the time, the claim that you were the murderer is just an ordinary claim that they deal with generally all the time. To them the claim is entirely ordinary.

Your second claim (“I ate lunch on the moon.”) only seems extraordinary because we have already run over the circumstantial evidence against that claim in our minds (ie; that no space agency has recently sent any rockets to the moon and your name has never come up in relation to astronauts). Again, the claim only becomes subjectively “extraordinary” after we examine the evidence (in this case circumstantial – but it IS still powerful enough for us to reject your claim!).

If you (or anyone) maintained the claim, then we (anyone) would simply require evidence in support of that claim – and notably, the only evidence you could supply would be ordinary evidence. For example you were on a particular rocket that launched from a particular place and that landed on the moon at a time that allowed you to have lunch there). That is ordinary evidence. Nothing “extraordinary” about it at all.

So I am still waiting for Sagan’s “extraordinary evidence” to be defined and an example of it provided. Until then, it remains a nonsense phrase.

 

[Jeff Wagg]

It's just as extraordinary to claim you have had lunch on the moon as it is to be able to prove it with a photograph, some rock samples, and a signed letter from NASA.

You have to fight Occam's razor here. If there's a simple and common explanation for something, you have to overcome it with stronger or better evidence, thus making your explanation the simpler one.

 

[Rramjet]

You're trying to get anecdotes in as evidence aren't you?

That is an extraordinary claim! Therefore I require you to provide extraordinary evidence in support of it. What’s that? You cannot? Oh dear…

 

[Resume]

So I am still waiting for Sagan’s “extraordinary evidence” to be defined and an example of it provided. Until then, it remains a nonsense phrase.

You typed "evidence is evidence." Perhaps you might provide an example.

Anecdote doesn't get it.

 

[Alan]

A claim is called "extraordinary" based on how the evidence for it and the evidence against it compare.

Do you understand that, at least?