Why do extraordinary claims require extraordinary evidence?

[cj.23]

Why do extraordinary claims require extraordinary evidence? This has got me thinking, and i have asked the question in several places, and no one really seems to know. Now the excellent Linda has demonstrated that this is true in certain types of Bayesian analysis in another thread I think, and i am not looking to restart the great semantic row with Larsen over what constitutes proof and what constitutes evidence.

My problem is I might just not understand what Sagan meant by it. JB Rhine started off with the level of significance in his parapsychology experiments a magnitude higher - and I think others had used similar measures in other fields. Yet surely, by any understanding, quite ordinary evidence will demonstrate the truth of any claim regardless of its apparent unlikelihood, if it is true? Indeed the Randi Challenge is based upon that very principle - it asks for quite ordinary proof of an exceptional claim?

Now Sagan may have meant the evidence must be qualitatively different
(so supernatural faith as evidence for a supernatural God? I doubt this was his meaning), quantitatively - lots of photos of Bigfoot, but no proof? -- or something else entirely?

I admire Sagan greatly, but I really don't understand the point he is making here. Can anyone put it in context or explain how it works?

cj x


Edit: fixed bolding, added PS.

 

[Blue Mountain]

The context in which I interperet the statement is that very often people will put forth a claim that "overturns basic physics/ chemistry/ biology/ whatever as we know it." This is an extraordinary claim, because the claiment is saying "what you've known until now is wrong, and I can show you why." Therefore he needs extraordinary proof of his theory, or it will be consigned to the dustbin of history along with many others.

Example: Relativity. Overturned physics as we knew it, which up to then had been based on Newton's mathematics and laws, and the postulate that there was an absolute fixed frame of reference. Einstein said there was no fixed frame of reference, that the speed of light was the same no matter what the motion of the observer was, and that mass increases as an object speeds up, then offered up extraordinary proofs by way of mathematics and a bold prediction based on the orbit of Mercury. The mathematics proved to be correct, as well as his prediction, and Einstein became one of the most celebrated physicists of the twentieth century.

Given the enormous body of evidence that has built up supporting the theory, it would take extraordinary proof to overturn Einstein and Special and General Relativity. Indeed, attempts are made all the time, mostly by amateurs, and they fail, sometimes miserably, sometimes spectacularly.

 

[Big Les]

I assumed it had to do with the high stakes of something that will require us to throw out much of what we think we already know, will affect our lives profoundly, or is otherwise Earth-shattering. So if a close friend insists that someone you both know is having an affair, you might not insist upon extraordinary evidence. If someone tells you the Earth is spherical, assuming you've had a basic education, you won't need evidence at all, you can take the statement at face value. But if someone claims to be able to read minds... that's where the extraordinary claim comes in, and the evidence for it must also be extraordinary, or at least corroborated, reproducible, etc, before we give it credence and look at our existing knowledge-base.

ETA - sorry, I forgot to refresh the page and wandered off for a bit! I've said much the same thing.

 

[polkablues]

Look at it this way: suppose you were to examine two claims. One being "My bedroom is haunted by a ghost," and the other being, "My bedroom is inhabited by a previously unclassified species of insect." Obviously, with no evidence, neither claim can be accepted. But the ghost claim, as it flies against the basic tenets of science and rational thinking that our understanding of the world is based upon, would require a much higher threshold of evidence to be accepted than would the insect claim.

Suppose you had evidence of both: a photograph of a ghost in your bedroom, and a photograph of an insect in your bedroom. In the case of the insect, you could simply take the photograph to an educated entomologist (preferably several), who could examine it and tell you with a certain degree of certainty whether it is an unknown species or not. This is because insects, as a concept, are understood as part of our reality, are accepted as real, and have been thoroughly studied and catalogued for many years. The ghost, on the other hand, is not an understood or accepted part of our reality, and would therefore require both more evidence and more definitive evidence than we would ask for in the other case.

 

[EeneyMinnieMoe]

It's pretty self-explanatory: you have quite an extraordinary claim that would overturn everything we think we know, I'm gonna give you the benefit of the doubt and not immediatly negate it but you better show some incredible evidence that overturns everything we know otherwise I'm not buying.

One example: conspiracy theories. Nine-eleven was carried out by the American government as a pretext to invade Iraq? Allright. What's your proof? Cause it better be good. It better not be a bootleg video that quotes one hijacker's flight instructor as saying 2 years before the attacks he wasn't experienced enough to fly a plane.

 

[cj.23]

Thanks chaps - again, I can see why in a Bayesian analysis, and I see your thinking - but really I don't see why the evidence required (extraordinariness) should be in any way related to the (perceived implausibility) of the outcome. Three things I find deeply extraordinary - the diversity of life ( and the fact it exists on this planet at all), the Big Bang, and the already cited Relativity. Heck, I find all kinds of things exceedingly unlikely - yet they can be established by ordinary evidence, and the normal application of scientific method and rational analysis. If we had really demanded "extraordinary" proof then Relativity may not have been investigated, as Einstein was still fairly obscure I believe, and the (normal) proofs would not have been discovered. I am not sure I like the saying - it seems to imply special circumstances apply to claims we find personally dubious or distasteful, whereas in fact I rather think that perfectly normal evidence will suffice to prove a claim, if it is true.

And talking of claims being true - I attributed the phrase to Sagan, but was it not originated by Marcello Truzzi (sp?) now I think about it? Anyway, Sagan certainly said it, even if he did not originate it.

My test case is the Challenge - does the challenge require extraordinary evidence? I would say not. It just requires a simple demonstration of evidence. Critics of the challenge, and I seem to meet many and argue with them, tend to think special pleading is used to dismiss the "powers" psychics demonstrate -- well if anyone ever completes the challenge we will know, but I wonder if Sagan's dictum might also be partly responsible for this PR issue. "Ordinary scientific evidence will suffice!" would do for me.

cj x

Edit: corrected Big Band to Big Bang. Glenn Miller may be unusual, but not that far out!

 

[joobz]

it seems your whole issue is with the subjective term extraordinary.
Claims require evidence to be validated.

I can see your complaint, but I've always taken that statement to be directed at the layman and not the scientist. The level of detail required for any publication in a well respected journal would be considered "extraordinary" to a person who conducts paranormal investigations. To such individuals, blinded, double blinded and statistical analysis will seem extraordinary.

 

[scotth]

Extraordinary evidence in the case means evidence that is completely verifiable and examinable. That would be opposed to a large stack of anecdotes or experiments that can't be duplicated.

It doesn't mean supernatural or anything beyond what is said above. In this sense, the Bible has no extraordinary evidence. Bigfoot has weak circumstantial evidence. All of it is stuff that could easily be faked. Plaster impressions, horrible photos, and movie clips that look just like a man in a suit.

Extraordinary evidence for a Bigfoot could be as little as some fur or waste. These things can be analyzed for identity. They could get DNA from either fur or waste and know definitively if it was something new. One carcass (don't they ever die of anything?) would be extraordinary and would seal the case in a stroke. Despite all the long lack of evidence in the past, everyone could know for sure, there is such a thing as Bigfoot.

 

[slyjoe]

I think what Sagan was saying was that if your claim goes against a lot of other evidence, then it must be pretty extraordinary. A piece of evidence is not taken on its own - the question is how it fits in with other evidence. If we have a boatload of evidence pointing us in one direction, the evidence to point us in another direction has to be extraordinary because it would have to overcome the previous evidence, not just be a piece of evidence on its own.

Kind of what EenyeMinnieMoe Said. Got to give credit where credit is due!

 

[ObscureReferenceMan]

I like to approach it in the opposite direction. Ordinary claims require ordinary (i.e. not especially rigorous) evidence. For example; I tell you that I make the best blueberry pancakes. You might accept my word, or ask me to make a batch. Nothing extraordinary there. But if I say I can survive without eating, you won't take my word for it. You'd probably want to watch me very closely over the course of many days, monitor my weight, have a doctor on hand to make sure I don't die, etc. That's a little more "extraordinary".

Just my take on it.

 

[Arkan_Wolfshade]

I apologize as I am too tired to read this thread in its entirety atm, however I did want to toss in Hume's quote as I feel it is relevant:
"The plain consequence is (and it is a general maxim worthy of our attention), "That no testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous, than the fact, which it endeavours to establish; and even in that case there is a mutual destruction of arguments, and the superior only gives us an assurance suitable to that degree of force, which remains, after deducting the inferior.""

 

[Skeptical Greg]

Why do extraordinary claims require extraordinary evidence?

They don't .. For any claim, any ( supporting ) evidence will do.

It's just one of those catchy phrases like " Absence of evidence is not evidence of absence .. "

Which sounds good, but is far from true.. When something is missing, you have a pretty good clue it's not there..

 

[fls]

I have to admit that I don't like the use of the statement to counter paranormal claims. Mostly because people have quite different conceptions of what "extraordinary" means. So it can give the impression that skeptics are asking for an unreasonable level of evidence in order to be ornery (I presume).

A quick review tells me that it was Carl Sagan's summary of Hume's writings on miracles. The excerpt that I consider relevant - "That no testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous than the fact which it endeavors to establish" - does not seem to need any improvement to me, but I'm not normal I suppose. It's an easily applicable guideline for the evaluation of claims. And while it originally referred to eye-witness testimony, it serves just as well to refer any other kind of information.

It makes it clear why it's almost impossible for anecdotes, by themselves, to serve as evidence of miracles. The fact that stories can be false through inadvertant or deliberate deception is well-established, making falsehood not at all miraculous.

I think what gets lost in the translation to "extraordinary claims require extraordinary evidence" is that most of the things we now take for granted are backed by extraordinary evidence (as has already been pointed out). It's not that we are asking something extra from paranormal researchers. It's that we are asking for the same thing as is used for normal research. Relativity (I'll start with Special) did have extraordinary evidence. The results of the Michelson-Morley experiments were unexplained and unrefutable. It would be more remarkable that a theory that completely explained the previously inexplicable was false, than it would be for the theory to be true.

I agree that all we really want is the ordinary amount of evidence. But I think that the people we tend to be dealing with are naive about what is ordinarily required. "Extraordinary" conveys the idea that it needs to be more than they realize, but it unfortunately also conveys the idea that it's an unusual requirement.

Linda

 

[Cuddles]

I am not sure I like the saying - it seems to imply special circumstances apply to claims we find personally dubious or distasteful, whereas in fact I rather think that perfectly normal evidence will suffice to prove a claim, if it is true.

But the point is why do you consider something an extraordinary claim? If I claim to have a cat in my house most people would not consider it extraordinary. They might not believe me, but a picture of a cat with me in a house would probably suffice as evidence, because it is well known that cats exist and are often kept in houses. On the other hand, if I claim to have a dinosaur in my house, this would be considered fairly extraordinary. A simple photo would no longer be sufficient because there is a lot of evidence that dinosaurs do not exist any more, and would be unlikely to make good pets if they did. The extraordinary claim requires a lot more evidence than the ordinary precisely because it is extraordinary. Yes, evidence is evidence, no matter what the claim, but the amount and type of evidence required to accept a claim is very different depending on what other evidence would have to be rejected.

Edit : To continue Linda's point, let's assume that everything requires exactly the same amount of evidence. What we view as an ordinary claim would be something that already has lot's of evidence to support it. We have evidence that cat's exist and that they are kept as pets. I would only need a little more evidence to prove that I personally have one. To prove that I personally have a dinosaur I would first need to provide all the evidence that they exist and that they can be kept as pets, and then show that I actually have one myself.

 

[MRC_Hans]

Why do extraordinary claims require extraordinary evidence? This has got me thinking, and i have asked the question in several places, and no one really seems to know. *snip*
My problem is I might just not understand what Sagan meant by it. JB Rhine started off with the level of significance in his parapsychology experiments a magnitude higher - and I think others had used similar measures in other fields. Yet surely, by any understanding, quite ordinary evidence will demonstrate the truth of any claim regardless of its apparent unlikelihood, if it is true?

The problem is knowing if something is true.

Let us start by seeing what makes a claim extraordinary. As somebody already mentioned, an extraordinary claim is one that is profoundly against our expectations. Our expectations, from a scientific POV, is that any new claim will mostly fit in with what we already know. It is expected to expand, refine, possibly slightly modify our present body of knowledge.

An ordinary claim is perhaps not predicted by our present knowledge, but it is not contradicted by it.

An extraordinary claim is contradicted by our present knowledge. This means that the evidence for it has to not just provide a direct support for the claim, but it also has to overthrow the implicit evidence contained in our general body of knowledge.

To give an example: Suppose you claim that, under some specific condition, an object will fall up. You make an experiment that, with 95% statistical siginificance shows this to be true. In other words, there is only 5% probability that you are wrong. Now, if this was an ordinary claim, we would generally accept that it was proven (although it would always be important to repeat the experiment). However, for things to fall up, thousands of standard physics experiments would each have to have failed all of the thousands of times they have been conducted, volumes of physical laws would need rewriting, etc. etc.

Thus, for the extraordinary claim, it is much more probable that the evidence for it is due to an error or a statistical freak, than assuming that ALL the contrary evidence happens to be wrong.

This is the reason an extraordinary claim requires extraordinary evidence.

Hans

 

[Freethinker]

I think the easiest way to understand it is to go to the other extreme. Routine claims will be believed based on ordinary evidence.

If I post here that I met cj.23 and that he/she is a wonderful person who plays the mandolin beautifully, people will pretty well buy that and that will become part of their opinion of cj.23. If I post here that cj.23 has demonstrated to me the ability to levitate, I would be taken to task for failing to provide evidence to support my claim.

 

[drkitten]

I think the easiest way to understand it is to go to the other extreme. Routine claims will be believed based on ordinary evidence.

Yeah, that's more or less my approach, too.

An "extraordinary" claim is, more or less, one that's not believable. The more unbelievable a claim is, the more evidence it will take in its favor to shift it to being believable.

 

[SirPhilip]

Why do extraordinary claims require extraordinary evidence? This has got me thinking, and i have asked the question in several places, and no one really seems to know.

Because it encourages something called an "argument for authority", the single most dangerous thing in existence facing humanity. Mind you, it has nothing to do with heartfelt faith - the most boldly mad example was in fact the social darwinism of the Third Reich.

Indeed the Randi Challenge is based upon that very principle - it asks for quite ordinary proof of an exceptional claim?

That is always directed towards people who are fundamentally dishonest and openly declare they can do things they know that they cannot.

I admire Sagan greatly, but I really don't understand the point he is making here. Can anyone put it in context or explain how it works?

The supernatural does exist by implication, but no supernatural, or even supranatural event has ever been known to occur - it only exists in the form of mythology.

 

[Senex]

Why do extraordinary claims require extraordinary evidence? This has got me thinking, and i have asked the question in several places, and no one really seems to know. Now the excellent Linda has demonstrated that this is true in certain types of Bayesian analysis in another thread...

My problem is I might just not understand what Sagan meant by it. I admire Sagan greatly, but I really don't understand the point he is making here. Can anyone put it in context or explain how it works?

I believe Sagan was making a metaphor to this metaphysical question:

Why do naughty women need to be spanked?

It's best not to overthink the issue. Carl Sagan made me and thousands of others wonder about the billions and billions of stars in the night sky. Sagan applied logic to most everything in life except when his old lady had that look in her eye -- he knew when she was up to no good and needed a good spanking. His claim of her naughtiness and evidence of such were indeed extraordinary (he actually had none) but he knew anyway what needed to be done. That's why Sagan is one of my heroes.

Everyone knows that Thomas Bayes was a bottom (probably why the excellent Linda feels compelled to quote him.) I'm a Bayesian when it comes to the thinking that subjective belief as a way of interpreting probability works in everyday life; just read my posts.

 

[cj.23]

it seems your whole issue is with the subjective term extraordinary.
Claims require evidence to be validated.

I can see your complaint, but I've always taken that statement to be directed at the layman and not the scientist. The level of detail required for any publication in a well respected journal would be considered "extraordinary" to a person who conducts paranormal investigations. To such individuals, blinded, double blinded and statistical analysis will seem extraordinary.

Referee! I see your point, but I must object -- I conduct "paranormal investigations". In fact you can have a look at some of my (admittedly a bit woolly) proposed methodologies in this thread
http://www.internationalskeptics.com/forums/showthread.php?t=74627
-- see the 15th February 1:47pm post for a discussion. I don't claim it's particularly well though through, but i think it demonstrates some degree of consideration of methodology and possible issues! And to be honest, any ganzfeld work is going to fail to convince if you don't have a vague idea of stats and analysis? However years in the "ghosthunting" community means I do understand what you are saying, so I'm not objecting too much!

cj x