Eyewitness testimony/anecdotal evidence: Why is this bad evidence?

[jayman]

Why is eyewitness testimony and anecdotal evidence the lowest form of evidence in science?


What books would you recommend that explain why anecdotal evidence is not good evidence?


Any good videos out there that demonstrate the unreliability of eyewitness testimony?


I just want to be able to fully understand this and be able to explain it to someone who is new to skepticism.

Any and all help would be appreciated.

Thanks!
Josh

 

[Kell]

The main reason eyewitness and anectdotal evidence is bad, and unnacceptable for the scientific method, is that a specific testimony cannot be replicated.

Other broader reasons are more obvious things that are known about human nature, which are understood quite well: lying, pareidoilia, exaggeration, memory is subject to change, and perceptions are subjective based on who what where and how someone witnessed any claimed phenomena.

It may be that none of these are an issue for a given claim. But all of them may be issues also. There's no way to know based on the testimony alone. So other forms of evidence end up having to be used anyway.

 

[Here_to_learn]

Any good videos out there that demonstrate the unreliability of eyewitness testimony?

 

[novaphile]

Confirmation Bias

Why is eyewitness testimony and anecdotal evidence the lowest form of evidence in science?

There is something weird about the way we see the world as humans.

For example, I might think of calling my father on the telephone. This might occur once an hour, every day (hopefully only while I'm awake).

It's possible that I may think of making this call at the exact same time as my father calls me. OMG!!! He must be telepathic!!!

Of course, I've forgotten that I've been thinking about calling him every day for the last 3 months, thus forgetting that I've had that thought 1500 times without him calling me.

Confirmation bias is remembering the events that support an idea, while forgetting the events that don't.

With the calling my father example, it become more likely that he'll call me, as time passes, because I haven't called him. If I put off the call long enough, and thought that I should be calling often enough... the chances are always increasing that he'll call me while I'm having that thought.

Anecdotal evidence is collecting all of the events that seem to support an idea, like telepathy, while ignoring (often through confirmation bias) all the events that don't support it.

 

[The Don]

There was an experiment where observers were asked to watch a video of a group of people and keep track of the number of times they passed a basketball between them.

In the middle of the video a guy walks in wearing a gorilla costume and pounds his chest. A small proportion of people notice the gorilla. I was quite frankly shocked when I first saw this (on a BBC programme about the unreliable nature of eyewitnesses) and was part of the large % of people who missed the 6 foot gorilla.

The video is here:
http://viscog.beckman.illinois.edu/flashmovie/15.php

 

[tesscaline]

Any good videos out there that demonstrate the unreliability of eyewitness testimony?

Have you seen the movie "My Cousin Vinny"? Even though the movie is itself fictional, the court scenes in which "Vinny" demonstrates the flaws in the witness' testimony is pretty on the nail for at least one of the reasons why eye witness testimony is considered so unreliable.

 

[Gawdzilla Sama]

It's impossible to avoid some "editing" of events when you recall something. You can't be 100% certain that the things you remember are ALL related to event in question. Did you see a red car at the scene, or was that earlier when a red minivan cut you off in traffic? As time goes by the distance between the event and the incident grow relatively smaller and conflation happens.

As noted above, humans aren't good observers. And they're worse at painting an accurate word picture of events at a later date. For an investigator, it's like hearing someone hum a tune and trying to write the score for the music from that. A objective piece of evidence, like a tape recording of the music being played, is much better, fewer incidentals and biases thrown in.

 

[plumjam]

There was an experiment where observers were asked to watch a video of a group of people and keep track of the number of times they passed a basketball between them.

In the middle of the video a guy walks in wearing a gorilla costume and pounds his chest. A small proportion of people notice the gorilla. I was quite frankly shocked when I first saw this (on a BBC programme about the unreliable nature of eyewitnesses) and was part of the large % of people who missed the 6 foot gorilla.

The video is here:
http://viscog.beckman.illinois.edu/flashmovie/15.php

If they missed the gorilla, if anything, that's a positive thing. It means they were concentrating on what they'd been designated to do wrt the exercise, i.e. counting the number of times the ball was passed.
I'd guess the people most likely to notice the gorilla were the ones not giving much of a stuff about the counting.

 

[Gawdzilla Sama]

If they missed the gorilla, if anything, that's a positive thing. It means they were concentrating on what they'd been designated to do wrt the exercise, i.e. counting the number of times the ball was passed.
I'd guess the people most likely to notice the gorilla were the ones not giving much of a stuff about the counting.

It's also a problem when they're concentrating on driving and an accident happens right in front of them. They miss important details not related to their mission. And that's both normal and a problem when they're reported their version of events.

 

[Akhenaten]

A lot of it depends on the nature idea that's being put forth too.

If someone wants to convince you that 'this Winter is colder than last Winter' with a story about a burst water pipe, it's probably worth a pass, but if someone tells you they've got a cold fusion power plant in their basement, an anecdote isn't going to cut it.


Extraordinary claims need extraordinary evidence is the favourite cliché, I think.

 

[The Don]

BBC podcast:

http://news.bbc.co.uk/1/hi/programmes/documentary_archive/4560303.stm

(may only be available to UKains)

 

[Tumblehome]

Any good videos out there that demonstrate the unreliability of eyewitness testimony?

This card colour changing trick is good at demonstrating the limitations of our powers of observation.

 

[Paul2]

Extraordinary claims need extraordinary evidence is the favourite cliché, I think.

I think woos get frustrated because they don't understand the above idea and how eyewitness testimony must work as part of it.

1. "I saw you go to the store." No need for physical or extraordinary evidence here (although, did I see you clearly; was it someone else?).

2. "I saw an alien blimp." Physical evidence very much needed because of the extraordinary claim.

 

[Akhenaten]

Yep. Maybe that's why I called it a cliché without really thinking about it. You seem to hear the phrase so often but it just never gets in with some people.


PS Alien blimps are only extraordinary if there is no evidence of a gay rodeo being scheduled on the day of the sighting.

 

[Maia]

Well, in the context of the courts, eyewitness testimony is the only form of evidence which is considered to be direct evidence, and I think it's got enough safeguards around it for accurate use (such as cross-examination, the introduction of expert witnesses testifying about the reliability of suspect identification, etc.) It tends to be the most reliable through corroboration; I saw Person A rob the bank, you saw Person A rob the bank, she saw Person A rob the bank, it's on the security camera tape, and Person A's fingerprints are all over the counter. I just posted some info about individual case studies in another thread; these could maybe be called controlled anecdotal evidence. But they are still important forms of evidence as medical studies. I think what's really being asked here is a question about the reliability of statements like "I saw a UFO land in my backyard last night, and a bunch of aliens got out of it." But that's a little different from talking about the general "unreliability of eyewitness testimony."

 

[Gate2501]

I think what's really being asked here is a question about the reliability of statements like "I saw a UFO land in my backyard last night, and a bunch of aliens got out of it." But that's a little different from talking about the general "unreliability of eyewitness testimony."

Yes, I agree, I believe that what we are really talking about here is satisfying not a legal burden of proof, but an epistemological one.

When a person makes a statement about an event that does not fit within the body of knowledge that science has thus far accumulated, and is making a claim which would extend ontology, the burden of proof becomes very hard to satisfy in a way which would verify the validity of the statement. If we were to accept anecdotal evidence for this sort of claim, parsimony would be out the window, and we would believe in all manner of nonsense. It would be nearly impossible to sort through it all in an attempt to actually learn anything about our world.

We would get bogged down, neck deep in bullcrap.

 

[Gawdzilla Sama]

Well, in the context of the courts, eyewitness testimony is the only form of evidence which is considered to be direct evidence, and I think it's got enough safeguards around it for accurate use (such as cross-examination, the introduction of expert witnesses testifying about the reliability of suspect identification, etc.) It tends to be the most reliable through corroboration; I saw Person A rob the bank, you saw Person A rob the bank, she saw Person A rob the bank, it's on the security camera tape, and Person A's fingerprints are all over the counter. I just posted some info about individual case studies in another thread; these could maybe be called controlled anecdotal evidence. But they are still important forms of evidence as medical studies. I think what's really being asked here is a question about the reliability of statements like "I saw a UFO land in my backyard last night, and a bunch of aliens got out of it." But that's a little different from talking about the general "unreliability of eyewitness testimony."

What about video tapes, still pictures, etc.? Not direct? And physical evidence, fingerprints, DNA, etc.? Not direct? And also objective, whereas eyewitness testimony is subjective.

 

[Maia]

physical evidence, fingerprints, DNA, etc.? Not direct?

Nope, circumstancial. But circumstancial doesn't mean "worse" or less reliable or trustworthy. It really doesn't have the meaning that it has in popular use.

 

[jayman]

Extraordinary claims need extraordinary evidence is the favourite cliché, I think.

What makes a claim extraordinary?

Why does that claim require extraordinary evidence?

What is extraordinary evidence?


Why is the claim,
"I saw an alien spacecraft hovering over my house last night" extraordinary?

Why is the claim "I saw Dan at the store today" not extraordinary?


Just more questions non-skeptics might ask.

 

[Akhenaten]

Nope, circumstancial. But circumstancial doesn't mean "worse" or less reliable or trustworthy. It really doesn't have the meaning that it has in popular use.

I think one difference between acceptable evidence to a court and acceptable evidence for science is that the court system is adversarial and destructive (of the other guy's evidence), whereas science is more co-operative (as a whole), and cumulative.

I'm sure you can say this better than I can.