Anecdotal evidence is scientifically unacceptable? Myth?

[PartSkeptic]

My posts show that I have posted a number of anecdotes, and the response from skeptics is that only repeatable peer-reviewed evidence is acceptable.

In the last few weeks I have had to research possible cures for sarcoptic mites in dogs and the possibility that the dog mite can infect humans.

The internet abounds with false and contradictory “facts”. Many claim that these “facts” are supported by “scientific peer-reviewed experiments”.

It should be a simple matter to find a cure that works, and explanations as to the whys and wherefore – but it is not. And sufferers are turning to anecdotal evidence from other people.

Many of the home-grown cures are often mentioned in older medical journals that used anecdotes as a basis for the cure – they were observed to work most of the time. There are now newer “cures” that are touted and promoted and sold – also anecdotally supported.

Over and over one gets posts that slam these cures as dangerous quackery, and urge others to go to a doctor and get a medical prescription.

Many report these prescriptions as failing them, and they are suffering.

I did a lot of research. I found that many scientific reports included anecdotal “reports”. The reason is that there is a tremendous variation in the human condition, and science uses strictly controlled laboratory conditions that are extrapolated to humans. Human testing is forbidden, and now animal testing is under fire.

I can tell you that doctors will insist that their patients condition is not medically possible, and yet it happens. This happens more often than one thinks.

So I challenge skeptics – would you bend your inflexible “science or death” rule and accept that there are times that anecdotal evidence sometimes has more truth that the science experiments?

What is seriously worrying is that politicians and big corporations appear to be “manipulating” the scientific “evidence” to dumb down the population and push them all to conventional doctors and medicines at great cost despite many that fail to achieve a cure.

I have my research to back up what I am saying.

 

[Daylightstar]

...
I have my research to back up what I am saying.

Where is it?

 

[Loss Leader]

So I challenge skeptics – would you bend your inflexible “science or death” rule and accept that there are times that anecdotal evidence sometimes has more truth that the science experiments?

No. Anecdotal evidence is important in that it can help scientists form testable hypotheses. However, a properly controlled test is always better.

In the case where testing may be unethical (such as medication dosing for children), anecdotes may be all that are available. In that case, they should be gathered in as great a number as possible. They should be reported by neutral observers, such as doctors. Those same observers should go out of their way to collect information from those who haven't volunteered their stories. As much biographical data should be collected as possible. And then trained individuals should use statistical methods to code and sort the information for significance.

Any given anecdote on its own is very close to worthless.

 

[Tiktaalik]

Of course there is a place for personal experience/anecdote. For example, following injury, I've been assured by my doctor that hydrocodone is a widely-accepted and effective pain medication, borne out by much scientific evidence. However, my personal experience is that it does me no good, because I vomit it up before it can take effect.

But that's a bit different than accepting an anecdotal cure when there is no scientific evidence supporting it whatsoever, or when assertions fly in the face of evidence. This is where the problem lies.

 

[Robrob]

I have my research to back up what I am saying.

That would move your anecdotal evidence into the realm of actual evidence. You should post it.

 

[tsig]

My posts show that I have posted a number of anecdotes, and the response from skeptics is that only repeatable peer-reviewed evidence is acceptable.

In the last few weeks I have had to research possible cures for sarcoptic mites in dogs and the possibility that the dog mite can infect humans.

The internet abounds with false and contradictory “facts”. Many claim that these “facts” are supported by “scientific peer-reviewed experiments”.

It should be a simple matter to find a cure that works, and explanations as to the whys and wherefore – but it is not. And sufferers are turning to anecdotal evidence from other people.

Many of the home-grown cures are often mentioned in older medical journals that used anecdotes as a basis for the cure – they were observed to work most of the time. There are now newer “cures” that are touted and promoted and sold – also anecdotally supported.

Over and over one gets posts that slam these cures as dangerous quackery, and urge others to go to a doctor and get a medical prescription.

Many report these prescriptions as failing them, and they are suffering.

I did a lot of research. I found that many scientific reports included anecdotal “reports”. The reason is that there is a tremendous variation in the human condition, and science uses strictly controlled laboratory conditions that are extrapolated to humans. Human testing is forbidden, and now animal testing is under fire.

I can tell you that doctors will insist that their patients condition is not medically possible, and yet it happens. This happens more often than one thinks.

So I challenge skeptics – would you bend your inflexible “science or death” rule and accept that there are times that anecdotal evidence sometimes has more truth that the science experiments?

What is seriously worrying is that politicians and big corporations appear to be “manipulating” the scientific “evidence” to dumb down the population and push them all to conventional doctors and medicines at great cost despite many that fail to achieve a cure.

I have my research to back up what I am saying.

My friendly, family doctor is trying to kill me?

 

[Dinwar]

I did a lot of research. I found that many scientific reports included anecdotal “reports”.

Probably true. However, I strongly urge you to examine this more closely. Anecdotes aren't forbidden in science; they're merely not considered valid proof of anything. They can support an idea, but alone they cannot prove it.

Secondly, you need to re-examine your concept of what a scientific experiment is. There are two types of science: experimental and historical. Experimental science includes things like physics, chemistry, genetics--things you can run experiments on in a lab (or supercolider, or whatever). It's science conducted one variable at a time. Historical sciences include things like paleongology, structural geology, archaeology, and astronomy--stuff where we rely on what are called natural experiments, and where half the time the study is examining what the variables even are, rather than isolating them. Studies in the historical sciences are not anecdotes, but rather carefully controlled and constructed scientific experiments--just, not the type you learned about in middle school.

Don't worry if this is new to you--it's new to a surprising number of scientists, so I don't expect the general public to be widely aware of the difference. There's something I call physics envy, by which I mean people assuming that to be legitimate science it must imitate physics (or, more commonly, an idealized version of physics). The reality is far different, however. Science is done as often with a topo map and colored pencils as it is with computer simulations and subatomic particle accelerators.

This is applicable to discussions of medicine because case studies represent times when medical experiments follow the historical science model. They don't know the variables, and half the time the whole point is to determine those; however, they are carefully and methodically examined to determine every bit of data possible from the experiments. These are about as far from anecdotes as anecdotes are from oranges.

If you could provide an anecdote that was more true than a peer-reviewed study, it would help. We'd be able to examine why, and whether it was actually an anecdote or not. It's surprisingly easy to confuse that issue.

 

[JoeMorgue]

So anecdotes equal evidence if you completely redefine "anecdote" and "evidence" to mean completely different things.

 

[Trevor from Toronto]

As with the other posters, I challenge you to present your research to make your point.

Otherwise, anecdotal is only useful to help create hypothesis for developing scientific tests and/or as advertising for a product (at all levels in which the term advertising may be used). Anecdotal evidence used to convince people that something works without any scientific tests to back it up is merely false advertising.

 

[Axxman300]

Case in point:

Alfred Wegner - He proposed continents moved. His thesis fell apart where he suggested they floated like boats.

His thesis failed - but...

Using the data he collected, and the anecdotal evidence they combined it with new bathymetric imagery, and fossil evidence the theory of Plate Tectonics became a reality. So while Wegner was right that the continents moving, he was wrong about the mechanics.

This is not the same as having an EMF meter go crazy in an abandoned house as "proof" of ghosts. It is sure as heck not the same as saying I took a Gee-Whiz pill for sixty day and I lost 200 pounds, grew my hair back and picked up four inches in my pants.

 

[John Jones]

[...]
I have my research to back up what I am saying.

Put up or shut up, as they say in poker games.

 

[Dinwar]

So anecdotes equal evidence if you completely redefine "anecdote" and "evidence" to mean completely different things.

Well, anecdotes are data, and when they support a conclusion they count as evidence. Not high-quality evidence--they can never rule out alternative explanations, and they aren't exactly strong support for any position--but they are evidence none the less.

Using the data he collected, and the anecdotal evidence they combined it with new bathymetric imagery, and fossil evidence the theory of Plate Tectonics became a reality. So while Wegner was right that the continents moving, he was wrong about the mechanics.

What specific anecdotes were used?

The bathymetric data wasn't anecdotal--it was a systematic study, all be it for a different purpose (it was a war, what can you do?). The fossil evidence was rigorously compiled--it's one of those historical science natural experiments. And you missed...well, most of the story.

So while Wegner was right that the continents moving, he was wrong about the mechanics.

Everyone knew that continents moved. Any grade school kid can piece together the continents and see that they appear to fit, and it takes a special kind of stupid to not think there's a reason.

You may want to re-examine this whole line of reasoning. The issue with Wagner is that his proposal was pretty much bunk. He proposed that continents moved through the oceanic crust--which is about the equivalent of marshmellows moving through gram crackers. In fact, marshmellows, gram crackers, and melted chocolate serve as a very good model for tectonic theory. His hypothesis was rejected for the very sound reason that he had no mechanism.

This is a fairly complex episode in scientific history, one which certainly wasn't decided on any anecdotes. It was rigorous, methodical, and extremely heated. This will not help your case.

 

[Brian-M]

Many of the home-grown cures are often mentioned in older medical journals that used anecdotes as a basis for the cure – they were observed to work most of the time.

If they were relying solely on anecdotal "evidence" for these journal articles they wouldn't have any way to know that they're observed to work most of the time. The anecdotes would be highly susceptible to to reporting bias and confirmation bias.

There's nothing scientific about anecdotes. But you can apply scientific methods to test whether or not anecdotal claims are accurate.

For example, when quinine-resistant strains of malaria began to emerge a new treatment was needed. In the 1960's the Chinese army set up a research program named Project 532 to find it. This program went through a list of almost 5000 traditional Chinese remedies for malaria, all of which were presumably supported by anecdotal "evidence".

Finally in 1972 they found one (just one out of close to five thousand traditional remedies) that proved effective in the treatment of malaria (an extract from the leaves of annual wormwood now called Artemisinin).

If you were just going to rely on anecdotes, then all of those treatments were equally credible. It took actual scientific research to find the truth behind these anecdotal claims.

 

[Brian-M]

I can tell you that doctors will insist that their patients condition is not medically possible, and yet it happens. This happens more often than one thinks.

Can you give examples of this, and evidence that the doctors are wrong? Or is this claim just based on anecdotes?

I did a lot of research. I found that many scientific reports included anecdotal “reports”. The reason is that there is a tremendous variation in the human condition, and science uses strictly controlled laboratory conditions that are extrapolated to humans. Human testing is forbidden, and now animal testing is under fire.

Wrong.

Human testing is mandatory for new drugs and treatments. This testing is known as a clinical trial....

Phase 0 trials are the first-in-human trials. Single subtherapeutic doses of the study drug are given to a small number of subjects (10 to 15) to gather preliminary data on the agent's pharmacodynamics (what the drug does to the body) and pharmacokinetics (what the body does to the drugs).[19]

In Phase 1 trials, researchers test an experimental drug or treatment in a small group of people (20–80) for the first time to evaluate its safety, determine a safe dosage range, and identify side effects.

In Phase 2 trials, the experimental treatment is given to a larger group of people (100–300) to see if it is effective and to further evaluate its safety.

In Phase 3 trials, the treatment is given to large groups of people (1,000–3,000) to confirm its effectiveness, monitor side effects, compare it to commonly used treatments, and collect information that will allow it to be used safely.

In Phase 4 trials, postmarketing studies delineate additional information, including the treatment's risks, benefits, and optimal use.

Before pharmaceutical companies start clinical trials on a drug, they conduct extensive preclinical studies.

Before a drug reaches the market it has already been tested on thousands of people.

 

[fuelair]

Case in point:

Alfred Wegner - He proposed continents moved. His thesis fell apart where he suggested they floated like boats.

His thesis failed - but...

Using the data he collected, and the anecdotal evidence they combined it with new bathymetric imagery, and fossil evidence the theory of Plate Tectonics became a reality. So while Wegner was right that the continents moving, he was wrong about the mechanics.

This is not the same as having an EMF meter go crazy in an abandoned house as "proof" of ghosts. It is sure as heck not the same as saying I took a Gee-Whiz pill for sixty day and I lost 200 pounds, grew my hair back and picked up four inches in my pants.

That last is kind of a small hunk of crap. Many people have claimed movements of well above 12 inches in threads here!!!

 

[haibut]

Case in point:

Alfred Wegner - He proposed continents moved. His thesis fell apart where he suggested they floated like boats.

His thesis failed - but...

Using the data he collected, and the anecdotal evidence they combined it with new bathymetric imagery, and fossil evidence the theory of Plate Tectonics became a reality. So while Wegner was right that the continents moving, he was wrong about the mechanics.

This is not the same as having an EMF meter go crazy in an abandoned house as "proof" of ghosts. It is sure as heck not the same as saying I took a Gee-Whiz pill for sixty day and I lost 200 pounds, grew my hair back and picked up four inches in my pants.

Wegener (with an 'e', not. that it matters). He proposed (correctlyish) that continents move, but didn't have the data to get the mechanism right. I don't see where anecdote fits in this?

 

[xtifr]

There's also the issue of extraordinary claims. If I say I saw an opossum running through my yard, few would doubt me, and my anecdote can be treated as confirming evidence of the well-known fact that opossum territory includes my neighborhood. If I say I saw a unicorn running through my yard, though, there's good reason to be skeptical.

 

[C_Felix]

Anecdotal evidence is what starts science, to me at least.

Think of Mythbusters.
"I know a dude who did this to his car and that happened!"

Then questions begin.
Then tests begin to see if this does result in that.

But, hearing that some dude did this and resulted in that isn't evidence.

https://yourlogicalfallacyis.com/anecdotal

 

[Pyrts]

Anecdotes are studied in anthropology, but often in the context of "people believe this" and "this means this within a culture or a subculture." There is also a type of historical (to use a term brought up earlier) research called the "case study" -- here the studies are understood to be "this is what a person thinks/observes/believes in their life experience" but is not taken to be a universal truth.

Anecdotes are also studied by folklorists.

However, although some of the above would suggest biochemical or medical research, no one would accept anecdotes over thorough testing.

 

[Dinwar]

There is also a type of historical (to use a term brought up earlier) research called the "case study" -- here the studies are understood to be "this is what a person thinks/observes/believes in their life experience" but is not taken to be a universal truth.

Case studies in general are a relevant topic here. They might superficially resemble anecdotes, in that they aren't double-blinded, you generally only have one, you can't really control the variables, etc. That said, there is a significant difference: rigor. A well-constructed case study--be it a study of an individual in sociology, of a particular case of an illness in medicine, a specific fossil in paleontology, or a particular astronomical event in astronomy--will always be very precise, very thorough, and attempt to examine all aspects of the thing in question that can be examined in a systematic manner. This is a far cry from "My cousin Bubba said he once saw a flying saucer."

I'd like to know how PartSkeptic defined "anecdote" in their research. Right now I'm sort of stuck offering examples of scientific studies that might be mistaken for anecdotes; it'd be more useful if we knew what the real criteria were.