to be clear, i wrote:
"It's evidence that can only be used to suggest hypotheses for further studies. it's not evidence that can confirm anything. is that better?"
you go on to disagree with this, so you're claiming that anecdotal evidence can be used to confirm hypotheses.
now to your post...
No. I disagree with you. In fact, if the hypothesis is about an individual, it's the other way around. Controlled studies usually only tell us what the average effect was in a group, not what the effect will be in an individual. They might suggest the hypothesis that a drug will benefit an individual, but the hypothesis is confirmed by the experience of the individual. In fact, controlled studies might suggest that the drug will ineffective for an individual, and the drug, when actually administered to the individual, is beneficial.
statistical models are differentiated from deterministic models in that they have a random component to them. so every individual has a treatment effect and a random effect. no drug is 100% effective on everyone. clinical trials tell us what percentage of people benefit from it. if you find a way to predict, perfectly, that a drug is effective on a particular person, let me know. we'll make a killing at the race track.
a well run controlled clinical trial generally has the following components:
1. Randomization. This helps assure us that we don't have a biased sample of, say, zinc believers only in our sample.
Anecdotes. sample is self-selected (non-random) and is usually comprised of believers. Example: 1000 big foot sightings in the past 100 years. seems like a lot of anecdotal evidence. but this number is never compared to the 100's of millions of visits to forests around the world during the same time who see nothing. anecdotal evidence is self-selected and biased in favor of the claim.
2. Masking. this helps assure that the respondent isn't biased by knowing he took the zinc or a placebo.
Anecdotes. hopelessly biased in this way
3. Placebo controlled. We have a comparison group that we know didn't get the treatment. so if we end up with 90% of the people saying zinc worked we wouldn't get so worked up about it if 90% of the placebo group said the same.
Anecdotes: only looks at zinc group. so if you got 90% of the people in the zinc group saying it worked you would be pretty ecstatic about it. and could be completely wrong. again, anecdotes are hopelessly biased.
4. Safety. controlled trials assess safety. anecdotes don't. does no good to take a drug that's effective if it can kill you.
5. statistically adequate sample size. controlled trials tend to have lot's of people based on the number of people required to confirm the results. Anecdotes are convenience samples with as little as 1 person.
for these reasons, controlled trials can provide evidence that we can generalize to the population. anecdotes can
never do this.
this is the 4th time i've brought this up and you've never addressed it. it's a major part of the reason why anecdotal evidence is only good for hypothesis generation. and you disagreeing with this puts you at major odds with the vast majority of the statistics world (i've never seen, nor heard, of a any statistician or stat book saying one can use anecdotal evidence to confirm a claim. it's always to generate hypotheses).
But the division between hypothesis-generating and hypothesis-confirming evidence is artificial.
no, it isn't (see above)
There are good experiments that give us reliable evidence and lousy experiments that are completely worthless, and we'd learn more about the subject from anecdotes.
So? all anecdotal evidence is biased and therefore completely worthless. only 'lousy' experiments do the same. you have to judge the good from the bad experiments but you can dismiss all anecdotes as bad, in the context you're describing as bad here.
There are hypotheses that require carefully controlled studies to confirm, and hypotheses that don't. Aspirin was known to be an effective pain reliever long before randomized controlled trials were invented.
can you imagine how many 'pain relievers' were tried before they stumbled across aspirin?!?! how many people died from taking junk medicine based only on their own judgment and friends' anecdotes? this is not the, ahem, best approach.
Similarly, the hypothesis that beer can get you drunk was settled with finality in the Stone Age.
yeah, and birds learn to not eat monarch butterflies without clinical trials either. but lot's of birds got sick or even died beforehand. we're not in the stone-age anymore.
be interesting to hear how you get around all the bias issues associated with anecdotal evidence that i described above. again, it's the same reason why people are not convicted based on anecdotal evidence alone. there must be corroborating evidence.