From the other thread:
Friends,
It is interesting to note that no one on this list chose to acknowledge that the Shang comparison article is a classic example of data dredging.
Data dredging refers to a specific activity. Large amounts of information are collected, then the parameters of interest are chosen based on which fall furthest from the mean. Homeopathic studies are rife with examples - the childhood diarrhea studies where the very specific outcome just happens to coincide with the one point where statistically significant differences were found, the allergic rhinitis studies where the parameter of interest is suddenly changed at the end of the study when it is discovered that the parameter chosen a priori is not statistically significant, etc.
Shang et. al. is not an example of data dredging. The parameters were chosen a priori and the only information collected relates to those parameters. And they were those necessary to minimize chance and bias (randomized, controlled, double-blind), and to make the comparison fair for homeopathy (matched on the basis of disorder and outcome).
You have demonstrated (by failing to provide a single example otherwise) that homeopathy is used in situations where improvement is expected in the absence of effective treatment. And the tendency is to measure 'soft' outcomes. It would be unfair to compare that to medicine trials which achieve differences in hard outcomes in situations that would otherwise be unremitting. It would be like comparing the mortality rate in a group that consists of frail, elderly people (homeopathy) with a group that consists of a general population (medicine). What would be fair is to select out the frail, elderly people from the general population, and to take a sample from them for comparison.
The only way that Shang would have gotten a negative result against homeopathy is by selecting the parameters he established...and because he previously contacted the Lancet and formally told him that his hypothesis was to prove that homeopathic medicines have no effect, he self-selected his parameters and his data to prove it.
That would not be possible as they
did not have the information from other parameters available to them. The only information they collected was that based on their pre-selected parameters. Your criticism makes no sense.
Is it interesting that he chose 98, not 100, subjects as his minimim definition for inclusion...just so that he could include a headache trial with a negative result.
At least one of you above acknowledged that s/he wondered why only 1 of the 3 studies with 300+ subjects was included in the trial. Linda asserted that the study in the Journal of Alternative and Complementary Medicine was granted a high rating than the trial in PEDIATRICS. That speaks volumes about the rating system that Shang used.
No it doesn't. It says that while there is a trend for higher quality studies to be published in higher impact journals, there is variation on this characteristic. The rating system that Shang et. al. used is a standard system that is commonly used.
Anyway, because the 3 trials were conducted with the same design, and even though this pooled data was published in a major pediatrics journal, Shang chose to ignore that (whooops).
Not a 'whoops'. To have done so would be unjustifiable. Pooling the data leaves the biases intact -
the very biases that Shang et. al. are trying to avoid with their choice of larger trials. It would have been a gross error to include those studies.
I loved Dr. Peter Fisher's response to the Shang work:
http://ecam.oxfordjournals.org/cgi/content/full/3/1/145
His assertion below is worthy:
"Shang et al. (1) state that ‘eight trials of homoeopathic remedies in acute infections of the upper respiratory tract ... indicated a substantial beneficial effect ... sensitivity analyses might suggest that there is robust evidence that the treatment under investigation works. However, the biases that are prevalent in these publications, as shown by our study, might promote the conclusion that the results cannot be trusted’. They state that eight studies is too few to question their conclusion about the whole set of publications. Their conclusion about the whole set, however, was also based on eight studies. Is eight enough for a conclusion or not? Or does it depend on what that conclusion is?"
It depends on what that conclusion is. It's the difference between ruling-in an alternative explanation and ruling-out all other alternative explanations. Homeopathy is without any other evidentiary support. In order to prove that it is effective, all reasonable alternative explanations must be ruled-out. Ruling-in the possibility that the effects are due to placebo means that homeopathy studies fail to provide proof of effectiveness.
And yet, Shang's final conclusion are based 8 homeopathic studies and 6 allopathic ones. Remember...the mission of this article was to compare MATCHED studies. Please someone tell me (us) how the final studies were matched in any way? Was one of the allopathic studies a test of a drug for weight-loss (as was done with the homeopathic studies)? Curious minds want to know...
That is wrong. The purpose was to compare matched
groups. Your criticism is irrelevant.
Come on...you folks claim to be good in evaluating data, and you've barely started to provide an adequate review of this comparison article. Go for it...
So far I have demonstrated that all your criticisms are invalid, erroneous or irrelevant. I'm happy to continue going for it. Next?
Linda