FutileJester said:
I don't suppose you have any links for those effect levels?
This is going back a bit, but hey, I need to sleep!
For FutileJester.
Think about the two meanings of the word "significant". In common usage it often means something like "big enough to be important". However, in medical statistics it means something different, something like "only a small chance that the difference seen between two groups was caused by chance".
In the latter context, you usually specify approximately how likely it is that the effect was pure chance. p<0.001 is the goodie, only one chance in a thousand that the difference seen was only chance. p<0.01 means that there's a one in a hundred chance that the difference is due to chance, and p<0.05 means that there's a one in twenty chance.
p<0.05 is the lowest significance that is accepted for scientific purposes, and some authorities don't think that's much cop and only start listening at p<0.01.
You see, if you have two groups with NO difference between them, really, and run the study 20 times, one of these runs would be expected to show up significant at p<0.05. So, if you weren't entirely honest, you could chuck the 19 in the bin and publish the 20th.
How many groups of ten cat owners did you have to ask before you found the one where eight of them said their pussies preferred Whiskas? This is the constant suspicion with a lot of the homoeopathic studies.
There's also the matter of what's called "multiple endpoints". If you measure 20 different variables between the groups, then again even if there was no actual difference at all, you'd expect one of the 20 variables to be "significant" at p<0.05.
This is how apparently properly-conducted studies can appear statistically "significant", even though there was no real difference between the groups. Then of course people can quote the study to support an assertion that the remedy under test was found to have a "significant" effect.
But then think about the common meaning of "significant" - big enough to be important. This is what people sometimes think that last statement means, when it doesn't mean that at all.
If you look at any of these published studies that claim to have found a "significant" effect of a homoeopathic remedy, you'll find that most of them are only significant at the p<0.05 level (where did they hide the other 19 studies?), and most importantly, that the magnitude of the alleged effect is tiny.
The study on the diarrhoeic children is a good case in point. I'm quoting from memory here, but I think it was something like one fewer loose stool in the treated group on day four of the study, but no difference between the groups on day three or day five. This could be an example of multiple endpoints, test every day then trumpet the "significant" result on the one day the yarrow sticks fell in your favour. Or another way of looking at it is that the diarrhoea seemed to stop a few hours earlier in the group taking the homoeopathic remedy. What they didn't mention of course is that the standard medical treatment would have stopped the diarrhoea dead in its tracks on day one or two.
The Reilly studies on asthmatic children are similar. The kids filled out a lot of questionnaires about how many times they coughed each day and stuff like that, and by trawling through the data they managed to find some apparent "significant" differences in the homoeopathic group. But if you look at the magnitude of the alleged effect, from the point of view of an asthmatic kid, it's hardly worth having. If you weren't actually counting the coughs, you probably wouldn't notice any difference.
There is no homoeopathic remedy which has an obvious, self-evident effect on anything, and no disease process homoeopathy can point to and say, thanks to our medicine, this scourge is conquered.
Homoeopaths are fond of quoting the "miracle cure" cases. They write them up as individual anecdotes and boast about them all the time. The epileptic dog that never had another fit after two doses of salicea 30. The woman whose cancer went away after a course of homoeopathic salt. The dog with the respiratory infection which miraculously recovered after a 1M dose of pulsatilla.
The thing is, there is only one of each of these. No second case of sudden recovery in the same circumstances has ever been reported. Doctors and vets all know very well indeed (and are very thankful for) the fact that patients sometimes get better on their own for no real reason anyone can see. The homoeopaths, however, like to imagine that every spontaneous recovery is due to their medicine.
But then when you get down to doing actual clinical trials with groups of patients, the miracle cures vanish, and there's never an effect which is obvious to the "naked eye" as it were. Statistics, yes, but only demonstrating tiny effects. Look at any of the studies which have been linked to as supporting homoeopathy, and you'll see what I mean. If you were the patient, would you really be grateful for such a marginal improvement?
Rolfe.
