Okay, first the number of chemicals, whether it's 100,000 or a zillion, has no relevance to the analysis.
Did you read the paper that I referenced? It explains why the number of potential candidates for a homeopathic treatment is very important. If I want to test whether or not water that has come in contact with a specific chemical can help my cough, there are thousands of chemicals to be tested, and so I would be performing thousands of tests to try each one.
The only thing that is of interest with respect to the testing of homeopathic medicines in controlled trials is the same thing that is of interest with respect to testing non-homeopathic medicines in similar trials: Do they work better than placebos?
Exactly. And how do we know if they work better than placebo (because there are always going to be differences between the treated and the untreated groups, especially if there's bias)? We look for a difference that is so large that we'd only expect to see it one time in twenty due to chance, in the absence of bias. We don't know for sure that it works, but there's only a 5% chance that it doesn't. Except that when we do a thousand tests, 50 of them are going to be labelled as "positive" due to chance.
Second, to assume that for every homeopathic trial that is a true positive, there will be 10,000 trials that are false positives has no basis in reality.
You have mixed up what are assumptions and what are conclusions. My assumptions were that any chemical is a potential candidate for effect (and there are at least 100,000 chemicals present on earth), and that any medical condition is a potential candidate for relief (and there are at least 100 different medical conditions in humans). What is unrealistc about those assumptions?
I simply applied a straightforward 2x2 analysis to those assumptions and reported the result. If you think the result is unrealistic, then what numbers would you use?
I believe what you're thinking of are medical tests, such as mammograms, that have a high ratio of false positives to true positives.
No, I wasn't trying to find a comparable medical test and estimating the result from that.
However, false positives are no more likely in the testing of a specific homeopathic medicine (such as cough medicine) than they are in the testing of a specific non-homeopathic medicine.
That was the whole point of the exercise!!! How do we figure out how likely it is to have a false-positive? We calculate it using a 2x2 table. We don't just assume it's some arbitrary number.
Third, the notion that negative test results don't matter flies in the face of all statistical theory.
No it doesn't. Whether or not
my mammogram represents a true-positive or a false-positive has nothing to do with whether or not the woman ahead of me in the line has a negative mammogram.
Again, the same thing is true of the testing of homeopathic medicines as it is of non-homeopathic medicines: All things being equal (test quality, number of test participants, etc.), the more often a medicine outperforms a placebo in a controlled trial, the more probable it is that it is an effective medicine.
NO!!! The probability that it is an effective medicine depends upon the pre-test probability (in the form of odds) and the likelihood ratio. When you talk about the results of a clinical trial, all that it determines for you is the likelihood ratio.
You cannot assume that the post-test probability is the same as the likelihood ratio!!! That is a meaningless assumption!! You are essentially saying that "a*b=b" no matter what the value of "a" is.
The whole point of
this post was that if you apply the results of the trial to the
pre-test probability like you are
supposed to do in order to make a valid conclusion, the
post-test probability is dramatically different when you are looking at studies of homeopathic treatments. The reason that this isn't usually mentioned specifically in conventional medicine trials is because the pre-test probabilities are usually close to 1 - i.e. they represent a special case.
Linda