Wolfman
Chief Solipsistic, Autosycophant
Okay, let me start by saying that I'm a fan of James Randi, and a huge supporter of what he does. But I consider it important to regard his statements and claims with an equal degree of skepticism to what I would regard the claims of anyone else.
In the latest SWIFT newsletter, he has this to say about dowsing tests conducted in Australia as part of his challenge:
1) He begins by defending the numbers he publicly presented by saying that, "the dowsers themselves – not we – insisted that the results of all their attempts should be combined". However, that in no way meant that he himself could not release more complete figures -- figures which would have presented a more accurate picture of the actual results. In fact, if it had turned out that splitting the findings would have made the results look more questionable, I'm sure he would have done just that. However, in this case, since combining the results made it look less credible, he only cited those results, and did not present more specific results on each individual study until challenged to do so.
I find it difficult to see how this is any different from what those in the wonderful world of woo will do -- to present results in the manner which most favors their argument, even if (or particularly if) another more detailed explanation of those facts may weaken their claim.
2) Mr. Randi also says, "The significance of that number is critical to making any conclusion, and it was not high enough to cause any wonder", and goes on to a comparison with flipping coins, and data mining. Now, I appreciate his point -- but if this is, in fact, the case, why did he even bother conducting this test? Even if the results had shown a significant result to support the dowsers' claims, would he not have responded by simply saying that it was not statistically significant, and the test needed to be done again?
There are constant complaints that Randi's million dollar challenge is rigged, and would this not seem to support such a claim? He agreed to a test in which he stated in advance that he would pay a reward if the dowsers were able to meet a certain criteria; but then he turns around and states that such results would be essentially meaningless with such a limited number of samples to be drawn from.
Please understand, I'm no defender of woo. But if James Randi is going to appoint himself as the debunker of fraud, it seems to me that he should be held to at least an equal standard (if not higher) than that to which he holds those he criticizes. In this case, I really have questions as to whether he does that or not. While I admit that my understanding of this particular case is limited (which is why I am writing this, to gain more explanations and understanding of the actual situation), from what has been written, it seems to me that in one single article he has managed to demonstrate a willingness to print only those results that most benefit his own arguments, and to weaken the claim that his million dollar challenge is fair.
In the latest SWIFT newsletter, he has this to say about dowsing tests conducted in Australia as part of his challenge:
I have two problems with Mr. Randi's 'defense' of his own claims in this regard, based on his own statements and arguments.There are several facts about which this gentleman – and Simon Welfare, who I would have thought might have contacted me about the matter – are unaware. First, the dowsers themselves – not we – insisted that the results of all their attempts should be combined; they hoped to make the strongest possible case, and had agreed to equally divide the A$50,000 prize if they won it. In such matters, the applicants’ needs are always met when they cannot compromise the protocol or the results of the tests. Second, because of the perceived anomaly, those same dowsers were once again tested, this time in Perth, Australia, very shortly after the tests cited, because they felt they would do better in that location. Though I knew that this additional test would be done, and I approved that process, I had to return to the USA, so could not be present. They failed spectacularly, obtaining results that almost exactly mirrored the Sydney tests in reverse, so that the overall testing procedures showed null results.
However, Mr. Hart-Davis, quoting the Fairley-Welfare book, describes the 22 per cent result (the chance expectation was 10%) as being “well above chance,” without taking into account or even citing the number of trials conducted. The significance of that number is critical to making any conclusion, and it was not high enough to cause any wonder.
In any case, it is not proper to select out any section, portion, or aspect, of a set of tests to establish a significant result, unless, of course, the intention to do that is announced in advance. Though that method may be used for preliminary procedures that will help to design a final confirmatory test, it is not applicable unless it's so announced. Tossing a coin, one cannot select out a sequence of six “heads” to show a perfect score, if the goal was to obtain “heads.” That’s called, “data-selecting,” and it is the one most often-invoked no-no in amateur testing procedures…
1) He begins by defending the numbers he publicly presented by saying that, "the dowsers themselves – not we – insisted that the results of all their attempts should be combined". However, that in no way meant that he himself could not release more complete figures -- figures which would have presented a more accurate picture of the actual results. In fact, if it had turned out that splitting the findings would have made the results look more questionable, I'm sure he would have done just that. However, in this case, since combining the results made it look less credible, he only cited those results, and did not present more specific results on each individual study until challenged to do so.
I find it difficult to see how this is any different from what those in the wonderful world of woo will do -- to present results in the manner which most favors their argument, even if (or particularly if) another more detailed explanation of those facts may weaken their claim.
2) Mr. Randi also says, "The significance of that number is critical to making any conclusion, and it was not high enough to cause any wonder", and goes on to a comparison with flipping coins, and data mining. Now, I appreciate his point -- but if this is, in fact, the case, why did he even bother conducting this test? Even if the results had shown a significant result to support the dowsers' claims, would he not have responded by simply saying that it was not statistically significant, and the test needed to be done again?
There are constant complaints that Randi's million dollar challenge is rigged, and would this not seem to support such a claim? He agreed to a test in which he stated in advance that he would pay a reward if the dowsers were able to meet a certain criteria; but then he turns around and states that such results would be essentially meaningless with such a limited number of samples to be drawn from.
Please understand, I'm no defender of woo. But if James Randi is going to appoint himself as the debunker of fraud, it seems to me that he should be held to at least an equal standard (if not higher) than that to which he holds those he criticizes. In this case, I really have questions as to whether he does that or not. While I admit that my understanding of this particular case is limited (which is why I am writing this, to gain more explanations and understanding of the actual situation), from what has been written, it seems to me that in one single article he has managed to demonstrate a willingness to print only those results that most benefit his own arguments, and to weaken the claim that his million dollar challenge is fair.