• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

JREF Challenge Statistics

Mercutio said:
For the third time, then, I invite you to explain the error of my reasoning.
Click to expand...

Your reasoning against a possible meta analysis and seeing statistics is

Suppose we have a number of people claiming to be able to influence the outcome of a coin toss (or to predict it, if you prefer). If we were testing this claim against chance, we would have a simple binomial problem, testing against P = .50, and we would set up a suitable number of trials. If, though, we listen to the claimants, and adjust our tests accordingly, ...
Click to expand...

One always listens to claimants to help design the test, sure. But when it comes down to analyzing the observed data, for any test, one does something like

ztai chi = (actual hits-hits expected by chance) / stuff

and not

zmercutio = (actual hits-hits expected by what claimant says) / stuff

and one compares ztai chi, not zmercutio, to a standard normal distribution, for example, to get a probability that will help us assess the claimant's performance.

...we can save time. Let us take the extreme example in which claimants say they have complete control, and will always be able to determine the coin's face. We can test this very easily--just start flipping. There is a .5 probability that (by chance alone) any given person will fail after one toss. But that person can stop then. ...
Click to expand...

Besides the error above, your error here is saying the person can just stop then. Surely rules will be built into such a hypothetical preliminary test to prohibit optional stopping. Both claimant and tester agree on the number of trials beforehand, this is well known. Are you really claiming that JREF or another skeptical organization, when testing a claim of a statistical nature, will agree to test someone with only one or two, or even a few trials? That is doubtful to the extreme.

Your argument seems to be with the design of the test, not meta analysis.

... The trial is over. If, on the other hand, the person got the first one right, then there is a .5 probability on the next trial (again, by chance alone). With enough claimants, we may have some people who are getting 5, or 10, or more coins called correctly before making a mistake (this all by chance alone--of course, if they *can* influence the outcome perfectly, they will never make the mistake. And yes, I recall that I am taking the extreme 100% position here, but it extrapolates to lesser claims). Now...if we take these data and combine them, they will very likely be significant. Why? Because we quit the trials earlier when they failed earlier, artificially boosting the number of successes.
Click to expand...

I'm really not interested in "if"s and "may"s from hypothetical data from hypothetical tests being "very likely significant". Without seeing the actual data, one doesn't know if the combined data will be significant or not. That is one of the points of the exercise. One can always dream up situations for anything where something can possibly go wrong, that is not impressive. Actually seeing the data is another story.

Statistics on the table, please?

Let's start out easy: How many preliminary tests are done each year?, and How many of these tests are statistical in nature? How many of those taking the preliminary test have been tested for dowsing?
 
Last edited:
Thank you.

I am happy now; I know that it is not my reasoning that is wrong.

Your error is that you are continuing to look at each individual test, rather than at the cumulative nature of combining them in the meta-analysis. You also continue your misunderstanding of the reasoning behind alpha, but that is ok. The two z tests you compare are precisely the difference between laboratory parapsychology work (where the tc z reasoning determines alpha) and the challenge (where the merc z reasoning determines alpha and the cutoff). My hypothetical coin-flip example was merely a demonstration which combined the multiple challenge tests into one test, for ease of understanding. Your analysis here confirms that it is a sound argument.

My argument is with the test design, certainly; it is a design that is appropriate for the challenge, but inappropriate for a meta-analysis.

I do thank you for finally putting your reasoning on the table. My mind is at ease now.
 
Mercutio said:
Your error is that you are continuing to look at each individual test, rather than at the cumulative nature of combining them in the meta-analysis.
Click to expand...

I'm interested in a possible meta analysis, as well as looking at results from individual tests. Your reasoning why a meta analysis would be inappropriate on hypothetical tests is not convincing IMO for reasons discussed.

You appear to believe I'm saying let's get the data and do a meta analysis no matter what. I'm not. I'm saying let's possibly consider it to test the incredible notion of skeptics making results negative, but regardless, let's see the data being made more easily available because that would be of interest. The latter is the main point here.

The two z tests you compare are precisely the difference between laboratory parapsychology work (where the tc z reasoning determines alpha) and the challenge (where the merc z reasoning determines alpha and the cutoff).
Click to expand...

I didn't say anything about determining alpha. Typically alpha is set to .001 for the JREF preliminary test. The z's I posted have to do with p-values, not alpha.

-What % of claimants are dowsers? It is said that this is the most common claim, but it would be best to get actual numbers if possible, for example.

-How much does it cost, on average, to get tested, for both the skeptical organization and for the person being tested? This is an interesting question too I believe. Don't care about specific numbers here, just ballparks for an average dowsing test for example.

Not really sure why there is even any debate about such an approach. But I'm sure I'll find out the grave errors I've made in next few replies! :D
 
Last edited:
From page 1, Mercutio wrote:

What percentage of the applicants have a claim that is statistical in nature?
Click to expand...

Great question! I'd like to know that and other details. Maybe someday we will. :D
 
Look through KRAMER's threads--perhaps the first question should be "what percentage of applicants have a claim that is comprehensible?". After all, if we limit ourselves to those who can put a meaningful sentence together, we are artificially narrowing the population!
 
Mercutio said:
Look through KRAMER's threads
Click to expand...

I looked through Kramer's threads, and didn't find the answer. Maybe I missed it, or maybe it wasn't addressed.

As I opined

The JREF has been making strides in making information on "the claims received, the correspondences exchanged between the JREF and the applicant, and subsequent protocol negotiations and test results" electronic, but not, as far as I can tell, the numerical results of past preliminary tests, which in my opinion are just as, if not more, interesting and relate more to the science.
Click to expand...

It would be nice, ideal perhaps, if in the future there was a webpage one could go to and actually see all of this information in an organized format (as far as I'm concerned, names of applications can be omitted; I just want to see what matters in terms of the design and results), so one wouldn't have to stop life and fly to Florida, or search hundreds of posts (of only recent tests it must be noted) for something that may or may not be there.
 
T'ai Chi said:
It would be nice, ideal perhaps, if in the future there was a webpage one could go to and actually see all of this information in an organized format (as far as I'm concerned, names of applications can be omitted; I just want to see what matters in terms of the design and results), so one wouldn't have to stop life and fly to Florida, or search hundreds of posts (of only recent tests it must be noted) for something that may or may not be there.
Click to expand...

We've already been through this. That costs money. Are you willing to contribute to make this possible, or do you just want to sit on your ass, making demands?

Where does the claimant say anything about what he expected to get?

I can't find where he says this. Perhaps you could show us where?
 
Mercutio said:
I don't do lists.

I could put my request in limerick form...

Is the problem with you, or with me?
I've asked you, times one, two, and three--
Please show my mistake--
That's all it would take--
So put up or shut up, T'ai Chi.
Click to expand...

:dl:

You are the master! I bow before you.
 
I don't remember if anyone has specifically mentioned this in this thread, but I think the thing that is hanging Tai Chi up is the fact that the JREF tests are not conducted as academic studies might be. If he is really looking for a more in depth understanding of the academic study of the paranormal, his queries would be much better directed at the parapsycology department of a major university. It is important that he understand that the JREF is not in any way engaging in studies of paranormal phenomena, nor are they interested in doing so. There are people already doing that (some actually responsibly).

The JREF's mission statement is clearly laid out on this website:
"Its aim is to promote critical thinking by reaching out to the public and media with reliable information about paranormal and supernatural ideas so widespread in our society today."
Click to expand...

The $1,000,000 challenge is simply a tool to promote interest in the foundation. It's adversarial nature is designed to attract controversy so as to better publicise the foundation and its work. While the challenge is the most visible part of the JREF's activities, and the one we talk most about on these forums, I suspect Randi would tell you that the work he does at seminars, in classroom, and as a consultant (the stuff that is invisible to the majority of the general public) is far more important than demonstrating that Achau Nguyen cannot actually send his thoughts to another person.
 
Gr8wight said:
I don't remember if anyone has specifically mentioned this in this thread, but I think the thing that is hanging Tai Chi up is the fact that the JREF tests are not conducted as academic studies might be. If he is really looking for a more in depth understanding of the academic study of the paranormal, his queries would be much better directed at the parapsycology department of a major university. It is important that he understand that the JREF is not in any way engaging in studies of paranormal phenomena, nor are they interested in doing so. There are people already doing that (some actually responsibly).
Click to expand...

Consider the fact that T'ai Chi has repeatedly derided JREF and Randi for not doing science at all.

Consider the fact that T'ai Chi claims that science can't be done outside the lab.

Consider the fact that T'ai Chi shows no interest in paying for the extra work it would take to meet his demand of access to the files.

You take a guess what T'ai Chi is up to.
 
T'ai Chi,

Are you willing to contribute to make this possible, or do you just want to sit on your ass, making demands?

Where does the claimant say anything about what he expected to get?

I can't find where he says this. Perhaps you could show us where?
 
CFLarsen said:
Consider the fact that T'ai Chi has repeatedly derided JREF and Randi for not doing science at all.

Consider the fact that T'ai Chi claims that science can't be done outside the lab.

Consider the fact that T'ai Chi shows no interest in paying for the extra work it would take to meet his demand of access to the files.

You take a guess what T'ai Chi is up to.
Click to expand...

Oh, I have never had a doubt. The T' is short for troll.
 
Gr8wight said:
Oh, I have never had a doubt. The T' is short for troll.
Click to expand...
Perhaps. But T'ai Chi does not merely want to troll. Although there is definitely an issue of clamoring for attention, his primary goal is to cast as much doubt about skepticism and skeptics as possible.

Not valid criticism, which would be admirable, worthwhile and welcome, but an ongoing campaign to smear. Cheap pot shots and snide remarks, after which he runs away from real debate are his trademarks.
 
You must log in or register to reply here.

Back
Top Bottom