Split Thread How are MDC protocols designed and carried out? (Split from FAQ thread)

Bryan,

1. Why shouldn't the JREF turn away claims that they regard as time wasting. I mean, really.... If you have a power that defies explanation, but is barely detectable above chance, how do you know you have the power?
2. Do you have any evidence that the JREF is turning away such claims?

Tom
 
That's EXACTLY what I'm saying! :)

I think that James Randi was a bit unfair on that TV show, when he demanded that the "psychics" pass TREMENDOUSLY difficult tests. Of course, if they themselves had made such claims before the start of the show, then that's their own problem; one could hardly criticize Randi for demanding that they pass a test which they themselves claimed to be able to pass.

But what if their only claim is that they can "sense" the result of a coin flip only 55% of the time? Is James Randi (or an "officer" of the JREF) going to throw them out of the office, or are they going to give them a legitimate test for the one million dollar prize, as long and difficult as that would obviously be? :confused:

The fact that the JREF apparently doesn't make the answers to these questions abundantly clear beforehand (before somebody actually walks in the door and submits a proposal for a test) is what bothers me...
And I call baloney on this. You accuse Randi of cooking the numbers yet you present none. In fact, by your own admission you don't recall the specifics. You saw the tests. What was the actual accuracy required an dwhat is the accuracy you feel should have been required? And how did you calculate those numbers. Until you show your math you are just going on an emotional reaction that has no value.
 
That's EXACTLY what I'm saying! :)

I think that James Randi was a bit unfair on that TV show, when he demanded that the "psychics" pass TREMENDOUSLY difficult tests. Of course, if they themselves had made such claims before the start of the show, then that's their own problem; one could hardly criticize Randi for demanding that they pass a test which they themselves claimed to be able to pass.

Since that's what happened, then your other points are moot. The protocol is agreed with the claimants, based on what ability is being claimed. The JREF is not unilaterally imposing anything.
 
But what if their only claim is that they can "sense" the result of a coin flip only 55% of the time? Is James Randi (or an "officer" of the JREF) going to throw them out of the office, or are they going to give them a legitimate test for the one million dollar prize, as long and difficult as that would obviously be? :confused:

Check out the interminable discussions with Pavel Ziborov on this forum. He had a claim similar to that, which was going to be just above the odds for chance, but had trouble pinning down exactly what would demonstrate his ability.
 
The unofficial rule of thumb seems to be that applicants should be required to beat chance odds of around 1:1000 in the preliminary test. That means that for every 1000 applicants JREF tests one would be expected to pass even if there is no such thing as a paranormal ability, which seems a reasonable place to put the bar to me. But as has already been pointed out the claims tested vary considerably, making hard and fast rules impossible to set in advance.

Here's a typical protocol from the archive:

http://www.internationalskeptics.com/forums/showthread.php?t=118952

Perhaps Mr Shelton can tell us if he considers the protocol and success criteria used to test this applicant acceptable, and if not what exactly his problem with them is.
 
Last edited:
The fact that the JREF apparently doesn't make the answers to these questions abundantly clear beforehand (before somebody actually walks in the door and submits a proposal for a test) is what bothers me...

No, JREF *negotiates* the answers to these questions beforehand (before the applicant agrees that the protocol is fair and the test something they can do -- and only *then* is the test scheduled).

Why does this bother you?
 
Last edited:
Bryan,

1. Why shouldn't the JREF turn away claims that they regard as time wasting. I mean, really.... If you have a power that defies explanation, but is barely detectable above chance, how do you know you have the power?

It seems perfectly reasonable to me that the JREF would first do a "preliminary test" (as "Pixel42" talked about in a later post this morning) which would require approximately a 1 in 1,000 chance (or less) of passing purely by chance. If the applicant passes it, a somewhat more rigorous test could then be part of the "formal test", which determines the fate of the million dollar prize. The first "preliminary test" would be the obvious way to avoid "wasting time" by applicants who don't have any real ability.

2. Do you have any evidence that the JREF is turning away such claims?

No, but I don't know that much about how these tests have been conducted. It troubles me that from talking to many of you folks here on this forum, it's been like pulling teeth to get some of you to speak about the tests clearly and concisely! ;)
 
I think the JREF would accept a test where someone could predict a coin flip 55% of the time. This would take a tremendous number of trials. Assuming that the applicant never gets tired, this could probably done in a day. Ususally though, psychic powers tend to be exhausting after just a few trials, so this would require weeks or months. Keep in mind that the applicant is responisble for all the costs of the test. Eventually, the people administering such a long long test will want to be paid for their time. If the applicant never gets tired or if he or she is willing to pay for the time of the administrators, I would expect the JREF to accept a success rate of 55% in a coin flip test.

I don't speak for the JREF, though.

Ward
 
No, but I don't know that much about how these tests have been conducted. It troubles me that from talking to many of you folks here on this forum, it's been like pulling teeth to get some of you to speak about the tests clearly and concisely! ;)

To find out how the tests were conducted you should read the individual threads about each test, they are all different.

No single description of a test protocol exists. Even in cases where more than one applicant claims the same ability the details of their ability may vary,and the conditions under which they are able to demonstrate it are different. You'll find several different protocols mentioned for telepathy, remote viewing and dowsing, for example. The applicant defines his ability, and makes the initial suggestion of the test protocol. JREF then accepts, or suggests alterations to the protocol before an agreement is reached.
 
Last edited:
Check out the interminable discussions with Pavel Ziborov on this forum. He had a claim similar to that, which was going to be just above the odds for chance, but had trouble pinning down exactly what would demonstrate his ability.

And what exactly does "pinning down" mean in this context? You mean PROVING his ability? Or simply explaining what his ability is supposed to be?
 
The unofficial rule of thumb seems to be that applicants should be required to beat chance odds of around 1:1000 in the preliminary test. That means that for every 1000 applicants JREF tests one would be expected to pass even if there is no such thing as a paranormal ability, which seems a reasonable place to put the bar to me. But as has already been pointed out the claims tested vary considerably, making hard and fast rules impossible to set in advance.

I'm becoming more and more puzzled that so many of you talk so much about the difficulty of setting "hard and fast rules". I don't doubt that watching-over the applicants and setting rules to maintain honesty and prevent larceny can be difficult, but merely calculating the odds of success through the laws of probability should be a relatively simple matter.
 
I think the JREF would accept a test where someone could predict a coin flip 55% of the time. This would take a tremendous number of trials. Assuming that the applicant never gets tired, this could probably done in a day. Ususally though, psychic powers tend to be exhausting after just a few trials, so this would require weeks or months. Keep in mind that the applicant is responisble for all the costs of the test. Eventually, the people administering such a long long test will want to be paid for their time. If the applicant never gets tired or if he or she is willing to pay for the time of the administrators, I would expect the JREF to accept a success rate of 55% in a coin flip test.

The only thing you haven't talked about here is HOW MUCH statistical significance you think should be required in such a successful test; but other than that one missing factor, I think you wrote a very lucid and sensible post! :)
 
I don't doubt that watching-over the applicants and setting rules to maintain honesty and prevent larceny can be difficult, but merely calculating the odds of success through the laws of probability should be a relatively simple matter.
And yet when someone gave you an actual example:

There was a recent (at least in terms of MDC tests) test of a woman who claimed she could make someone have to urinate. What are the odds for that test?

you replied:

I have no idea what they are, or what they should have been.

I have no idea how they should have been figured out.

In cases where a protocol is agreed where the odds of chance success can be calculated (the majority of cases), a success criteria of beating odds of around 1:1000 is typically used. But such a calculation is not always "a simple matter"; in some cases it is impossible, so another way of reaching agreement on the success criteria must be found, which is why no hard and fast rules can be set.
 
Bryan,

And yet when someone gave you an actual example:...

And in that case, it was only 5 out of 10 correct to move beyond the preliminary steps. How, again, is that unfair, when the claimant agreed to all of the protocols?
 
The fact that the JREF apparently doesn't make the answers to these questions abundantly clear beforehand (before somebody actually walks in the door and submits a proposal for a test) is what bothers me...
What will you do if the JREF changes nothing and continues on without making answers to your questions abundantly clear beforehand to your satisfaction. Will this continue to bother you for the rest of your life? If so, how will you possibly deal with this? Is your level of botheredness something psychological therapy might be able to help with? How will you cope?
 
Last edited:
Bryan, if you have a specific paranormal ability you would like tested, just state your ability clearly and succinctly. Forums members would be pleased to give their judgement on what they would consider success to be and JREF itself would be very pleased to help you work out a test protocol.

Here's an example:

The PowerBall is a lottery game played in the United States in which 5 white balls are drawn without replacement from a set of 59 and 1 red ball is drawn from a set of 35. The odds of correctly guessing 3 white balls plus the red ball are 1 in 12,245.

Joe Lucky states that he can pick the 3+1 combination using his innate paranormal ability. JREF arranges a preliminary test and Joe successfully picks the winning combination while being observed.

However, even though the odds are 1 in 12,245, it is still possible for a person to choose 4 correct numbers through random chance. After all, hundreds of people do just that every drawing.

The question for Joe becomes how often and consistently can he pick his winning sets? Suppose he can pick a 3+1 winner for each drawing held over the next two weeks, i.e. 4 independent drawings. That becomes a much more solid demonstration of ability than a single instance and could be part of a protocol for winning the MDC.

The question for you becomes how can JREF possibly devise a "hard and fast" set of rules that cover every possible paranormal claim? Is dowsing the same as picking lottery numbers? Is being a card guesser the same as being a pee-starter? Is a premonition that an airliner will crash the same as picking up psychic vibes that the hot chick at the bar digs you?

If you have an ability, spit it out and let people help you figure out how to demonstrate it.
 
Last edited:
Bump for Bryan, who apparently missed this:

The fact that the JREF apparently doesn't make the answers to these questions abundantly clear beforehand (before somebody actually walks in the door and submits a proposal for a test) is what bothers me...

No, JREF *negotiates* the answers to these questions beforehand (before the applicant agrees that the protocol is fair and the test something they can do -- and only *then* is the test scheduled).

Why does this bother you?
 
Point 1:

Having re-read this thread from the beginning, I note that there has not been any discussion of this:

[...] When random choices are a clear and obvious element of such a test (as many of them truly are) [....]

Mr. Shelton, could you please explain what you mean by the highlighted statement?

Do you, for instance, mean that the applicant makes random choices?

That the JREF makes random choices?

That during a test, some random elements are introduced by the applicant or the JREF or just good old Ma Nature?

Point 2:

In any case, I am confused about why it is unfair to set a standard that in most cases is much lower that the applicants claim to be able to meet.

I say, "I can detect Substance Y, by whatever method, infallibly, 100% of the time. Never miss. Never. Not even when I have a hangover and a bad cold. Never."

JREF says, "That's fine. But we will be generous to you. We will allow 1 miss per 10 trials, for 10 10-trial sequences." *

How is that unfair? Does the answer relate to Point 1, above?


(My very poor statistical skills say that should be odds of 1 in 1,000. If I am wrong, someone please correct me. However, I think the idea should be clear.)
 
[snip]In cases where a protocol is agreed where the odds of chance success can be calculated (the majority of cases), a success criteria of beating odds of around 1:1000 is typically used. But such a calculation is not always "a simple matter"; in some cases it is impossible, so another way of reaching agreement on the success criteria must be found, which is why no hard and fast rules can be set.

I recommend that the JREF _not_ test people who claim to be able to make people have to urinate. Stick to matters of a more purely probabilistic nature, where more time is spent merely determining the odds of numerical success, and far less time is spent making sure that the rest of the test is fair.
 
Bryan, [snip] And in that case, it was only 5 out of 10 correct to move beyond the preliminary steps. How, again, is that unfair, when the claimant agreed to all of the protocols?

I don't think it's unfair (although I haven't looked closely at the numbers of that test).
 
Last edited:

ISF - Join now!

Every member here is approved by hand. No bots, no spam, just people who care about evidence and honest debate.

Membership is free!

Create your free account

Back
Top Bottom