[Split Thread] How are MDC protocols designed and carried out? (Split from FAQ thread)

[Bryan Shelton]

"I recommend that the JREF _not_ test people who claim to be able to make people have to urinate. Stick to matters of a more purely probabilistic nature, where more time is spent merely determining the odds of numerical success, and far less time is spent making sure that the rest of the test is fair."

So, telekinesis, clairvoyance, and a host of other paranormal abilities shouldn't be testable under your ruling?

Telekinesis would be a simple and fine thing to test, if anybody ever actually made such a claim. It sounds to me like a test of clairvoyance is much more like to fall into the category of testing probability, like I said before.

 

[Pixel42]

Why don't they specify (and reveal) the exact terms beforehand? Why do they turn this into a guessing-game?

This has been explained to you multiple times. I really don't see how anyone could make the explanation any clearer.

Telekinesis would be a simple and fine thing to test, if anybody ever actually made such a claim

http://www.internationalskeptics.com/forums/showthread.php?t=97889

If you have a suggestion as to how exact terms could be specified in the rules which would be applicable to both this claim and the Patricia Putt claim I linked to earlier by all means share it.

 

[wardenclyffe]

Telekinesis would be a simple and fine thing to test, if anybody ever actually made such a claim.

What are the odds that someone could successfully move an item telekinetically? How would you figure that out?

Ward

ETA: Pixel42 beat me to it (as usual). What are the odds?

 

[Bryan Shelton]

"Perhaps even more importantly, such odds should remain the same for ALL such tests."

All tests of the same claimed ability *do* have, *must* have the same odds.

You appear to want the same odds regardless of what the ability is, and that is not possible.

If the claimed ability is to be able to levitate unassisted 12 feet off the ground indefinitely, the odds approach unity against. The odds of knowing a 2 digit number, randomly chosen are 1:100 against. What would be fair in the first case would be grossly unfair in the second.

You appear to be confused. I thought it was obvious that I was referring specifically to tests of a purely probabilistic nature; examples would be someone who claims to "sense" the next drawing of a card from a shuffled deck; which side of a coin is going to occur after being "flipped", etc. If somebody comes along who claims to be able to levitate, or fly like a bird, then that's a much simpler claim to test, and doesn't involve probability at all.

 

[Bryan Shelton]

What Bryan Shelton seems to be saying is that levitation should not be tested. Only tests that are essentially guessing games should be allowed.

Please correct me if I'm wrong.

I don't really have a problem with making simple physical tests. I was only momentarily thrown for a loop when somebody talked about testing a person who claimed to be able to make somebody have to urinate!

 

[Bryan Shelton]

"Why don't they specify (and reveal) the exact terms beforehand? Why do they turn this into a guessing-game?"

This has been explained to you multiple times. I really don't see how anyone could make the explanation any clearer.

I think there's a great deal of misunderstanding going on in this thread. What I was specifically referring to in my quote above has to do with stating the EXACT ODDS that a person without any special abilities at all would have in passing both the "preliminary test", and the "formal test". One person (I'm forgetting his or her name) has talked about suggested probabilities for those tests, but hasn't responded to the question I gave him or her about calculating those odds. I'd hardly call that "multiple times"!

 

[Bryan Shelton]

What are the odds that someone could successfully move an item telekinetically? How would you figure that out?

How big is the object, and how far would it move?

 

[Pixel42]

I think there's a great deal of misunderstanding going on in this thread.

I think you're right.

What I was specifically referring to in my quote above has to do with stating the EXACT ODDS that a person without any special abilities at all would have in passing both the "preliminary test", and the "formal test". One person (I'm forgetting his or her name) has talked about suggested probabilities for those tests, but hasn't responded to the question I gave him or her about calculating those odds.

I mentioned that the JREF rule of thumb seems to be that applicants should typically beat chance odds of 1:1000 in the preliminary test, in cases where such probabilistic success criteria are applicable, and said why that seems reasonable to me. As no-one has ever done so there is no precedent for the final test but the expectation seems to be that the success criteria would be set higher, though I've always thought a simple repeat of the first test would be adequate as that gives combined odds beaten of 1: 1,000,000.

As for how these odds are calculated: I assume you're not asking which formulae are used, but how it is decided what a reasonable success criteria should be. I can't speak for JREF, but clearly a compromise between odds high enough that they won't end up mistakenly awarding the prize to someone who was just extremely lucky and low enough that a true paranormal ability won't be excluded is required. That compromise is a judgement call, which is why negotiation is necessary with each applicant to agree success criteria which are mutually acceptable.

 

[jsfisher]

I don't really have a problem with making simple physical tests. I was only momentarily thrown for a loop when somebody talked about testing a person who claimed to be able to make somebody have to urinate!

How was the test for induced urination not a simple physical test? And just so I can plan ahead, will this be a no true Scotsman response or more of a goalpost relocation (which you already just did in the above quote, by the way)?


That aside, though, it seems like your unrequited desire is that the Million Dollar Challenge be fair (whatever that means) to people with no claimed ability whatsoever. "I have no special power, but I'd like a shot at $1,000,000 on pure luck." That is not the purpose; it would be foolish to expect the JREF to cater to such a requirement.

 

[TSR]

You appear to be confused. I thought it was obvious that I was referring specifically to tests of a purely probabilistic nature; examples would be someone who claims to "sense" the next drawing of a card from a shuffled deck; which side of a coin is going to occur after being "flipped", etc. If somebody comes along who claims to be able to levitate, or fly like a bird, then that's a much simpler claim to test, and doesn't involve probability at all.

And yet your proposal is that it be subject to odds by dint of those odds have been established before the claim is even made.

Which why they are not.

All paranormal, I would go so far as to say most paranormal claims are not of a probabilistic nature -- and *that* is why those odds are not spelled out prior to a claim.

 

[Pup]

How big is the object, and how far would it move?

That's exactly the kind of questioning that goes on in the negotiations between the JREF and the subject. Not so easy to state definite odds ahead of time, is it?

There was a contestant who claimed to be able to make a candle flame move by telekinesis. I don't think she even got to official negotiation levels, just informal self-testing and discussion, but a forum search should bring it up. The odds were almost impossible to figure, because of course the candle flame was always flickering--and that's how she was fooling herself into thinking she was affecting it with her mind. I think the discussion was getting down to marked quadrants and amount of time leaning toward each quadrant or something, compared to random odds.

One generally thinks of telekinesis, like levitation, as a 100%, do-it-or-don't kind of ability, but she managed to reduce it to one of those 55% situations.

 

[Bryan Shelton]

What I was specifically referring to in my quote above has to do with stating the EXACT ODDS that a person without any special abilities at all would have in passing both the "preliminary test", and the "formal test". One person (I'm forgetting his or her name) has talked about suggested probabilities for those tests, but hasn't responded to the question I gave him or her about calculating those odds.

I mentioned that the JREF rule of thumb seems to be that applicants should typically beat chance odds of 1:1000 in the preliminary test, in cases where such probabilistic success criteria are applicable, and said why that seems reasonable to me. As no-one has ever done so there is no precedent for the final test but the expectation seems to be that the success criteria would be set higher, though I've always thought a simple repeat of the first test would be adequate as that gives combined odds beaten of 1: 1,000,000.

I'm glad you're admitting that "there is no precedent for the final test". I've had SCADS of people complaining that this issue has supposedly been explained to me numerous times, even though I thought I had made it perfectly clear to everybody that I want the odds of successfully completing the entire thing (both the "preliminary" and the "final" tests) fully and completely stated. I hope this is now FINALLY understood, so I can stop having to go over this, over and over and over ad nauseum...

As for how these odds are calculated: I assume you're not asking which formulae are used, but how it is decided what a reasonable success criteria should be. I can't speak for JREF, but clearly a compromise between odds high enough that they won't end up mistakenly awarding the prize to someone who was just extremely lucky and low enough that a true paranormal ability won't be excluded is required. That compromise is a judgement call, which is why negotiation is necessary with each applicant to agree success criteria which are mutually acceptable.

I think if an applicant clearly states his claim at the beginning, that will automatically help determine the necessary parameters of the subsequent tests, assuming that they're always intended to achieve the same odds of success for a person who doesn't really have any "special abilities". For example, if a person says that he can "sense" a card drawn from a shuffled deck of playing cards, and he can do it correctly 1 out of every 30 times, it can be calculated straight away how many times he needs to do it correctly, to achieve better than 1:1000 odds. That's why I just don't like the word "negotiation" in this context. Don't "negotiate" with the guy, just listen to his claim, then plug the appropriate numbers into a mathematical formula, which then determines how long the tests will last.

 

[Bryan Shelton]

"I don't really have a problem with making simple physical tests. I was only momentarily thrown for a loop when somebody talked about testing a person who claimed to be able to make somebody have to urinate! "

How was the test for induced urination not a simple physical test? And just so I can plan ahead, will this be a no true Scotsman response or more of a goalpost relocation (which you already just did in the above quote, by the way)?

I don't understand the second sentence of your question. Sorry.

That aside, though, it seems like your unrequited desire is that the Million Dollar Challenge be fair (whatever that means) to people with no claimed ability whatsoever. "I have no special power, but I'd like a shot at $1,000,000 on pure luck." That is not the purpose; it would be foolish to expect the JREF to cater to such a requirement.

I don't want it to be "fair" to a person who has no special ability. Not in the sense that he has a significant possibility of actually winning.

 

[Bryan Shelton]

"How big is the object, and how far would it move?"

That's exactly the kind of questioning that goes on in the negotiations between the JREF and the subject. Not so easy to state definite odds ahead of time, is it?

No, of course not. But how many applicants ever make such claims? Do some of them ever stare at Brownian (sp?) motion through a microscope, and claim to be able to affect with their minds exactly how the microscopic particles of matter are colliding with each other, and changing each other's trajectories??

No, I doubt that happens very often. I think the typical claims by applicants have to do with more mundane issues, like they did with all the people on that TV show I talked about a few days ago; like that pretty woman who was given the set of keys (or whatever those things actually were), and told to "sense" the properties of each one. Every single applicant on that show was just like that.

 

[EdG]

Why don't they specify (and reveal) the exact terms beforehand? Why do they turn this into a guessing-game?

I'll give this one last try and then give up. There can be no exact terms beforehand because there are different paranormal claims. Mind-reading and dowsing and telekinesis and card-guessing et al are all different claims and cannot be proved or disproved using the same test. What part of that don't you understand?

 

[Bindamel]

It seems to me that your issue is that the test needs hard numbers attached to it before the applicant has even applied. People have tried to point out to you why that isn't in the applicant's best interest, and perhaps you just don't see it.


But, if you're interested, here are some threads on MDC statistics and probablility, and how they could be handled when an applicant realizes they aren't perfect.



http://www.internationalskeptics.com/forums/showthread.php?t=177657

http://www.internationalskeptics.com/forums/showthread.php?t=133513

http://www.internationalskeptics.com/forums/showthread.php?t=69495

http://www.internationalskeptics.com/forums/showthread.php?t=61257

In the last one of these, you were there pretty much asking the same question you are now, so none of this may be new to you.

 

[steenkh]

Bryan,

There are so many misunderstandings in this thread, but everybody who have have followed the MDC over the years should able to explain them to you. In fact, I think everything has already been said, but I will try to do it in my way.

First of all, the applicants hardly ever regard their abilities in terms of probabilities. Most of the times they claim they can do whatever they do all the time. But they also have a curious inability to explain just what they can do. Most of the correspondence with the applicants is used to pinpoint what it is that they claim. It happens all the time that they change their claims during the correspondence, particularly when the protocol is negotiated.

It has happened several times that the JREF negotiator is lowering the odds that the applicants suggests, because the JREF sees no need to make a huge number of tests when fewer will do. Odds lower than 1:1000 have also been offered by the JREF.

Although no applicant has ever reached the final test, it has been stated several times that the final test will be performed exactly as the preliminary test (presumably barring that obvious deficiencies in the protocol have been found). This means that the final odds will be 1:1,000,000, based on two tests at 1:1000 each.

The JREF is committed to test every paranormal claim, not just those of a probabilistic nature, which is why they test even silly things like psychically induced urination. They only reject claims that brings the claimant in danger, and breatharianism because testing it is cumbersome, and breatharians are invariably cheating. Previous exposure of breatharians has also not made these frauds less popular.

When calculating the odds it must be kept in mind that the JREF is testing people who usually claim they have this ability for sure. This is like testing if people can ride a bicycle. The test is constructed so that the odds of taking the test are extremely low if you cannot ride a bicycle, but if you can, there should be no problem whatsoever.

 

[Pup]

No, of course not. But how many applicants ever make such claims? Do some of them ever stare at Brownian (sp?) motion through a microscope, and claim to be able to affect with their minds exactly how the microscopic particles of matter are colliding with each other, and changing each other's trajectories??

No, I doubt that happens very often. I think the typical claims by applicants have to do with more mundane issues, like they did with all the people on that TV show I talked about a few days ago; like that pretty woman who was given the set of keys (or whatever those things actually were), and told to "sense" the properties of each one. Every single applicant on that show was just like that.

I mentioned the candle flame telekinesis applicant.

http://www.internationalskeptics.co...hp?t=31133&highlight=candle+flame+telekinesis

Enjoy. There was more discussion like that, but that's a sample.

 

[Almo]

I want the JREF to clearly state the chances of success (by ALL people) right up front. This business of "negotiation" is for the birds.

Why? The JREF can do whatever the hell they want. It's their challenge. The way it's done now is just fine; many tests have had protocols negotiated and tested.

 

[Bryan Shelton]

"Why don't they specify (and reveal) the exact terms beforehand? Why do they turn this into a guessing-game?"

I'll give this one last try and then give up. There can be no exact terms beforehand because there are different paranormal claims. Mind-reading and dowsing and telekinesis and card-guessing et al are all different claims and cannot be proved or disproved using the same test. What part of that don't you understand?

I've repeated this over and over; I'll give this one last try to explain to you, and then give up: when I talk about "exact terms" like I did in that quotation above, I'm mostly referring to the EXACT ODDS OF WINNING (by a person with no special abilities). That would apply to mind-reading, dowsing, and card-guessing (not sure about telekinesis).

 

[Bryan Shelton]

It seems to me that your issue is that the test needs hard numbers attached to it before the applicant has even applied. People have tried to point out to you why that isn't in the applicant's best interest, and perhaps you just don't see it.

On the contrary, I can fully appreciate that the "preliminary test" needs to have at least MODERATELY hard numbers, just to separate any truly gifted people from the slackers, liars, and thieves.

But, if you're interested, here are some threads on MDC statistics and probablility, and how they could be handled when an applicant realizes they aren't perfect.

http://www.internationalskeptics.com/forums/showthread.php?t=177657

http://www.internationalskeptics.com/forums/showthread.php?t=133513

http://www.internationalskeptics.com/forums/showthread.php?t=69495

http://www.internationalskeptics.com/forums/showthread.php?t=61257

In the last one of these, you were there pretty much asking the same question you are now, so none of this may be new to you.

Thanks for the links! I'll be sure to read those, when I have time!

 

[jsfisher]

I've repeated this over and over; I'll give this one last try to explain to you, and then give up: when I talk about "exact terms" like I did in that quotation above, I'm mostly referring to the EXACT ODDS OF WINNING (by a person with no special abilities). That would apply to mind-reading, dowsing, and card-guessing (not sure about telekinesis).

Yes, I understand this and have each time it has been stated. One thing you haven't made clear, though, is why you think this is at all important. The claimant's role in the Challenge is to perform some feat that seems paranormal in nature. The JREF's role is to eliminate mundane explanations for the feat. Sometime it may reduce to a game of chance, and sometimes not.

By the way, I have a paranormal ability. I can completely stop my heart on command for a period of 60 seconds then resume a normal heart-beat. For my test protocol, we shall have a registered nurse (selected by the JREF) take my pulse at my left wrist (the left arm having the stronger pulse than the right) and timing the period during which the pulse stops. 60 seconds or more, I win.

What shall we use for odds in this case? Is probability really a concern for this proposed protocol?

 

[wardenclyffe]

So, according to Bryan Shelton, and please correct me if I'm wrong, (I'm really not trying to be snarky) the JREF should state the exact odds of winning by chance alone. This must take into account a varied number of cards from a 52 card deck, a six-sided die, a random number generator a roulette wheel, etc., etc..

If something comes up where the number of trials cannot be divided to get the exact odds, applicants will be disqualified or they will be allowed to perform a third or half of a trial in order to get the exact odds.

And all of this applies, except where it doesn't, like claims of levitation, telekinesis, making people pee, etc..

Am I understaning the proposal correctly?

Ward

 

[jsfisher]

So, according to Bryan Shelton, and please correct me if I'm wrong, (I'm really not trying to be snarky) the JREF should state the exact odds of winning by chance alone. This must take into account a varied number of cards from a 52 card deck, a six-sided die, a random number generator a roulette wheel, etc., etc..

If something comes up where the number of trials cannot be divided to get the exact odds, applicants will be disqualified or they will be allowed to perform a third or half of a trial in order to get the exact odds.

And all of this applies, except where it doesn't, like claims of levitation, telekinesis, making people pee, etc..

Am I understaning the proposal correctly?

Ward

I believe he is looking for a statement in the MDC rules something like this: The test protocol must exclude the possibility of a successful performance on chance alone at a probability level of 1/1,000 in the preliminary test and 1/1,000,000 in the final.

He has not been clear as to why this is important.

 

[wardenclyffe]

I believe he is looking for a statement in the MDC rules something like this: The test protocol must exclude the possibility of a successful performance on chance alone at a probability level of 1/1,000 in the preliminary test and 1/1,000,000 in the final.

He has not been clear as to why this is important.

Or even possible.

Ward

 

[steenkh]

I've repeated this over and over; I'll give this one last try to explain to you, and then give up: when I talk about "exact terms" like I did in that quotation above, I'm mostly referring to the EXACT ODDS OF WINNING (by a person with no special abilities). That would apply to mind-reading, dowsing, and card-guessing (not sure about telekinesis).

Why are are the odds of winning without ability important at all? The really interesting odds should be how likely you are at winning the challenge if you have the abilities. If I was going for the challenge, that would be my interest, or rather how are the odds of losing the challenge if you have the ability?

 

[xtifr]

Putting too much emphasis on the odds would encourage gamblers who know they have no ability, but hope to get lucky and walk away with a million dollars. People routinely play the lottery at much longer odds.

 

[Rasmus]

Why are are the odds of winning without ability important at all? The really interesting odds should be how likely you are at winning the challenge if you have the abilities. If I was going for the challenge, that would be my interest, or rather how are the odds of losing the challenge if you have the ability?

Exactly.

The only reason to allow the possibility that a test could be passed by chance alone would be that it might be impossible or at least highly impractical to rule it out, especially if the claimed ability isn't claimed with a 100% success rate, too.

On the other hand, where applicants claim the impossible - there is no reason to be quite as strict, and certainly no reason to somehow increase their chances of winning by luck alone.

If I say i can make a brick levitate by the power of my mind from 2 meters away, even when the brick is sealed off in a glass box, etc. then there is no reason to demand i get it floating at once, or for an hour, or whatever.

all i have to do is get it off ground within a reasonable time frame - aka how ever long someone is willing to watch a brick - and for long enough to rule out a brick just being thrown or dropped.

On the other hand, i cannot easily imagine a test scenario where my random chances of winning would even be measurable.

 

[gnome]

It's actually in the applicant's favor... the question of chance odds is still on the table, rather than restricted. The only effect a prior restriction could possibly have would be to close the door to the challenge for some applicants.

In the end, it's whatever you can get the JREF to agree to.

 

[Gr8wight]

I've repeated this over and over; I'll give this one last try to explain to you, and then give up: when I talk about "exact terms" like I did in that quotation above, I'm mostly referring to the EXACT ODDS OF WINNING (by a person with no special abilities). That would apply to mind-reading, dowsing, and card-guessing (not sure about telekinesis).

Bryan,

In order to quote the EXACT ODDS OF WINNING, the JREF would have to know the EXACT TEST DESIGN BEING USED. This is information they don't have until after the test is finished being designed... BY THE APPLICANT.

 

[xtifr]

As far as I can see, the tests are almost always designed so that the odds of winning are 100%.

If course, that's only true if the claimed ability is real. If it's not, the odds should be 0%, but that's often an unattainable goal, so the JREF is willing to accept higher odds than that when necessary. How much higher should be of no concern to the candidate, though, who should only expect to win if he or she can actually demonstrate the ability. This is not a casino!

 

[Rasmus]

This is not a casino!

... even though I have long thought that the JREF should take a moderate sum - between 1$ and 10$ of each participant so that playing the lottery would be the better deal to further try and discourage the fakers.

To be fair, if possible, the exact odds of winning should be made public for each actual test if the concept applies. For science. And to make sure everybody knows they understand how the test works, i.e. if I work out the same probability I know i didn't overlook a step or anything. But that's clearly after a protocol has been agreed on.

 

[Ladewig]

I am of the opinion that the TV show tests cited previously should be completely ignored. I have no idea how old the TV show was but all tests designed in 2012 will have two exceptionally important things that may have not been present in the TV show.

1) The claimant will clearly state what his or her claim is AND how often he or she can successfully perform the task. Therefore, complaints about Randi setting impossibly high success rates are irrelevant. If the JREF responds to an applicant's claimed success rate by saying, "we will need to see a higher success rate as part of the MDC," then the applicant is free to walk away, to accept the higher number, to practice until he or she achieves that level of success, or to ask how and why the higher success rate was calculated. NONE of these options is unfair in any way.

2) The current protocols require that an applicant pass an open test before passing a closed test. If a psychic claims that she can handle five door keys and describe what types of buildings they unlock, the the testing procedure starts with the JREF representative giving five keys to the applicant AND saying, "this key is from a NYC condo, this key is from a Georgia high school, this key is from a Missouri farmhouse...." The psychic is then asked if she is able to detect the psychic vibrations from the clearly-identified keys in such a way as to constitute a success for the trial. Only if the psychic agrees that her psychic powers are working correctly will the second half of the test be performed: five unidentified keys are handed to the psychic.

Therefore it is inappropriate to claim that the success rate was set at too high of a level. The psychic must agree in writing that he or she can pass the test as agreed upon.

Also, therefore it is inappropriate to claim that the test made the applicant too nervous. The closed test occurs only if the psychic agrees that these conditions are acceptable and that this location is acceptable. The open test also prevents the psychic from later claiming (a) negative thoughts from the skeptics foiled my power (b) Ley lines or other geopsychic forces foiled my power, (c) the gold leaf on the cover of a nearby encyclopedia foiled my ability to dowse for precious metals. N.B. all these excuses have been used in past tests.

 

[shuttlt]

The open test also prevents the psychic from later claiming (a) negative thoughts from the skeptics foiled my power

Surely in the open tests, the sceptics expect the psychic to be successful, where as in the closed test they expect the psychic to be unsuccessful? Won't the negative psychic vibrations caused by the sceptics only impact the closed test? If I was a psychic, that's what I'd claim, anyway...

 

[Rasmus]

Surely in the open tests, the sceptics expect the psychic to be successful, where as in the closed test they expect the psychic to be unsuccessful? Won't the negative psychic vibrations caused by the sceptics only impact the closed test? If I was a psychic, that's what I'd claim, anyway...

No, strictly speaking, I do not expect the psychic to be successful in the open test.

Say I did the card-prediction thing, where I'd look at the back of a deck of playing cards and were asked to tell the order of the cards.

In the open test, I would be given a deck in a known order and I would be asked if i could see or sense etc. the cards that way. I'd fully expect the psychic to not be able to see that the first card is the ace of spades, even when they are told what it is.

 

[zooterkin]

The open test also prevents the psychic from later claiming (a) negative thoughts from the skeptics foiled my power (b) Ley lines or other geopsychic forces foiled my power, (c) the gold leaf on the cover of a nearby encyclopedia foiled my ability to dowse for precious metals. N.B. all these excuses have been used in past tests.

But wasn't the 'gold leaf' excuse used even after an open test?

 

[Ladewig]

But wasn't the 'gold leaf' excuse used even after an open test?

Indeed. When it was pointed out that the applicant said everything was working during the open test, the applicant came out with the classic line "I lied when I said everything was acceptable after the open test."

 

[Ladewig]

There is one more reason to not complain about how the success statistics are calculated. Once the number is agreed upon, the JREF emphatically encourages the applicant to go home and practice to make sure that the applicant can hit that threshold. When the applicant arrives for the test, he or she is not surprised by a newly-calculated success threshold. The applicant has had as much time as he or she wants to ensure that he or she can perform at the agreed-upon level.

 

[DevilsAdvocate]

Look at it this way. Let’s say the person passes the MDC:

Chance that the person passed without having any paranormal ability is 1 in 1,000,000.
Chance that the person passed with a paranormal ability is 999,999 in 1,000,000.

Setting those odds even higher does not change the chance that a person with an actual ability can pass the test—it only makes harder for a person without an ability to pass the test BY CHANCE. There is no need for JREF to set a specific standard for the risk they are willing to take for the MDC to be passed by chance because it does not affect the chances of a person with an actual ability to pass the challenge.

For example, let’s say I can predict coin flips. I might claim to be able to predict the flips 100% or the time, or 70%, or 90% or 50.02%. It doesn’t matter. A test can be devised for any of those claims where the odds of winning by chance remain at 1 in 1,000,000.

Let’s say I claim to be able to predict coin flips correctly 70% of the time. Consider these protocols:

Protocol 1: I must predict flips correctly 70% of the time and there is a 1 in 5 chance that I could win by pure luck.

Protocol 2: I must predict flips correctly 70% of the time and there is a 1 in 1,000,000 chance that I could win by pure luck.

For me, both test are the same. If I can actually do what I claim, if I can actual predict flips correctly 70% of the time, I will pass the test with either protocol. If I can do what I claim I can do, I don’t care what the odds are of winning by chance because I will pass using my ability and have no intention of simply passing by luck. In fact, I would prefer protocol 2 because then when I win it will be more conclusive that I do have the ability that I claim to have and that I didn’t just pass by luck. Of course the difference in the protocols makes a difference to JREF because with protocol 1 they may well have to pay the $1 million to somebody who doesn’t actually have a paranormal ability. But that’s JREF’s risk and doesn’t affect a person with a paranormal power from being able to pass the test simply by doing what they claim they can do.

So there is no reason for JREF to set some standard for wining by luck. The only people who need to be concerned with such a standard are people without powers who want to win by luck, JREF, and the rest of us who might ponder whether somebody winning the MDC would be considered an actual demonstration of a paranormal power or a just a demonstration of lucky guessing. It does not affect an applicant with an actual ability (other than the number of trial the protocol may require) so there is no point in setting a standard.

 

[DevilsAdvocate]

As far as I can see, the tests are almost always designed so that the odds of winning are 100%.

If course, that's only true if the claimed ability is real. If it's not, the odds should be 0%, but that's often an unattainable goal, so the JREF is willing to accept higher odds than that when necessary. How much higher should be of no concern to the candidate, though, who should only expect to win if he or she can actually demonstrate the ability. This is not a casino!

Exactly. The applicant can either do what they claim they can do or they can. If the applicant has the ability they claim, they will pass the test. If the applicant DOES NOT have the ability they claim, they will either fail or pass by luck. For an applicant who has an actual ability, it doesn’t matter what the odds are of wining by luck because they will pass using their ability.