determining what constitutes a success

It should read: "Note that T'ai Chi ignored Flange Desire's order." :rolleyes:

"Whould YOU say that I passed or failed?" looks a lot like a question to me.

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

My concern is the implication of unfairness in Tais last sentence.
Tai says that the claimant is really perfoming acceptably, because they were incorrect about their own assesment.
I say that if the ball does not go through the goalposts then it is not a goal,
regardless of how well you think you can kick a ball, and regardless of how much you miss by.
 
T'ai, when you say you want to know what the claimants can do, rather than whether they can do what they claim, are you saying that the JREF should test them for any level of paranormal ability? Because if that's what you're saying then (from my limited statistical knowledge) you are massively increasing the practical difficulties of running tests.

e.g

Someone claims to be able to consistently predict 80% of the tosses of a perfectly fair coin. Now, we all know that if he could actually consistently predict 50.000001% of coin tosses (in a test which absolutely excludes any other factor known to science) then he would have a small measure of paranormal ability. But if you want to test whether someone is able to predict 50.000001% of coin tosses then you have to do a HUGE number of tests in order to be comfortable (with a reasonable measure of confidence) that someone can in fact do that. I'm sure someone here, whose statistical education is more recent than mine, could calculate how many tests would be required to know with 95% confidence (which may be too small for JREF purposes) that someone can predict 50.000001% of coin tosses, assuming they get the bare minimum of correct predictions. I suspect it runs to the millions.

On the other hand, it's remarkably easy to test someone's claim of 80% correct predications. The number of trials needed for 95% confidence is probably well under a hundred (that's just a guess - it's nearly 10 years since I did my A-level in statistics, and I've since gone into law where there's little call for it!).

Now of course, the example you gave cited someone who could actually predict 60% of coin tosses (which I suspect would still be testable with a non-ludicrous number of trials), but your argument would seem to extend right down to my 50.000001% example. Any boundary you draw that is below the claimant's claimed success rate is arbitrary.

Now of course, the above doesn't really help the JREF if they run up against an applicant who actually claims to be able to predict coin tosses with a 50.000001% success rate - but there are remarkably few of them about!

Matt
 
Success or failure is based on a customized set of parameters agreed upon during each individual protocol negotiation, is it not? There are some basic standards that JREF uses for repetitive claims - no need to re-invent the wheel - but even those are subject to negotiation as long as the agreement doesn't compromise the measure of success/failure to the point where random chance results would be indistinguishable from a real success.

Besides, even random chance won't cut it - you need to succeed with the preliminary test, then do a repeat performance for the final test. The odds of random chance "winning" consecutively like that have got to be astronomical.
 
Last edited:
T'ai, when you say you want to know what the claimants can do, rather than whether they can do what they claim, are you saying that the JREF should test them for any level of paranormal ability? Because if that's what you're saying then (from my limited statistical knowledge) you are massively increasing the practical difficulties of running tests.

e.g

Someone claims to be able to consistently predict 80% of the tosses of a perfectly fair coin. Now, we all know that if he could actually consistently predict 50.000001% of coin tosses (in a test which absolutely excludes any other factor known to science) then he would have a small measure of paranormal ability. But if you want to test whether someone is able to predict 50.000001% of coin tosses then you have to do a HUGE number of tests in order to be comfortable (with a reasonable measure of confidence) that someone can in fact do that. I'm sure someone here, whose statistical education is more recent than mine, could calculate how many tests would be required to know with 95% confidence (which may be too small for JREF purposes) that someone can predict 50.000001% of coin tosses, assuming they get the bare minimum of correct predictions. I suspect it runs to the millions.

On the other hand, it's remarkably easy to test someone's claim of 80% correct predications. The number of trials needed for 95% confidence is probably well under a hundred (that's just a guess - it's nearly 10 years since I did my A-level in statistics, and I've since gone into law where there's little call for it!).

Now of course, the example you gave cited someone who could actually predict 60% of coin tosses (which I suspect would still be testable with a non-ludicrous number of trials), but your argument would seem to extend right down to my 50.000001% example. Any boundary you draw that is below the claimant's claimed success rate is arbitrary.

Now of course, the above doesn't really help the JREF if they run up against an applicant who actually claims to be able to predict coin tosses with a 50.000001% success rate - but there are remarkably few of them about!

Matt

Coin flip requirements based on requiring the test to be at the 1 in a million range of chance occurrence:

84% -- 50 trials
61% -- 500 trials
54% -- 5,000 trials
51% -- 50,000 trials
50.4% -- 500,000 trials
50.1% -- 5,000,000 trials

Caveat: I think I did this right, but I would be happy to stand corrected if I did not.
 
Thanks Bindamel, that's just what I needed!

T'ai, doesn't that answer why the JREF tests claims largely based upon the success rate claimed, rather than in an effort to establish whether there is a minute, unexplained effect? Even if you try to test for a 51% success rate, you're massively increasing the number of tests required.

In any event, if someone claims 80% success and it's clear that they cannot achieve that rate (or anywhere near that rate) of success, why shouldn't the JREF, quite reasonably, conclude that the applicant is probably a fantasist and that to go ahead and test for a very small paranormal ability is a waste of their time? Is it really very likely that the applicant is someone with the true ability to predict 51% of coin tosses, but they are simply so self-deluded that they claim an 80% success rate? I'm far more inclined to conclude that they are totally deluded, and have no paranormal ability whatsoever. But hey, that's what makes me a skeptic. :D

Matt

P.S. As a semi-related aside, many people on this board at many times have stated that, despite being skeptics, they think it would be really cool if there really were people out there with paranormal powers (a sentiment with which I wholeheartedly concur). But wouldn't it just be a real kick in the teeth if there were people out there with paranormal powers, but that they were so slight as to serve absolutely no useful function within human society? What if we discovered that the only paranormal ability that REALLY exists is spoon-bending (and it doesn't work on forks or knives, or even large spoons for that matter). That would be the most depressing thing ever - far more depressing to my mind than the likely reality that no one has cool psychic powers. I think I'd prefer not to know if that was the case!:p
 
The other question that this brings up is if someone thinks they have a minute ability, like being able to predict coin flips 51% of the time, how did they discover it in the first place? No one records and analyzes 50,000 coin flips for no reason.
 
The other question that this brings up is if someone thinks they have a minute ability, like being able to predict coin flips 51% of the time, how did they discover it in the first place? No one records and analyzes 50,000 coin flips for no reason.
Why not? You could do that in less than a week. And then you would have to replicate it to be sure... People do stranger things than that. Maybe not more boring, though. If I was volunteering to conduct a trial, I would not be happy if Randi agreed to a protocol like that!
 
Coin flip requirements based on requiring the test to be at the 1 in a million range of chance occurrence:

84% -- 50 trials
61% -- 500 trials
54% -- 5,000 trials
51% -- 50,000 trials
50.4% -- 500,000 trials
50.1% -- 5,000,000 trials

Caveat: I think I did this right, but I would be happy to stand corrected if I did not.

There's a minor issue that sort of damns this whole enterprise. I didn't bother to check your math -- but as near as I can tell, you're doing the sums based on the idea that a coin flip is a 50/50 proposition? (In other words, your null hypothesis is a 50/50 split?)

This is where part of the problem with T'ai's whole approach comes in. If I claim to be able to predict coin flips with 51% accuracy -- well, THAT may not be paranormal. Coin flips aren't actually 50/50, depending upon how you flip them. Even a "fair" flip is biased to land 'tails' (typically) because the weight distribution of the coin is uneven. Try it yourself with 50,000 flips and see!

Unfortunately, this means that T'ai's hope to do a grand metaanalysis simply won't work, because we don't know the proper null hypothesis to apply for many of the tests that have been done.
 
T'ai, when you say you want to know what the claimants can do, rather than whether they can do what they claim, are you saying that the JREF should test them for any level of paranormal ability? Because if that's what you're saying then (from my limited statistical knowledge) you are massively increasing the practical difficulties of running tests.

See my example why. The person could be mistaken about their beliefs (in fact, it is known that people are regularly mistaken, so this seems an obvious point).

The scenarios don't make difficulties go away. Someone can simply say 'I believe I can get 50.000000001%' and then what? Obviously, in the real world we live in, there are limited resources, and either way we'd have to take those into account.
 
In any event, if someone claims 80% success and it's clear that they cannot achieve that rate (or anywhere near that rate) of success, why shouldn't the JREF, quite reasonably, conclude that the applicant is probably a fantasist and that to go ahead and test for a very small paranormal ability is a waste of their time?

If someone is tested for 80% success and they don't pass, I doubt they'd then claim a smaller percent.

But in these hypothetical scenarios, your guess is as good as mine.
 
See my example why. The person could be mistaken about their beliefs (in fact, it is known that people are regularly mistaken, so this seems an obvious point).

The scenarios don't make difficulties go away. Someone can simply say 'I believe I can get 50.000000001%' and then what? Obviously, in the real world we live in, there are limited resources, and either way we'd have to take those into account.

Well, we'd have to take those into account if our intent is to demonstrate conclusively that there is nothing paranormal about applicants or their abilities.

However, if someone were to create an orginization with the intent to simply TEST WHAT THEY CLAIM, then perhaps we could ignore such possibilites and simply TEST THEIR CLAIM.

Perhaps someone with more resources at their disposal will wish to contact such people to examine whether or not there might still be something paranormal, even if they themselves cannot identify it within a degree of success. If you know of any such individuals with an interest in doing so and resources to expend, perhaps you could direct them to contact JREF to put them in touch with interested applicants.
 
Well, we'd have to take those into account if our intent is to demonstrate conclusively that there is nothing paranormal about applicants or their abilities.

However, if someone were to create an orginization with the intent to simply TEST WHAT THEY CLAIM, then perhaps we could ignore such possibilites and simply TEST THEIR CLAIM.

You're right of course - but I think it's valid to ask "Why is the JREF's aim to simply test specific claims rather than to be a seeker of ultimate truth?". Superficially, seeking the ultimate truth is an attractive idea. Superficially, it would be good to test coin-tossers to see if they have any ability to predict coin tosses beyond chance and award the million on that basis. T'ai seems to be suggesting that that would be a better goal. Whether he thinks it's a better goal for the community in general, or for the JREF in particular he doesn't make clear, but I've tried to fill in the gaps :D

However, when you get down to it, seeking ultimate truths is damn impractical. One problem is the statistical one I set out above. DrKitten's point is also a good one - sometimes (often?) we simply do not know what the null hypothesis is, and so the JREF has to plump for a suitably high threshold which is intuitively beyond the realms of mere chance.
 
Tai:
I claim I can view skullplates using xray vision,
but in fact can do no such thing.
(yes, in this example my belief is contrary to reality, same as all woos)
I state my expected success rate as 93.276%.
I can really perform at 0%.
When tested I have a success rate of around 10%.
I am performing acceptably!!!
Whould YOU say that I passed or failed?
This is still ignored by T'ai Chi.

It's "Yes" or "No" question, but apparently that is too hard.
 
You're right of course - but I think it's valid to ask "Why is the JREF's aim to simply test specific claims rather than to be a seeker of ultimate truth?". Superficially, seeking the ultimate truth is an attractive idea. Superficially, it would be good to test coin-tossers to see if they have any ability to predict coin tosses beyond chance and award the million on that basis. T'ai seems to be suggesting that that would be a better goal. Whether he thinks it's a better goal for the community in general, or for the JREF in particular he doesn't make clear, but I've tried to fill in the gaps :D

However, when you get down to it, seeking ultimate truths is damn impractical. One problem is the statistical one I set out above. DrKitten's point is also a good one - sometimes (often?) we simply do not know what the null hypothesis is, and so the JREF has to plump for a suitably high threshold which is intuitively beyond the realms of mere chance.

There are many people devoted to finding any evidence of the supernatural. Some of them are genuine scientists working in respectable labs/universities and not just woos who already believe. Randi just takes a different approach that is as much to do with discrediting the more outspoken frauds as it is to do with providing evidence for anything.

As you have said, seeking ultimate truths is damn impractical. Presumably Randi feels this is best left to professional scientists, while his prize is more suited to targeting people who claim easily observable results. Offering a large sum of money is also more likely to appeal to those who want publicity and fame rather than those who want the ultimate truth, which is exactly the sort of person Randi wishes to debunk (unless they are right of course :p).
 
There's a minor issue that sort of damns this whole enterprise. I didn't bother to check your math -- but as near as I can tell, you're doing the sums based on the idea that a coin flip is a 50/50 proposition? (In other words, your null hypothesis is a 50/50 split?)

This is where part of the problem with T'ai's whole approach comes in. If I claim to be able to predict coin flips with 51% accuracy -- well, THAT may not be paranormal. Coin flips aren't actually 50/50, depending upon how you flip them. Even a "fair" flip is biased to land 'tails' (typically) because the weight distribution of the coin is uneven. Try it yourself with 50,000 flips and see!

Unfortunately, this means that T'ai's hope to do a grand metaanalysis simply won't work, because we don't know the proper null hypothesis to apply for many of the tests that have been done.

I am new here and have been following this thread with much interest! Sorry about "jumping in" right of the bat before introducing myself. But, how about this challenge:

If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:

1) I will pray to my deity and let him know that we are going to conduct an experiment to prove his existence. He will then inform me, in advance, of the outcome of that experiment. I will write the 100 "heads & tails" results (that is, the exact sequence of "heads & tails" that will occur) down on a paper of paper, seal that in an envelope, and then mail it to your office.

2) You conduct the "coin toss" experiment with a coin of your choosing and at a time and place of your choosing, recording the results.

3) If the results of your experiment agree exactly with my a prior claims, then we replicate the experiment again. I pray to my deity a second time, find out from him/her/it what the second sequence of "Heads & Tails" will be, write that information down, seal it in an envelope, mail it to your office, etc.

4) If the replication agrees with my second results, I win the $1 million dollars (which I will donate, in its entirety, to charity)!

5) If not, we agree that I will never perform this experiment again nor will the James Randi foundation agree to a "third" attempt.
 
I am new here and have been following this thread with much interest! Sorry about "jumping in" right of the bat before introducing myself. But, how about this challenge:

If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:

1) I will pray to my deity and let him know that we are going to conduct an experiment to prove his existence. He will then inform me, in advance, of the outcome of that experiment. I will write the 100 "heads & tails" results (that is, the exact sequence of "heads & tails" that will occur) down on a paper of paper, seal that in an envelope, and then mail it to your office.

2) You conduct the "coin toss" experiment with a coin of your choosing and at a time and place of your choosing, recording the results.

3) If the results of your experiment agree exactly with my a prior claims, then we replicate the experiment again. I pray to my deity a second time, find out from him/her/it what the second sequence of "Heads & Tails" will be, write that information down, seal it in an envelope, mail it to your office, etc.

4) If the replication agrees with my second results, I win the $1 million dollars (which I will donate, in its entirety, to charity)!

5) If not, we agree that I will never perform this experiment again nor will the James Randi foundation agree to a "third" attempt.
Well, the test would have to be performed in person. With all individuals present(both you and whoever performs the toss). But besides for that it should be a valid claim for the challenge.

Sincerely
Tobias
 
I am new here and have been following this thread with much interest! Sorry about "jumping in" right of the bat before introducing myself. But, how about this challenge:

If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:

I doubt that the protocol would be accepted like that. You would have no way to control whether the tossing of the coins was done fairly or if the JREF cheated during the control. So, for a final test at least (and probably for the preliminary test, too) you would have to be present at a test-site.

But I am sure that sending an application that contains a prediction of 10 or 20 correct predictions would certainlyx draw their attention ...

Also, the JREF probably won't care what you do with the money, and if you fail you could still re-apply after a year. There's no point in changing the rules here; if you want to give to charity: great! If you never want to apply again ... well, that would be great if it meant you understood that you have no super-powers.
 
I am new here and have been following this thread with much interest! Sorry about "jumping in" right of the bat before introducing myself. But, how about this challenge:

If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:

1) I will pray to my deity and let him know that we are going to conduct an experiment to prove his existence. He will then inform me, in advance, of the outcome of that experiment. I will write the 100 "heads & tails" results (that is, the exact sequence of "heads & tails" that will occur) down on a paper of paper, seal that in an envelope, and then mail it to your office.

2) You conduct the "coin toss" experiment with a coin of your choosing and at a time and place of your choosing, recording the results.

3) If the results of your experiment agree exactly with my a prior claims, then we replicate the experiment again. I pray to my deity a second time, find out from him/her/it what the second sequence of "Heads & Tails" will be, write that information down, seal it in an envelope, mail it to your office, etc.

4) If the replication agrees with my second results, I win the $1 million dollars (which I will donate, in its entirety, to charity)!

5) If not, we agree that I will never perform this experiment again nor will the James Randi foundation agree to a "third" attempt.

First of all: Welcome to the Forum, Jehanne.

If you are serious about your claim - which I also consider both easily testable and meeting the Challenge's criteria - I suggest you start a seperate thread in this subforum in which we can discuss solely your claim.

Again: Welcome.
 
You're right of course - but I think it's valid to ask "Why is the JREF's aim to simply test specific claims rather than to be a seeker of ultimate truth?".
It is a valid question - and one with a valid answer. :)

JREF was formed by people who do not believe in the supernatural. The reason they test specific claims is because that's what they do best by design. Seeking ultimate truth falls within the realm of science and/or philosophy; debunking fraudulent claims of the supernatural by JREF is a complementary activity which happens to benefit those who are seeking the ultimate truth.
 

ISF - Join now!

Every member here is approved by hand. No bots, no spam, just people who care about evidence and honest debate.

Membership is free!

Create your free account

Back
Top Bottom