Some offhand questions

[Lenoxus]

Hello. Skeptic here — I've just introduced myself in the introduction forum. I've been reading a bit about the challenge, its history, and the various excuses people have made for not participating in it. I've become curious about a couple things, and am wondering whether any of the following has ever happened…

1. The challenge generally excludes anyone who could put themselves or others at harm if their claim is tested, for example, the breatharians who claim they can survive without food or water. I'm curious… have any applicants made claims which, if true, would put someone at risk, but would otherwise be harmless — and have these claims ever undergone the prelimary tests? For example, suppose someone claims that she can make flowers wither with her mind, but that some 10% of the time, the psychic beams go in the wrong direction and make the experimenter really sick. Dunno, this probably hasn't happened, but there is a pretty wide varity of claims.

2. Many psychics attach to their services the phrase "for entertainment purposes only" to avoid liability. Have any psychics, when challenged to take part in the Challenge, said "Hey, can't you read? What I do really is just for entertainment, like with a professional magician or a stand-up comic. Asking me to do this 'for real' is as dumb as asking someone at Weta Worshop to provide you with a 'real' orc." (Again, this probably hasn't happened, but it would be nice if it had.)

3. Some folks have argued that the Challenge is unfair because the p-values and effect sizes called for are too extreme. Have any applicants claimed to have paranormal abilities with only a small effect size? I understand that one of the ganzfeld parapsychologists had some discussions or something with Randi about doing a large-scale test, but it never reached the preliminary testing stage. Has anyone else made a "moderate" claim, ie, that they can dowse at a 15% success rate in a one-out-of-ten situation? Obviously, testing such a claim would be a bit more expensive, because you would need to have a lot more samples, and would probably have to tighten the controls a lot. I'm just curious if anyone's initial claim has even been, well, humble.

So, there ya go. I'd try searching through the currently-online applications, but it's tricky to Google this stuff, there being a lot of possible synonyms and whatnot. (Though I could be wrong about that, of course.) If you happen to be in a position to say "yes" or "no" to any of these questions, well, I'd be much obliged.

 

[wardenclyffe]

As far as I know (and I know a little more than most, but I'm no expert), the answers to all of your questions are "no."

I'm not aware of any claims similar to the example you give in your first question. I know that James Randi has said he would not test someone who claimed they could fly by jumping out a high window.

Not aware of anyone who would fit into category 2.

And I'm not aware of anyone who's applied with a very moderate claim. Perhaps some have made some early moves to do so, but I don't know if any have actually applied. If they did apply, perhaps protocol negotiations broke down because such a claim usually requires many many more trials which usually increases both the monetary and time expense to run the test. The monetary expense is all on the back of the applicant and the time expense is for everyone involved. A test like that can become prohibitive for both sides.

That's all I can offer,

Ward

ETA: If you find examples of any of these types of applicants, please post them here. I'd be interested in knowing about them.

 

[Startz]

The very long negotiations over the application of PAVEL ZIBOROV (posted elsewhere in the Challenge forum) had a strong element of your point 3, although the applicant did not claim any unfairness. Pavel was claiming something with a fairly small effect size, and JREF declined to engage in a long test. (There were no doubt other issues as well.)

 

[xterra]

As Wardenclyffe pointed out, Randi said he wouldn't "test someone who claimed they could fly by jumping out a high window"; I think it is fair to extend this to similar kinds of claims, such as stopping a speeding locomotive by standing in its way. It would be at least irresponsible, and at worst (arguably) criminal to participate in the demonstration or challenge of such a claim.

The second question, about "entertainment only," can be taken two ways. First, as the OP says, as a way to avoid liability by someone who claims that he or she is really clairvoyant or psychic. The other way is that a person is really doing cold or warm reading and is saying in effect "this is not real; it is a trick, an illusion."

Sylvia Browne, Uri Geller, et. al. on the other hand, have claimed that their abilities are real; and as far as I know, they don't add the entertainment-only escape clause. They also don't allow themselves to be tested under rigorous conditions. Or maybe not at all.

As for the third question, Wardenclyffe is again correct. The number of trials required would be prohibitively expensive in both time and money. Remember that the applicant pays for the test. Why would a person who believes his or her ability works 15% of the time bother with the challenge?

 

[EdG]

Why would a person who believes his or her ability works 15% of the time bother with the challenge?

If a person could pick a single number accurately in roulette 15% of the time, she would come out ahead as the payout is 35 to 1. Thus, betting $10 on 7 spins would cost $70 while winning $350. So, to answer your question, they would not bother with the challenge except to prove skeptics wrong, as they could earn far more money at the roulette wheel than what the challenge pays.

 

[xterra]

Odds and reasons

If a person could pick a single number accurately in roulette 15% of the time, she would come out ahead as the payout is 35 to 1. Thus, betting $10 on 7 spins would cost $70 while winning $350. So, to answer your question, they would not bother with the challenge except to prove skeptics wrong, as they could earn far more money at the roulette wheel than what the challenge pays.

EdG, I'll concede the point. I have no idea of the odds in roulette.

But as far as I know, in my eight years of reading the challenge applications, there haven't been many people who thought their ability worked less than 100% of the time.

Would your gambler pay $3,500 to win $350? (I mean, just as an experiment, not as entertainment.) I certainly wouldn't pay that for either reason -- but then I am ... um ... frugal.

 

[wardenclyffe]

Why would a person who believes his or her ability works 15% of the time bother with the challenge?

Well, if I felt I were able to dowse for a gallon of water in a box successfully 15% of the time, I'd apply to win a million dollars. I mean, it's a million dollars. A test like that would be time consuming, but not particularly expensive. Problem is, applicants like dowsers tend to become mentally and emotionally drained after just a few trials. So in that case, the test would probably have to go on for months in order to have enough trials for a clear 15% to be proven. Now, if I as a dowser do not become exhausted, we could probably do enough trials back to back in a day or a weekend.

Ward

 

[Lenoxus]

The very long negotiations over the application of PAVEL ZIBOROV (posted elsewhere in the Challenge forum) had a strong element of your point 3, although the applicant did not claim any unfairness. Pavel was claiming something with a fairly small effect size, and JREF declined to engage in a long test. (There were no doubt other issues as well.)

I've now read some of the negotiations in that case… I liked the various discussions for working out protocols, calculating probabilties, and trying to determine the details of the original claim. For some reason, I find it inherently fascinating, this stuff at the conjunction of magic and statistics.

Anyway, it looks like there were indeed a number of issues with that one, some of them nobody's fault like the language barrier. Also, Pavel wasn't exactly consistent about claims of a small effect size. I admired his honesty about his self-test results of 7-7-7-8-4 where 5-5-5-5-5 would have been exact chance (that 4 brings the average down to an unsurprising 6.6)… but he also once claimed that in a test with 30 he could expect to get 24 right, though other claims suggested much lower results. I'm not positive he understood the basic statistics involved.

In any case, it seems like on first principles, no one can claim that the MDC's standards are inherently unfair. Even the smallest proposed effect size will have a corresponding low p-value for some finite number of tests, though for really small sizes that number will be quite large.

I like to think that if I had a legitimate but low-effect paranormal ability then I'd be willing to foot the bill, given the large potential prize, so long as the expected value worked out. For example, say my ability means I have a consistent 1% chance of passing the final test; if doing the Challenge cost me $9,000, then the net expected value for me would be $1,000. (Hmm, I'm not so sure I'd actually be willing to invest that much given the vast likelihood of coming up short, which may be irrational of me; I'm glad I'm not a low-level psychic!) In general, I should be willing to pay anything less than the product of $1 million and the odds of my success, though the higher I think my chances are, the less I'll have to pay anyway. Huh, given that the usual final test has 1-in-a-million-odds (I think), should all applicants have to pay a dollar if they beat the preliminary and go onto the final?

Oddly enough, I've never heard of a statistician/mathematician who was even convinced of the existence of psi, much less that they themselves were psychic. (I guess they would spend their time at the casinos rather than the JREF, huh.)

 

[serpentine]

Since I am one of those people who has had 'paranormal' experiences...I, as yet, don't see how it's something that could be tested...

 

[EdG]

Would your gambler pay $3,500 to win $350?

I wonder why she would bother. Once word got out she could beat the house, she'd be blacklisted at every casino in the world.

Since I am one of those people who has had 'paranormal' experiences...I, as yet, don't see how it's something that could be tested...

In the case of my hypothetical gambler, the ability to pick an exact number an average 1 out of 7 spins of a roulette wheel is easily testable. Spin the wheel 70 times and she either can or can't hit the correct number 10 times.

Designing a test for other claimed paranormal abilities should not be impossible.

 

[Professor Yaffle]

I don't know whether it was accepted to the challenge, but there is the case of the (non) blinding of Randi by Paul Bethke.

He does seem to be in this list I found - of people who applied to the MDC?

http://www.docstoc.com/docs/72875570/Paranormal-Investigator-Application

 

[wardenclyffe]

Since I am one of those people who has had 'paranormal' experiences...I, as yet, don't see how it's something that could be tested...

You may be right. Many paranormal experiences are not properly testable. But lots of people here in the forum are able to come up with very clever ideas for testing things that seem, at first, to be untestable. If you've had paranormal experiences and you'd like a million dollars, what have you got to lose? If there's a way to test it, I think you've come to the right place.

Ward

 

[Pixel42]

Since I am one of those people who has had 'paranormal' experiences...I, as yet, don't see how it's something that could be tested...

At the risk of being accused of pedantry, I suspect that what you've had are experiences you can't currently explain. Whether the explanation is paranormal or mundane is something that can only be established by testing.

If the experiences are truly untestable then you will sadly never know whether they were paranormal or not, though if you share them someone may be able to at least suggest a non-paranormal hypothesis to explain them.

 

[Lenoxus]

I don't know whether it was accepted to the challenge, but there is the case of the (non) blinding of Randi by Paul Bethke.

Thank you!

If I considered such powers a real possibility, then I would have mixed feelings about giving someone the chance of a million dollars for demonstrating they could psionically hurt people… but really, if it were true, the world would have a right to know. I'd give such a person major kudos for going to the JREF instead of the CIA…

Anyway, the really important question is: if you set up a test where the claimant says he can blind the experimenter, do you call the test blinded? (Okay, I'm assuming I'm not the first to say that. )

 

[Dumb All Over]

Since I am one of those people who has had 'paranormal' experiences...I, as yet, don't see how it's something that could be tested...

Please share. Tell us about your experiences.

 

[tuoni]

Thank you!

If I considered such powers a real possibility, then I would have mixed feelings about giving someone the chance of a million dollars for demonstrating they could psionically hurt people… but really, if it were true, the world would have a right to know. I'd give such a person major kudos for going to the JREF instead of the CIA…

Anyway, the really important question is: if you set up a test where the claimant says he can blind the experimenter, do you call the test blinded? (Okay, I'm assuming I'm not the first to say that. )

I realise this isn't the MDC, but I vividly remember this, where a "Tantric Guru" was challenged by Sanal Edamaruku to kill him on live TV. It was actually quite pathetic to watch, to be honest. Major kudos to Sanal.

 

[Halfcentaur]

I realise this isn't the MDC, but I vividly remember this, where a "Tantric Guru" was challenged by Sanal Edamaruku to kill him on live TV. It was actually quite pathetic to watch, to be honest. Major kudos to Sanal.

Especially at one point when Sanal is laughing and the Guru can't help but start to laugh himself during a particularly energetic moment of heavy petting.

 

[NeilC]

2. Many psychics attach to their services the phrase "for entertainment purposes only" to avoid liability. Have any psychics, when challenged to take part in the Challenge, said "Hey, can't you read? What I do really is just for entertainment, like with a professional magician or a stand-up comic. Asking me to do this 'for real' is as dumb as asking someone at Weta Worshop to provide you with a 'real' orc." (Again, this probably hasn't happened, but it would be nice if it had.)

I could be wrong but I wasn't aware that the JREF went around hunting down magicians and mentalists and forcing them to take a challenge.

 

[Lenoxus]

I could be wrong but I wasn't aware that the JREF went around hunting down magicians and mentalists and forcing them to take a challenge.

Very good point. I guess I was considering the possibility of a genuine mix-up/miscommunication about someone's stchick.

Plus, I feel that by not doing this and instead implying that they have legitimate powers, those psychics who sometimes hide behind "entertainment purposes only" are being hypocrites, and if they were consistent, they would do the same thing when asked to prove their abilities. Instead they choose to engage in nudge-nudge-wink-wink-this-is-just-what-my-lawyers-make-me-do. Why can't they get registered as legitimate by a standards board? (Oh, that's right.)

 

[serpentine]

EdG...
"In the case of my hypothetical gambler, the ability to pick an exact number an average 1 out of 7 spins of a roulette wheel is easily testable. Spin the wheel 70 times and she either can or can't hit the correct number 10 times.
Designing a test for other claimed paranormal abilities should not be impossible."
---------
Ward...
"You may be right. Many paranormal experiences are not properly testable. But lots of people here in the forum are able to come up with very clever ideas for testing things that seem, at first, to be untestable. If you've had paranormal experiences and you'd like a million dollars, what have you got to lose? If there's a way to test it, I think you've come to the right place."

If I thought I had a knack for guessing what number's are written on the back of cards and such, then I might be willing to get tested, just for the fun of it. And I'd be buying a lot more lottery tickets.

I don't think some kind of testing is impossible, I just don't know what it could be in regards to certain kinds of abilities.
------------
Pixel...
"At the risk of being accused of pedantry, I suspect that what you've had are experiences you can't currently explain. Whether the explanation is paranormal or mundane is something that can only be established by testing.

If the experiences are truly untestable then you will sadly never know whether they were paranormal or not, though if you share them someone may be able to at least suggest a non-paranormal hypothesis to explain them."

Oh, I can 'explain' certain experiences alright.
I know what I have lived through, and I don't care much about proving anything.
I was taught not to believe in the 'paranormal', but then there comes a day, when you know you just have to admit that it's a reality. Simple as that.
And it is simple...it's just another part of being human...it is like a 'sixth sense'...
---------------
Dumb all over...
"Please share. Tell us about your experiences."

Since my experiences/dreams usually have to do with things deeply personal -death, or just plain awful things happening, I don't care to get into many details.
Dreams usually are in such symbolism that I have no idea what they are about until something happens and it's like the 'Aha' moment.
I'm aware not wanting to share details, might make others think I am making stuff up, or not genuine, or whatever, and of course, that's understandable.

Sometimes it's hard for me to imagine so many people haven't had a 'paranormal' experience, at least once in their lives.

 

[Pixel42]

Dreams usually are in such symbolism that I have no idea what they are about until something happens and it's like the 'Aha' moment.

http://en.wikipedia.org/wiki/Postdiction

I'm aware not wanting to share details, might make others think I am making stuff up, or not genuine, or whatever, and of course, that's understandable.

Most people who believe in the paranormal are not making stuff up or frauds, but simply mistaken. Just because you can't come up with a non-paranormal explanation of your experiences doesn't mean there isn't one. Unless you share the details we can't tell whether your conclusion that they were paranormal is justified, and (based on a great deal of past experience of such stories whose details were shared) will stick with the default assumption that it isn't.

Sometimes it's hard for me to imagine so many people haven't had a 'paranormal' experience, at least once in their lives.

Everybody does have such experiences. It's how they're interpreted that varies.

 

[Ladewig]

I wonder why she would bother. Once word got out she could beat the house, she'd be blacklisted at every casino in the world.

I am not convinced word would get out. If one were to spread one's play out over 90 Las Vegas casinos, then security will not really notice someone who walks out with 110% of one's stake every three months. Just don't join a players' club (which would allow the casino to easily track your winnings).

But even if you were completely certain that you would eventually get caught and blacklisted, why would that be a reason to not get a ten percent return on your bankroll once a day until you get caught?

 

[JoeTheJuggler]

3. Some folks have argued that the Challenge is unfair because the p-values and effect sizes called for are too extreme. Have any applicants claimed to have paranormal abilities with only a small effect size?

This isn't an answer to your question but a comment on the point it raises. Most of them claim 100% effect rates (using other words that mean the same thing), but then begin negotiating for any statistical advantage they can get. This demonstrates that at some level, they don't believe their own claims.

ETA: Similarly, many of them who make a living bilking people out of money will raise a host of quibbles when it comes time to testing that would never come up when dealing with their victims who come with cash. An astrologist, for example, will never turn away a paying customer because they don't know their exact birth time within one minute.

 

[hcmom]

Especially at one point when Sanal is laughing and the Guru can't help but start to laugh himself during a particularly energetic moment of heavy petting.

Someday Sanal will die, and then the Guru will claim he did it...

 

[EdG]

I am not convinced word would get out.

My response was in regard to a person bearing the expense of proving to JREF that their ability was real. I think if the person proved their claim, word would get out.

 

[xterra]

I could be wrong but I wasn't aware that the JREF went around hunting down magicians and mentalists and forcing them to take a challenge.

Now there's a picture to conjure with (pun intended)!

You -- you take the challenge or the JREF will wave its collective magic wand and turn you into a frog!

I was the one who originally asked "why would a person bother?" and was answered with the roulette example.

I agree that clearly, the casinos would ban such a person as quickly as they could.

 

[EdG]

Most of them claim 100% effect rates (using other words that mean the same thing

What is the basis of your claim? In reading the challenge protocols for Putt, success is defined as 50%. For Ziborov, it is 33%.

 

[Femke]

What is the basis of your claim? In reading the challenge protocols for Putt, success is defined as 50%. For Ziborov, it is 33%.

You do realise that the claim and the protocol are two separate things, right?

Usually the claimant comes in saying "I can <insert any paranormal claim>." They hardly ever say anything in percentages, but their claim is "I can do this", which suggests that they think they can do it. Period.

Let's say I say I can ride a bicycle. That does not include a percentage, but I think you will agree with me that the percentage is high. Not 100%, because I might fall off in a strong wind or when intoxicated.

When you agree to test my bicycle riding abilities, I would try and negociate a success rate as low as possible, to avoid that due to nerves I will trip and fall the first time, and fail while in fact I can ride a bicycle.
That is why the success rates for the protocols are usually quite friendly, slanted towards the claimant, if you will. Their imagined success rate will be much higher, so passing the challange should be easy.

 

[serpentine]

Pixel.
Re: postdiction.
The fact of something being symbolic does not discount the reality of it being related to future events. Since our minds dream in symbols all the time in regards to our day to day problems, and events, so our mind often uses the same avenue to relate symbolic meaning to future happenings. The arts deal with symbol all the time, and we use symbol constantly in daily reality. It's another language our brain uses as part of being human.
And there are sometimes premonitions that are not symbolic at all.

I've had too many premonitions during my lifetime to discount the plain reality of them.

"Most people who believe in the paranormal are not making stuff up or frauds, but simply mistaken. Just because you can't come up with a non-paranormal explanation of your experiences doesn't mean there isn't one. Unless you share the details we can't tell whether your conclusion that they were paranormal is justified, and (based on a great deal of past experience of such stories whose details were shared) will stick with the default assumption that it isn't."

Like I've stated before, I'm quite aware of what I have experienced so far. This subject is not new to me at all, since I've been familiar with it for more than 30 yrs. I've come to think of 'paranormal' experiences as simply a sensitive kind of intution. The fact of not knowing what your experience is about until the actual event, and things being vague as to dates, or times, does not negate the relationship. No-one but myself, can know this. I'm not ignorant about what so far, my experiences have been about.
All I'm willing to say right now is that the emotional energy surrounding a premonition is usually in direct relationship, but not equal to the actual later event. Usually, it was just my intution trying to prepare me for a sad, or traumatic event itself, other times an attempt to wake me up to what I was not consciously paying attention to.

Also, there are many children who become aware of their premonitions.
Like the child who simply stated to me one day that they dream something, and then it happens the next day.

Have a good one.

 

[Pixel42]

I've had too many premonitions during my lifetime to discount the plain reality of them.

This is an assertion which requires data to back it up. It is very easy to inadvertantly fool yourself into believing an assertion like this is true when it actually isn't. That's why the scientific method was invented: to enable us to gather the data needed to test such assertions. Until you have done so all you have is an impression, and precedent strongly suggests that it is a false impression.

Edited to expand on the "precedent" comment: dowsers believe they find water (or whatever they're dowsing for) far more often than would be expected if dowsing didn't work; astrologers believe that their readings are far more accurate than would be expected if astrology didn't work; homeopaths believe that their patients improve much more than would be expected if homeopathy didn't work; etc etc. All these are testable beliefs and, when tested, are invariably shown to be false. This is how we know that we have built in cognitive biases that can lead us to believe things which are not true, and why you can't simply assume that your subjective impression of the accuracy of your premonitions is correct.

 

[serpentine]

"This is an assertion which requires data to back it up. It is very easy to inadvertantly fool yourself into believing an assertion like this is true when it actually isn't. That's why the scientific method was invented: to enable us to gather the data needed to test such assertions. Until you have done so all you have is an impression, and precedent strongly suggests that it is a false impression."

No, I don't need data to 'back' anything up. You do. I know my life and my experiences, and you don't. So, try not to be so arrogant as to assume everyone automatically deludes themselves into something when they don't.
And no, it's not simply impression. It is quite distinctly relational.

"Edited to expand on the "precedent" comment: dowsers believe they find water (or whatever they're dowsing for) far more often than would be expected if dowsing didn't work; astrologers believe that their readings are far more accurate than would be expected if astrology didn't work; homeopaths believe that their patients improve much more than would be expected if homeopathy didn't work; etc etc. All these are testable beliefs and, when tested, are invariably shown to be false. This is how we know that we have built in cognitive biases that can lead us to believe things which are not true, and why you can't simply assume that your subjective impression of the accuracy of your premonitions is correct."

Wrong. I can, and do know how something relates to something else in my life. I've already had it made quite clear to me, and not always pleasantly so. If it was only simple belief it would be easier to disregard.
As I have stated before, it's a part of being human, and that I do know.

YOU need the proof that will satisfy YOU.

 

[Pixel42]

The person being arrogant here is you, insisting that you are not subject to the same unconscious biases and faulty perceptions as the rest of humanity.

Claims of the paranormal, whenever testing of them has been possible, have always been shown to be the result of the cognitive biases which are built into the way in which our brains work. Always. So unless you have evidence which demonstrates that there is some other explanation in your particular case I am completely justified in assuming that that is also the explanation of your experiences. You, of course, can believe whatever you like, but you have given no good reason why you or anyone else should believe that your perception that your premonitions are more accurate than would be expected by chance is correct. If you have such reasons by all means share them.

 

[Czarcasm]

YOU need the proof that will satisfy YOU.

Actually, if seems as if your satisfaction depends on your beliefs not being tested. You may hope there is a million dollars in the sealed box in front of you. You may believe there is a million dollars in the sealed box in front of you. You may brag to others about the million dollars in the sealed box in front of you.
But until it is opened and the contents are examined...all you've got is a box.

 

[hcmom]

No, I don't need data to 'back' anything up. You do.

Kinda depends on whether or not you want anyone else to believe you.

 

[LashL]

...

The 'quote' button is your friend.

 

[GzuzKryzt]

...
Also, there are many children who become aware of their premonitions.
Like the child who simply stated to me one day that they dream something, and then it happens the next day.
...

What you have described here is an excellent opportunity for a test: Let the person who claims a premonition write it down concisely and precisely and compare the results.

It is one of the many, many measures you can take to avoid fooling yourself.

 

[xterra]

What you have described here is an excellent opportunity for a test: Let the person who claims a premonition write it down concisely and precisely and compare the results.

It is one of the many, many measures you can take to avoid fooling yourself.

William, let me add to your reply, just for clarity.

Let the person who claims a premonition write it down before the event does/would/might happen, sign and date the document, seal it in an envelope in the presence of at least one witness, sign his or her name across the seal, and also date the envelope.

Then after the event does -- or does not -- happen, the premonition or prediction can be validated -- or not.

 

[SkepticScott]

I think you also need a control. Get a bunch of premonitions that failed but could still apply and have a third party evaluate all the premonitions without knowing which are the controls. After a while you can analyze the results and detect any statistically significant difference. It's amazing how easily a vague prediction can be made to fit the facts.

 

[Pixel42]

I think you also need a control. Get a bunch of premonitions that failed but could still apply and have a third party evaluate all the premonitions without knowing which are the controls. After a while you can analyze the results and detect any statistically significant difference. It's amazing how easily a vague prediction can be made to fit the facts.

Indeed. No-one disputes that premonitions occasionally come true, the question is whether they come true more often than would be expected as a result of chance/intelligent guesswork.

To find out it's necessary to not only record all premonitions and subsequent events and have criteria for what constitutes a hit agreed beforehand, but also to determine what that expected chance/guesswork accuracy rate is for comparison. The latter presents as many, if not more, challenges to the careful investigator than the former.

 

[xterra]

I think you also need a control. Get a bunch of premonitions that failed but could still apply and have a third party evaluate all the premonitions without knowing which are the controls. After a while you can analyze the results and detect any statistically significant difference. It's amazing how easily a vague prediction can be made to fit the facts.

This methodology might not be needed if a person were trying to determine if his or her own premonitions were valid. The prior recording of the premonition would be enough to evaluate confirmation bias. As Pixel42 notes here:

Indeed. No-one disputes that premonitions occasionally come true, the question is whether they come true more often than would be expected as a result of chance/intelligent guesswork.

To find out it's necessary [...] to determine what that expected chance/guesswork accuracy rate is for comparison. The latter presents as many, if not more, challenges to the careful investigator than the former.

As usual Pixel42 goes directly to the heart of the issue. Further, determining adequate criteria might be incredibly difficult, people being what they are -- as how could they be any other way?