Bias in Challenge Protocol?

[T'ai Chi]

Randi's challenge then, in a way (to me), is about opening new branches of science and learning.

You think then there would be peer review and journals, stuff like that.

 

[CFLarsen]

You think then there would be peer review and journals, stuff like that.

If you are thinking of those journals where Radin, Scwartz, etc. have to publish because their crappy experiments and non-existent results cannot be accepted in real scientific journals, sure.

As for "peer review"...well, you have to have qualified people to peer review something, but since the field is so narrow and they all know what they are working on, it is hardly "peer review".

 

[Moochie]

Oh. I was not aware of this. Thank you for telling me. Then perhaps someone can relay on his behalf should the situation arise (which appears doubtful).

No further action on my part is necessary, or warranted

M.

 

[gnome]

You think then there would be peer review and journals, stuff like that.

In all seriousness... what topics would you like to see the JREF publish and subject to peer review?

 

[Jackalgirl]

This has all been said before, so I will attempt to summarize:

1) There's nothing in the Challenge Rules that would prevent a claimant from having sympathetic observers be present.
2) The test protocol has to comprise multiple important factors:
a) it's mutually agreeable. That is, both the claimant and the JREF have to be okay with it.
b) the protocol should contain sufficient controls for alternate explanations or trickery ON EITHER SIDE. For example, people who claim telekinesis need to propose a protocol that excludes air currents and/or heat as a cause of the movement of the target object. Another example: double-blinding prevents representatives from the skeptical side from messing with the test.
c) its results are not subject to explanation or clarification. The claimant claims to be able to do x thing with y amount of accuracy. No judgement required, other than a verification that the claimant did or did not do x thing with y (or better) amount of accuracy.

The attitude of either the claimant or JREF is really irrelevant. Either claimants can do what they claim to do, or they can't. The fact that JREF representatives do or do not believe that they can will have no effect on an objective test of something that is actually happening.

Now, it's true that a bias on the part of someone with a hand in designing a protocol could have a profound effect: for example, if someone were to intuit the actual paranormal mechanism by which an effect is (supposedly) occuring and figure out a way to block it, and work that into the protocol -- that certainly would be unfair -- that is, if the claimant him/herself didn't know that's what was going on. But in all my years of following Randi's Challenge, I honestly don't think I've ever seen that. I /have/ seen some extremely sarcastic and dismissive messages from people like Kramer; however, they've usually been in response to several rounds of people being unable to explain what they propose to do, much less getting to the protocol-design phase.

Personally, I would think that any dismissive or negative attitude amongst the JREF representatives would be further incentive for me, if I were a claimant, to demonstrably prove them wrong and make them eat their words.

 

[Cuddles]

This is not a black or white proposition lol. I've been saying (ad nauseam) that Randi and the applicant should both present qualified candidates (whatever the expertise), rather than Randi presenting his candidates for both parties to choose from.

But you still do not seem to understand. No-one presents anyone. If an applicant comes up with a protocol, the JREF checks to see if it is any good. Since the odds are required to be 1:1,000,000 it seems logically to consult with a statistician to check the odds. Why do you think Randi should ask if a particular statistician is acceptable to the applicant? It's none of their business who he talks to. They should already have done the maths before presenting a protocol and are free to talk to any experts they like before and after the test. I just can't see why you are bothered that if an applicant suggests tossing coins, for example, that Randi will ask someone how many tosses they need to get correct for it to be a 1:million chance. Both he and the applicant are free to ask as many people as they like, but the answer will be the same every time.

Your point would be more valid if refering to people such as observers, but it is still a non-issue. At no point has Randi ever suggested that applicants can't provide their own friends to help out. You appear to be arguing about a problem that simply doesn't exist. Yes, of course we all agree that applicants should be allowed to choose people to help out and witness the tests. But they are allowed. The fact that they usually don't seem able to is no-one's fault but their own.

 

[gnome]

But you still do not seem to understand. No-one presents anyone. If an applicant comes up with a protocol, the JREF checks to see if it is any good. Since the odds are required to be 1:1,000,000 it seems logically to consult with a statistician to check the odds. Why do you think Randi should ask if a particular statistician is acceptable to the applicant? It's none of their business who he talks to. They should already have done the maths before presenting a protocol and are free to talk to any experts they like before and after the test. I just can't see why you are bothered that if an applicant suggests tossing coins, for example, that Randi will ask someone how many tosses they need to get correct for it to be a 1:million chance. Both he and the applicant are free to ask as many people as they like, but the answer will be the same every time.

Your point would be more valid if refering to people such as observers, but it is still a non-issue. At no point has Randi ever suggested that applicants can't provide their own friends to help out. You appear to be arguing about a problem that simply doesn't exist. Yes, of course we all agree that applicants should be allowed to choose people to help out and witness the tests. But they are allowed. The fact that they usually don't seem able to is no-one's fault but their own.

I should add that a JREF statitstician is not there to "judge" the test, but to advise JREF on whether they should choose to accept the protocol. The claimant still retains their right to accept or reject the protocol as well.

 

[ProbeX]

Well some of you are thinking that I don't understand the challenge. And I am just as confident that some of you are overlooking serious flaws in the challenge. And it is not enough to keep pointing out that the applicants choose the terms of the challenge. That is not a good defense or a buffer against bias procedure.

As it regards unilateral testing procedure, I have already stated:

Quote:

Their consultants also determine "experimental design"?? Why that contradicts the initial promise that both sides work together to negotiate the terms of the test.


Yes he stated that his consultants decide the terms of the challenge. And as stated earlier, yes there is a some latitude of [/i]subjective[/i] variance in concluding statistical outcome for alleged esoteric occurences. There seems to be a profound ignorance here, as it regards the field of statistics and its application in the context of objective testing (& research, which some of this involves). Also, ideally neither adversary should have familiar people w them during tests. There should be 3rd party, neutral involvement only.

I understand, as stated, Randi puts little weight on scientific procedure, as he believes scientists can be flawed. But science is the best measure of objectively determining what is real from what is fake (including what is or isn't proven as psi activity). Dr. Schwart's and Randi's approaches both could benefit from the use of proper science.

That I, you and Randi fall under the general cat. of "skeptic" does not relinquish any of us from the duty of addressing faulty procedure. And veteran magic pitted against "adversaries", as opposed to unbias, properly applied science, will render all positive or negative results useless in the eyes of most in the scientific world. And those are the foremost skeptics; the disciplined ones.

Ultimately the tackiest practice of all is Randi's attempt to mix formal, objective testing with gambling. Hardly a respectable "truth" standard lol.

 

[gnome]

Where does the gambling come in?

Also I'm trying to figure out what's biased about each side having their own people to rely on. The purpose is to generate a reliable test that the applicant will agree to.

 

[ProbeX]

Isn't it rather obvious this is a gamble?:

Randi has gathered funds he will theoretically lose to an "adversary", in the event that the adversary/opponent beats him in his challenge.

Doesn't sound like gambling to you? Here's more ...

Concurrently he is gambling w his (long-standing) reputation since he's invested in defending a paradigm (as a long-term skeptic), which will be threatened and potentially deconstructed by an adversarial party, should that party win.

Randi risks not just his reputation, but possible fallout from cash donors aligned w Randi's adversarial position. It's unlikely these skeptical adherents will execute cartwheels for joy after hearing their investments got awarded to a psychic.

As for why the challenge is biased - besides the fact of mixing gambling w alleged objective testing - I've supplied several reasons many xs over by now. If you aren't getting it by this point we're at an impasse.

 

[Darat]

...snip...

Randi risks not just his reputation, but possible fallout from cash donors aligned w Randi's adversarial position. It's unlikely these skeptical adherents will execute cartwheels for joy after hearing their investments got awarded to a psychic.

...snip...

The million dollars of the challenge prize was donated by one person and was donated for one use only - to fund the Million Dollar Challenge.

 

[ChristineR]

I have to agree with Darat on this one--although I had thought that the million came from a number of donors over a number of years. The people who gave that money wanted it used exactly this way. If they hear that someone wins the prize they'll be executing cartwheels for joy. It will be either a great advance in science, or a brilliant magic trick, both completely deserving of the prize--because that's what the prize is. A reward for a brilliant scientific discovery, or just for outsmarting Randi!

The money belongs to the JREF now, it's not like they'll get it back if no one ever wins.

 

[ProbeX]

The million dollars of the challenge prize was donated by one person and was donated for one use only - to fund the Million Dollar Challenge.

Ok, even if it's one donor, the same logic applies. The one donor, as you said, has donated $ strictly to a cause which will potentially line the pockets of an adversary of Randi and the foundation, and therefore him. How does this imply risk for both the donor and Randi?:

Where potentially relinquishing funds to an adversary is concerned, there is always the risk of a negative reaction and/or feeling of loss on the donor's part. With all due respect, it's plain silly to ignore this obvious fact of human behavior. People do not react predictably when losing money, regardless of the cause. Yes, she or he may rejoice in the idea of a seeming "breakthrough in science", but s/he may not. Why? For a number of reasons. A few will be listed but I will happily offer other scenerios upon request:

S/he may ultimately have problems w the test Randi applied ex post fact, on further consideration, thinking delegation of funds wasn't appropriate.

Btw, for reasons stated ad nauseam, the science world will not accept Randi's methods as scientifically sound. That the word "adversary" being applied to the test subject, in itself, is hugely problematic. Ergo a "scientific breakthrough" is out of the question lol.

... The fact of his "evidence" being refuted by the science world on grounds of sloppy approach and method is another potential trigger of animosity for a donor expecting the outcome of Randi's test to be taken considered a "scientific breakthrough". It's also possible the donor is expecting fame by associating. That is another expectation which has potential to have a backlash on Randi and his reputation.

I have to agree with Darat on this one--although I had thought that the million came from a number of donors over a number of years. The people who gave that money wanted it used exactly this way. If they hear that someone wins the prize they'll be executing cartwheels for joy. It will be either a great advance in science, or a brilliant magic trick, both completely deserving of the prize--because that's what the prize is. A reward for a brilliant scientific discovery, or just for outsmarting Randi!

The money belongs to the JREF now, it's not like they'll get it back if no one ever wins.

Please refer to thoughts above. "Brilliant scientific discovery" requires strict adherence to scientific conditions and methods that are lacking in the "Challenge" (as formerly noted).

Finally, (for both of you), how is Randi not gambling with his reputation and the danger of having his beloved, long-held adversarial paradigm toppled?.

 

[gnome]

in that sense you might call it a gamble... i would just point out that it is not a game of chance... or (theoretically) a gamble on the part of the participant. If they can reliably perform under agreed upon conditions, it should be a sure thing.

 

[ProbeX]

Gnome, even though we likely won't agree about this subject, I appreciate the reasonable & respectful tone you bring to our discussion.

My response to the above is, the initial onus of proper conductivity of testing, is on the person posting the challenge, not on the responding challengers.
That someone agrees to undertake a poorly designed test "challenge", does not justify, nullify or rectify that poor testing approach.

If a person wishes to promote a series of tests as objectively credible, he can't have stakes - even indirectly - in the outcome. Gambling is antithetical to the preservation of objective testing. The admixture is a universally accepted research & testing faux pas.

Though it may seem I'm against Randi himself, I'm not. Debunking charlatans and helping ppl see the ways in which they fool themselves is nobel. It's his haphazard approach that concerns me. Do it objectively or don't do it at all.

 

[gnome]

Gnome, even though we likely won't agree about this subject, I appreciate the reasonable & respectful tone you bring to our discussion.

Thank you. It is refreshing to have a less rancorous discussion from time to time

My response to the above is, the initial onus of proper conductivity of testing, is on the person posting the challenge, not on the responding challengers.
That someone agrees to undertake a poorly designed test "challenge", does not justify, nullify or rectify that poor testing approach.

This is quite correct--however, I think that JREF has a good record in this respect. Are you thinking of a particular case where the test methods were flawed?

If a person wishes to promote a series of tests as objectively credible, he can't have stakes - even indirectly - in the outcome. Gambling is antithetical to the preservation of objective testing. The admixture is a universally accepted research & testing faux pas.

Though it may seem I'm against Randi himself, I'm not. Debunking charlatans and helping ppl see the ways in which they fool themselves is nobel. It's his haphazard approach that concerns me. Do it objectively or don't do it at all.

Were it not for a large prize money offered, though, it leaves an excuse for the charlatan to ignore him... that there's no reason to pay him attention. With the challenge... people will legitimately wonder why someone who claims a talent wouldn't use it to grab a cool million, especially if they prove willing to sell their talents for chump change.

 

[ProbeX]

Well ok, I already gave one clear ex. of how method/procedure is flawed. In posted info about the Challenge, it has been stated that in the past, Randi occasionally sits in to observe his adversaries as they undergo testing. For example, I believe it's stated that Randi intended to sit in on Lou Gentile's testing (if Gentile had agreed to be tested). This practice is never, under any circumstances, acceptable as part of sound objective testing in the field of professional science, or as part of any professional, objective research/testing model or procedure. And that is specifically what Randi claims to do: research (of subjects and stat.s, etc.) and testing; quantitative in nature.

It's quasi-science, and for some reason this seems to be escaping many people supporting the Challenge. Perhaps it's because otherwise intelligent ppl find the idea of a magician publically humiliating bleevers and frauds, so sexy that they'll blindly jump on the Randi bandwagon, regardless of the glaring errors in his approach. Or perhaps they're poorly educated about properly executed, formalized testing procedures.

That he conducts research (of stats, etc.) and testing of adversaries may be cute to non-scientists and many emotional skeptics (not saying that's you), but it's laughable and oxymoronical to any of us who have ever conducted balanced, responsible testing of human subjects; who routinely execute countermeasures for illiminating tester bias leading to questionable results. More below ...

 

[ProbeX]

You said:

Were it not for a large prize money offered, though, it leaves an excuse for the charlatan to ignore him... that there's no reason to pay him attention. With the challenge... people will legitimately wonder why someone who claims a talent wouldn't use it to grab a cool million, especially if they prove willing to sell their talents for chump change.

Because he is using the lure of a large cash reward for a positive testing outcome, Randi is likely drawing a disproportionate % of charlatans as opposed to people who truly believe they have psi abilities, but are savvy enough to know his testing procedure is too flawed and biased to bother with.

So Randi - by design - is attracting a limited fraction of believers in psi, mostly ppl driven by fortune and fame, as opposed to those attracted by legitimate, respectable, research & testing w/in an objective context.

His decision to mix a large cash prize w testing has resulted in two misleading impressions.

1. That psi abilities don't/can't exist because no one has won his (objectively flawed) challenge.

2. Everyone attempting to prove psi abilities are greedy charlatans and whackos.

The first impression may or may not be true and the second is simply false.

And I can tell you, if I wished to be tested for alleged psi, I'd go to someone who leaves money out of it and concentrates on authentic research. I would never recommend anyone take his Challenge seriously LOL. Nothing against him personally, just my opinion.

 

[gambling_cruiser]

ProbeX just tell us which psi abilities exist

 

[robinson]

Ah, Grasshopper

His decision to mix a large cash prize w testing has resulted in two misleading impressions.

1. That psi abilities don't/can't exist because no one has won his (objectively flawed) challenge.

2. Everyone attempting to prove psi abilities are greedy charlatans and whackos.

The first impression may or may not be true and the second is simply false.

Astute skeptical reasoning. You might imagine your insight and intelligence will be welcomed here. Turning the spotlight of reason and logic onto the MDC.

And I can tell you, if I wished to be tested for alleged psi, I'd go to someone who leaves money out of it and concentrates on authentic research. I would never recommend anyone take his Challenge seriously LOL. Nothing against him personally, just my opinion.

Just working out a protocol for a demonstration, as suggested by the MDC, as well as reading the FAQ page, is very helpful for determining if something CAN be demonstrated. I try and look at it from Randi's point of view. I really think he hates frauds and cons who rip people off , in time and money and emotional damage, by claiming and charging for powers that don't stand up to scrutiny. The MDC is a tool, and it didn't start off as a million, it was 10 Grand of his own money at first.

And it isn't that only challenge.

Leading to a related yet fascinating offshoot here ...