What standards can one expect as an applicant?

[saizai]

I initially inquired about this challenge in August '05; again last August, and formally applied last November.

So far, my experience has been:

• no action yet taken to follow up on my application, start formal negotiations over terms, or start the claim - so far that's 5 months (by contrast in the challenge archives, notice how they respond when applicants are not speedy and polite in their responses)
• routine outright, unjustified rudeness from Randi
• routine rudeness from Kramer & Jeff before I formally applied
• major logical flaws and fundamental misunderstandings about the statistics of double-blind randomized controlled trials from Jeff
• polite brushoffs from Jeff after I formally applied

This is despite all of my correspondence being unfailingly polite, my application* being polite, clear, and articulate, etc. Perhaps Randi & Jeff are so used to dealing with rambling incompetents that they treat all others the same?

I am aware that in the last few months there has been a conference, some work on revising the challenge rules, etc. Certainly these explain a certain amount of delay. How much?

So the question is: as an applicant, what should I expect in the way of standards of treatment? How long is reasonable to be required to wait before receiving substantive reply for negotiation of "mutually agreeable" terms? How much rudeness is reasonable when I have not myself been rude?


* In case you feel like debating the merit of my application: I have since changed some of the specific terms in it by email; I left the original document intact. The only major change is a switch from three rounds to two, with the first round being the 'preliminary test' and having an arbitrarily decided Score Equation (tentatively proposed to be the SF-36v2 HRQOL); the second round being the 'final test' and having a SE based on data from the first round.

 

[Darat]

...snip...

So the question is: as an applicant, what should I expect in the way of standards of treatment? How long is reasonable to be required to wait before receiving substantive reply for negotiation of "mutually agreeable" terms? How much rudeness is reasonable when I have not myself been rude?


...snip...

Haven't you said that the JREF has declined your application? Or do I misunderstand your "polite brushoffs from Jeff after I formally applied" comment?

 

[GzuzKryzt]

Saizai, have you received a response to your application; a response being an acceptance or a rejection?

Wasn't that the JREF's main argument that your claim would be impossible to test because of the long duration? (One year for each phase?)

 

[saizai]

Haven't you said that the JREF has declined your application? Or do I misunderstand your "polite brushoffs from Jeff after I formally applied" comment?

You do in fact misunderstand. Jeff acknowledged and accepted my application. He has yet to actually start acting on it. The brushoffs I refer to are variants of "still haven't gotten to this yet".

Wasn't that the JREF's main argument that your claim would be impossible to test because of the long duration? (One year for each phase?)

I specifically asked this, and Jeff specifically said that the long duration is not in principle a problem, and that they would keep the challenge open so long as progress was being made.

(pointless personal insults and threat of violence)

Reported for violation of JREF Forum rules.

P.S. In case you misunderstood my challenge, I require that all patients participating - both the control and active groups - be undergoing normal mainstream medical treatment simultaneously.

P.P.S. Isn't it a touch ironic that, in a post where I discuss the rude responses I have gotten to my politeness, you then exemplify that unfortunate behavior?

 

[GzuzKryzt]

...
I specifically asked this, and Jeff specifically said that the long duration is not in principle a problem, and that they would keep the challenge open so long as progress was being made.
...

Weren't you required to modify the protocol in some way and didn't the discussion kinda run into the ground as it got technical with statistics? Wasn't the ball in your court?

Either way, I suggest you contact Jeff Wagg directly: challenge@randi.org

 

[saizai]

GK - The discussion, currently, is in Jeff's court. He has repeatedly said "later" when I have asked for followup.

The Atheist -

[FONT=arial, helvetica]The following are some of the behaviors that are not acceptable: [/FONT]

• [FONT=arial, helvetica][/FONT][FONT=arial, helvetica]
  [*]Advocacy of suicide or of violence to others: If you tell someone to kill themselves, or to harm someone else, you will be suspended without prior warning. Example: "Go kill yourself."
  [*]Extremely cruel or hateful content directed toward another user: This includes content in private messages.
  [*]You will not post anything that is considered to be potentially criminal. The posting of computer viruses, child pornography, or links to computer viruses or child pornography is prohibited. Posts made under circumstances indicating a considered likelihood of inciting a violent or felonious act, or a subjective intention or knowledge that its content will be used for, or in furtherance of, any criminal purpose are also prohibited. Such posts will be moved offline and referred to proper legal channels.
  [*]You will not post anything that is considered indecent, pornographic, obscene, or contains excessive reference to violence.[/FONT]

Given your two postings containing specific personal insults, and a direct threat of violence ("In fact, having read your claims and your site, I can tell you that I'd be a lot more than rude, I'd be downright violent."), and your previous suspension, I have requested that you be banned.

I will have no further correspondance with you.

 

[GzuzKryzt]

Keep knocking on their door. If your application has been formally accepted, you should insist on the next step to be taken.

It seems the JREF is understaffed, but I would not take that as an excuse for not acting upon an accepted Challenge.

 

[Jekyll]

I feel that the best way to go about doing this would be to set up the system, run the first round and then bring the definite protocol based on these results to the JREF.

This would:
1) show the JREF that you are capable of running and managing such a test and not time wasting, nor trying to piggy back on their reputation.

2) It would give a much cleaner and simpler test for the JREF to consider.

But hey nitpicking aside, it looks like a reasonable first pass at a protocol to me.

 

[Jackalgirl]

I initially inquired about this challenge in August '05; again last August, and formally applied last November....

Howdy, saizai --

I can't open your .DOC file, I'm afraid -- would you be willing to post it here, inside a message?

Is your application and/or correspondance in the Challenge Applications subforum anywhere? If so, I'd appreciate a link (I don't know your real name) so that I can take a look.

If there isn't a thread for it in the CA subforum, would you please post your latest protocol, completely described, here for us to take a look at? I'm presuming that you're at the protocol-negotiation stage, waiting for Jeff to respond to a proposed protocol, yes?

Lastly, to answer your question, I personally think that there are a couple of things going on. Understand that I'm not speaking for JREF, Kramer, or Randi here; I'm basing this on my impressions of things I've read in the Challenge Application section.

Firstly, I think that Randi and Kramer (and now, Jeff) expect all applicants to read the rules. Quite a few of the brusque replies I've seen have come because applicants clearly have not done so; for example, despite the Challenge info pages stating to direct all correspondance to challenges@randi.org, some people write directly to Randi and are upset when he replies brusquely (and invariably tells them to write to the challenges email). Or you'll have people who try to change the rules.

Secondly, I think that JREF also expects applicants to read the FAQ before asking questions. Again, it becomes frustrating when people keep asking the same frequently-asked questions...which are answered in that file -- or, at least, were before the Challenge changed. Questions/applications like "I don't believe the money exists" are very frustrating, because it's been demonstrated time and time again that it does exist.

Now that the Challenge has changed, JREF is looking to build a new FAQ and is open to new questions, but I would expect them to become a bit exasperated by questions that ask for information directly answered in the Application section ("Can I change rule number x?").

So, in other words, a large amount of frustration occurs because people do not do their research before jumping in with both feet.

Also, one of the points made in the original FAQ, if I'm not mistaken, is that the nature of the Challenge is an adversarial one. It cautions people against having thin skins, in essence. That's another source of friction: people were offended by, say, Randi's brusqueness, or Kramer's complete lack of patience with people who didn't understand the rules.

When the protocol negotiation phase breaks down, it's usually because the applicant doesn't truly understand the nature of a properly-controlled test (or, in some cases, doesn't want to design one because he/she plans to use trickery to win the money). Mutual frustration occurs because the applicant just can't grasp why the test has to be so controlled and detailed, and because JREF starts to perceive the applicant as trying to sabotage the negotiation phase.

So, to sum it all up, I would say that a potential applicant should expect:
-1) polite treatment as long as he/she:
--a) does his/her research
--b) understands and abides by the rules
--c) wholeheartedly participates in the process of designing a good controlled test and
--d) is polite him-/herself

I don't know what to tell you about the delay in responding, because I don't know what's going on. However, I would personally expect relatively quick response time, since this is one of JREF's most public and prominant functions (that is, the Challenge). So, for me, that's the most troubling aspect of this whole thing.

 

[rjh01]

I cannot see why this claim should be acceptable.
1. Saizal has not given any evidence that the experiment can work. In the old rule three affidavits would be required. I cannot see how Saizal can produce these.
2. Similar studies have been done before and failed.

Links
News Flash! Prayer doesn't work....Duh
Prayer tag

 

[jsfisher]

I cannot see why this claim should be acceptable.
1. Saizal has not given any evidence that the experiment can work. In the old rule three affidavits would be required. I cannot see how Saizal can produce these.
2. Similar studies have been done before and failed.

Links
News Flash! Prayer doesn't work....Duh
Prayer tag

More to the point, there is no claim. Instead, it is a proposal to conduct a study, and if the results are by chance anomalous from what we might expect, the applicant expects the $1,000,000 pay-out. Moreover, the applicant gets to diddle with the scoring function freely.

 

[ObscureReferenceMan]

Saizai,

You ask what standards one can expect? Well, I would say you should be prepared for a few things. Here's a short list off the top of my head (understand, I am not a JREF staffer, just a forum member and fan of Randi's):
1. Don't expect immediate answers to your queries. The JREF staff is very small, and does a lot of work. Keep this in mind. (Maybe Jeff's "brushoffs" are attempts to give a brief response, as opposed to no response. I don't know, I haven't seen examples of your correspondence. See below.)
2. Randi has made a career of "skeptical investigation", and has dealt with a lot of (to put it bluntly) crackpots. I'm not saying you are one, but try to understand where he is coming from. I love the guy, but he can be a bit... grouchy. (In Nabukadnezar's poll about the Peabrain debacle, I am mostly on Peabrain's side - I thought Kramer was a bit rude.)
3. Not reading or not understanding the rules might lead to brusk responses (see #2).

You list several experiences you have had so far with Jeff and Randi, yet you give us no evidence of their occurrence. As skeptics, you have to realize we will doubt you. Not call you a liar necessarily, but at least ask you to back up your claim. Would you post examples of the bad treatment you've received?

I read your application (briefly), and I see a few items that are unclear. I am not going to critique it, but I could see where Jeff &/or Randi might have a problem with it. Also, it doesn't seem like you have a paranormal claim. But if Jeff/Randi have approved it, that's a moot point. But a few question:
- Has your application been approved?
- Are you aware that the protocol must be agreed on be both parties (i.e. you and JREF)?

 

[Jeff Wagg]

Saizai, if you want to continue working on your claim please contact me via e-mail. I've just reread the 40+ e-mails exchanges we've had.

Because you contacted Randi directly and repeatedly, which we specifically ask you not to do, you've earned his ire.

I also sent you a postal letter not too long ago.

 

[petre]

Cool. Hopefully saizai will let us know when it arrives and what he needs to do to continue.

 

[colin]

--snip--
I read your application (briefly), and I see a few items that are unclear. --snip--

Just a few? That thing’s a nightmare for a small group like the JREF! It’s way too long and complex with too many variables for them to possible police. And what exactly is considered successful? Something slightly skewed to be a wee bit above statistical average?

The best tests are ones were the results are obvious. Let’s say you get someone who’s missing an arm. Then have thousands, no millions, of the faithful pray their hearts out to the deity of their choice for the person’s arm to grow back. Now if that worked that would be obvious and clearly demonstrate the power of prayer!

And from your website, I don’t think this is exactly true:

“There have been many studies before on this of course; however, there has never been one that was both of truly significant scope AND conducted properly, that showed noticeable or replicable effect. ”

Although there haven’t been many well run studies, the ones that were showed no benefit to prayer. In fact, in one recent one the prayed for group did slightly worse than the control group. Is that he kind of little blip that you’re looking for, but in the opposite direction, to win the mil?
Of course, there’s the fairly recent study that showed a big benefit to prayer in regard to women’s reproduction. Oh, wait, that was found to be completely fraudulent! Who knew?!

This is from your site too:

“However, this does NOT prove that prayer does NOT work - only that those studies were unable to demonstrate it.”

It seems to have some typos. I'll fix it for you:

However, this does prove that prayer does NOT work - because those studies were unable to demonstrate it.”

Your welcome.

 

[colin]

Because you contacted Randi directly and repeatedly, which we specifically ask you not to do, you've earned his ire.

Can't follow directions either? That's always helpful.

 

[saizai]

I can't open your .DOC file, I'm afraid -- would you be willing to post it here, inside a message?

It's a bit long. And this post is not about the merits or lacks of my claim. There is a separate thread with that.

Is your application and/or correspondance in the Challenge Applications subforum anywhere? If so, I'd appreciate a link (I don't know your real name) so that I can take a look

The subforum has not been updated in months. The name used should be Sai.

Firstly, I think that Randi and Kramer (and now, Jeff) expect all applicants to read the rules.... Secondly, I think that JREF also expects applicants to read the FAQ before asking questions.

I have done so, and was quite clear about it upfront. I have not asked anything that was answered in the FAQ unless it was specifically for clarification, in which case I referenced the FAQ. While I realize not everyone does so, I do not consider it fair to be treated as an idiot because others are.

Also, one of the points made in the original FAQ, if I'm not mistaken, is that the nature of the Challenge is an adversarial one.

Adversarial should not mean impolite. One does not get insults and needling in a well-run courtroom, though that is an adversarial challenge.

All your other comments do not apply to me.

I cannot see why this claim should be acceptable.
1. Saizal has not given any evidence that the experiment can work. In the old rule three affidavits would be required. I cannot see how Saizal can produce these.
2. Similar studies have been done before and failed.

Neither point is relevant. Affidavits are only and have only been required for super-extraordinary claims, e.g. that someone can fly at will without equipment.

More to the point, there is no claim. Instead, it is a proposal to conduct a study, and if the results are by chance anomalous from what we might expect, the applicant expects the $1,000,000 pay-out.

The 'claim' is that the study will produce a positive outcome. That I am personally agnostic about what its outcome will be is irrelevant (and is a proper stance for a researcher to take on speculative research).

Moreover, the applicant gets to diddle with the scoring function freely.

Not so. I get to "diddle with it freely" within constraints, i.e. the only ones logically necessary to preserve JREF's interests in preventing cheating, logical fallacy, etc. The scoring function is set before collecting the data it is going to be used to score, and does not have any access to the control/active group status of the person it is scoring. As such, there is no way for me to design a scoring function that would be unusually likely to create a false positive.

1. Don't expect immediate answers to your queries. The JREF staff is very small, and does a lot of work. Keep this in mind. (Maybe Jeff's "brushoffs" are attempts to give a brief response, as opposed to no response. I don't know, I haven't seen examples of your correspondence. See below.)

I don't. But 5 months?

2. Randi has made a career of "skeptical investigation", and has dealt with a lot of (to put it bluntly) crackpots. I'm not saying you are one, but try to understand where he is coming from. I love the guy, but he can be a bit... grouchy.

I am indeed aware of his background, and of some of the fairly... unpleasant people he has interacted with before. I am not one of them. I expect to be treated in keeping with my behavior, not with that of others.

3. Not reading or not understanding the rules might lead to brusk responses (see #2).

I did so.

You list several experiences you have had so far with Jeff and Randi, yet you give us no evidence of their occurrence. As skeptics, you have to realize we will doubt you. Not call you a liar necessarily, but at least ask you to back up your claim. Would you post examples of the bad treatment you've received?

I didn't consider it necessary, and at this stage would prefer not to out of politeness to Jeff and Randi. The question was 'what can I expect'; my experience is merely the trigger for asking the question. I am not putting my experience on trial.

- Has your application been approved?
- Are you aware that the protocol must be agreed on be both parties (i.e. you and JREF)?

I believe so, and yes. Your other questions are answered above.

Saizai, if you want to continue working on your claim please contact me via e-mail. I've just reread the 40+ e-mails exchanges we've had.

I did so. I am still awaiting a response to my last, per your previous response that you would get back to me by email. This thread is not an attempt to contact you, but an entirely separate discussion.

Because you contacted Randi directly and repeatedly, which we specifically ask you not to do, you've earned his ire.

I contacted Randi in only a couple instances, when
a) I was submitting, together with the challenge address;
b) I had no response from Kramer, and wanted to know what was up; and
c) to respond to his emails.

I don't think that any of that is inappropriate, or justifies hostility or rudeness.

I also sent you a postal letter not too long ago.

I have not received it. Please send me an email copy & let me know what address you sent it to; my mailing address has occasionally had some 'issues' with getting to me properly.

Just a few? That thing’s a nightmare for a small group like the JREF!

I'm sorry that a fairly normal double-blind randomized control trial count as 'a nightmare' for you.

It’s way too long and complex with too many variables for them to possible police.

The only variables necessary to "police" is whether recipients are contacted, and whether the blinding is secure. Both are relatively simple matters - an affidavit in the first case, and computer security in the second.

And what exactly is considered successful? Something slightly skewed to be a wee bit above statistical average?

Any statistically significant difference.

The best tests are ones were the results are obvious. Let’s say you get someone who’s missing an arm. Then have thousands, no millions, of the faithful pray their hearts out to the deity of their choice for the person’s arm to grow back. Now if that worked that would be obvious and clearly demonstrate the power of prayer!

Indeed that would be very nice. But I am not claiming that effects that large can be produced. I get to decide what I'm claiming and what I'm not. I'm sorry my claim is not as flashy as some others'.

Although there haven’t been many well run studies, the ones that were showed no benefit to prayer. In fact, in one recent one the prayed for group did slightly worse than the control group. Is that he kind of little blip that you’re looking for, but in the opposite direction, to win the mil?

I've read that study. It was interesting. I haven't yet decided whether I would prefer the claim to be a blip in either direction, or a blip in one (with double the acceptable range to still get the same statistical significance).

It seems to have some typos. I'll fix it for you:

However, this does prove that prayer does NOT work - because those studies were unable to demonstrate it.”

Fallacy of argument from ignorance, please try again.

And since you felt a need to correct my 'typo':

Your welcome.

That's "you're welcome".

"You're" = "you are". "Your" = second person possessive.

 

[jsfisher]

The 'claim' is that the study will produce a positive outcome. That I am personally agnostic about what its outcome will be is irrelevant (and is a proper stance for a researcher to take on speculative research).

If you are personally agnostic about the outcome, there is no claim, only a speculation. You are speculating that the laws of chance may look favorably on your project thereby netting you $1,000,000.

 

[saizai]

If you are personally agnostic about the outcome, there is no claim, only a speculation. You are speculating that the laws of chance may look favorably on your project thereby netting you $1,000,000.

Nothing in the challenge rules require me to be certain about the outcome, only to want to give it an honest try, which I do. That has been true of other (accepted and tested) applicants as well. As for making money, I have other ways to do so that have far higher expected ROI.

Your speculation about what my intentions or motivations are is just that - speculation. (And, incidentally, they're wrong... but I am keeping my motivations to myself. The $1m per se happens to be somewhat incidental to them.)

 

[Jekyll]

I've read that study. It was interesting. I haven't yet decided whether I would prefer the claim to be a blip in either direction, or a blip in one (with double the acceptable range to still get the same statistical significance).

If you intend to learn how the distribution is skewed from the initial sampling you should be betting on a single direction, even if this direction is based on something like the nth moment of the sample.

It doesn't make sense to spread your bets, and to say that you expect the first measurement to be either above or below the second based upon the initial measurements.

A two tailed distribution is basically an admission that you don't think your idea will work, and that you just want to hedge your bets.