The "Sylvia Browne Protocol": Unaddressed Flaws

[Clancie]

Dear Mr. Randi,

I have moved this from the other thread, as it was a derail. As I noted there, there are serious flaws in the protocol of "Sylvia Challenge", flaws that continue to be totally ignored .

These problems in the test you proposed to her, in fact, are so serious that, in my opinion, to leave them unaddressed undermines the credibility of the Challenge itself (particularly as this is the most detailed JREF Challenge protocol we know of).

This has only to do with the protocol, not with Sylvia or her (alleged) abilities. If any of the critical points below are in error, I will welcome your corrections:

• 1. Not Testing the Claimed Ability.
  
  You say that the Challenge is supposed to test the claim (in this case, mediumship). But your Sylvia protocol doesn't look for mediumship at all. It tests for cold reading.
  
• 2. The Results are NOT “Self Evident, no scoring needed”
  
  Hypothetically, Sylvia could get a perfect rating (“10”) from the one person being read, and still fail your Challenge protocol.. This is a basic design flaw, IF one is looking for someone with genuine paranormal ability. She could have demonstrated it, to the maximum extent possible in your test, and still failed, due to your design.
  
• 3 Collusion is Not Ruled Out .
  
  The protocol in no way rules out the possibility that raters (only two of them are needed to "throw" the results) are colluding with you/JREF to give scores that ensure that she doesn't win.
  
• 4. Success or Failure Depends on Scoring (and possible bias) of Those NOT Read
  
  It is a very poor protocol to have her read only one (1) of the ten people, and have her "Passing/Failure" dependent on the evaluations of people she has NEVER read (and who, even worse, realize that they were never read.
  
  They also know how the scoring will be weighted in advance--so, hasn't the possibility of rater bias has been intentionally interjected into the test design?
  
• 5. Odds are Quoted That Make No Sense. What is Their Source?
  
  I have never seen any source for the "Challenge" odds of 50:1. Who provided JREF with these odds?
  
• 6. How Will These All-Important ‘Raters’ Be Selected?
  
  If the participants are supposed to "believe" in Sylvia's powers (a criteria I think is completely unnecessary, btw), then why not have Sylvia involved in selecting them, rather than you?
  
  Or, more to the point, why not have a mutually agreed on third party (e.g. Larry King, as you suggested on CNN) do the selection--and the testing? At a minimum, it would help rule out the possibility of collusion.
  
• 7. How Does Your SB Protocol Rule Out Subjectivity in Judging? .
  
  According to the Challenge rules, there is to be no “judging”, no subjectivity in determining the result. The rating process for the Sylvia Challenge that you is, sadly, rife with subjectivity and, in fact, does not yield “self evident” results at all.
  
• 8. Finally, a Side Note: Is this the Preliminary test for Sylvia? Or is it, as it sounded on LKL, the Final, the test she takes “for the million”?

And, just to note again, I am no fan of Sylvia. This is not about her. It is, pure and simply, about whether or not the protocol you’ve proposed for testing her is (1) a good one, and (2) whether it is consistent with the Challenge principles. As far as I can tell, it is neither.

Thank you in advance for any clarity you can shed on these issues. I am more than willing to be shown I am mistaken on each of these key critical points.

 

[Clancie]

I also offered two suggestions of my own. They are below, followed by your reply.

Posted by Clancie

Two more issues related to that above:

Question #1:

If someone passes the preliminary test, my understanding is that you can then propose any changes you want to in the protocol for the final. If the claimant does NOT agree to changing--and wants to continue as agreed on in the prelim--is this a "deal breaker"?

And, if it is, how does that NOT just "create an out"--i.e. hypothetically, at least, the claimant has succeeded at the prelim and seems likely to succeed at the final as well, thereby claiming the million. Hypothetically, what prevents you/JREF from requiring changes in the protocol at that point, simply to take advantage of this loophole, knowing they will be completely unacceptable to the claimant? (Again, hypothetically speaking).

If that happens, isn't that simply creating a "deal breaker" in which..."no final test protocol has been agreed to so....no test is possible to be given"...i.e. no money is ever awarded?

********

Question #2:

Would you agree to a protocol for mediumship testing with 10 people, all of whom are read, silent sitter style (phone okay). The readings are transcribed. People identify theirs.

If 4 or more of the 10 correctly identify the reading that fits them, Sylvia wins. This test statistically would meet your desired criteria of 50:1 odds or better).

Wouldn't this be a preferable protocol, Mr. Randi, since it would actually be measuring the ability that has been claimed?

Or, if not, I'm curious....why?

Thank you in advance for your reply.

Posted by James Randi, in response to the above:

For Clancie: as for your second posting, you ask --

#1: "If someone passes the preliminary test, my understanding is that you can then propose any changes you want to in the protocol for the final. If the claimant does NOT agree to changing--and wants to continue as agreed on in the prelim--is this a "deal breaker"? "

Your "understanding" is incorrect. You've been reading the canards that have been circulated endlessly, despite my denials. I have no right -- nor does the applicant-now-become-claimant -- to change anything but the odds, at that point -- because the prize is now at stake. But Clancie, let's wait for someone -- ANYONE! -- to pass the preliminary test, shall we? I don't really have to plan for floods in the Sahara....

As for #2, we have a much more elegant, simpler, definitive protocol designed. We're in control, we know what we're doing.

James Randi.

 

[Interesting Ian]

Hi Clancie,

Randi probably doesn't peruse the forums much so most probably he won't see your questions. You could maybe send him an email to let him know that you've started this thread?

 

[Dancing David]

I thought it was more of a suggestion than a protocol.

I agree however with the subjective ratings issue, I think that iw would be better come up with a 'list of data' for each readee that contains significant life events and that the list be generated prior to the reading. Then there could be a match rate thatw as looked for as positive.

The only problem I see with it is that the readee may have life issues that they would not want to discuss that would come out of a reading or that there might be something that could be considered 'unimportant' by the readee in generating the list.

So there might have to be a second phase of generating the list where a third party could ask questions of the readee, like ' were you ever in prision or jail' to discover such information without confounding the reseults. What powers does Sylvia claim to have is a very significant question.

 

[Darat]

I also offered two suggestions of my own. They are below, followed by your reply.

Clancie - not quite so sure from your post so can you clarify?

It looks as if Randi is saying no changes between prelim and challenge (if the prelim is passed).

If SB ever does respond he has an updated or new (?) protocol to suggest to her?

 

[Clancie]

Posted by Darat

It looks as if Randi is saying no changes between prelim and challenge (if the prelim is passed).

Is he? I thought he meant there would be no changes in the prelim and final testing protocol, but that he can still change the odds he's quoting that someone needs to beat--i.e. he can change the criteria--the odds she needs to beat--for passing the final. Actually, I really don't know -what- he meant by that sentence.

Posted by Darat

If SB ever does respond he has an updated or new (?) protocol to suggest to her?

Did he say that? I don't see it, and I've never heard one mentioned. (It would seem an important detail, since she's already -accepted- the first one).

You think the "elegant" version referred to something new rather than the one he suggested on LKL?

I hope he will clarify, because if there's a new one, shouldn't it be mentioned instead of (or along with) the one linked on the website?

 

[CFLarsen]

Clancie,

Why not just email Randi and tell us what he said to your (giggle) list?

 

[davidhorman]

You say that the Challenge is supposed to test the claim (in this case, mediumship). But your Sylvia protocol doesn't look for mediumship at all. It tests for cold reading.

I still don't get this point. If Sylvia passes, she passes. If she fails, she fails. The test is to discern whether or not she can divine information specific to the true sitter. Doesn't matter how, or how not.

And given that there's no feedback and minimal information on the sitter, any "cold reading" she would do would amount to total guesswork.

It's like you're saying "Hey, that test with Natalia Lulova? That wasn't a test for being able to see through a blindfold, it was a test for not being able to see through a blindfold!" Well, tough, the null hypothesis stands if you can't pass the test.

Edit: the null hypothesis is not "cold reading" but "can't divine information specific to the sitter". Doesn't matter how (see above).

David

 

[Tricky]

I have moved this from the other thread, as it was a derail. As I noted there, there are serious flaws in the protocol of "Sylvia Challenge", flaws that continue to be totally ignored .

The other thread was not derailed. You simply balked at the question "how is your protocol better than Randi's"

• 1. Not Testing the Claimed Ability.
  
  You say that the Challenge is supposed to test the claim (in this case, mediumship). But your Sylvia protocol doesn't look for mediumship at all. It tests for cold reading.

It tests to differentiate cold reading from actual mediumship. How could you possibly test for mediumship without eliminating the possiblility of cold reading? You don't trust Randi to be honest. Why should he design a test that doesn't test Sylvia's honesty?

• 5. Odds are Quoted That Make No Sense. What is Their Source?

I have never seen any source for the "Challenge" odds of 50:1. Who provided JREF with these odds?

more than one person here has quoted you plausible explanations for the odds, to which you have not responded. I suppose you want to hear it out of Randi's mouth. But would you even understand it if he did? Are you an expert at statistics? If so, then explain your objections to the explanations shown so far. Frankly, I don't understand them either, but if anything, they look like much less than 50:1 for Sylvia. Please show us how you think they are greater. Otherwise, your objection is less than moot.

• 6. How Will These All-Important ‘Raters’ Be Selected?

If the participants are supposed to "believe" in Sylvia's powers (a criteria I think is completely unnecessary, btw), then why not have Sylvia involved in selecting them, rather than you?

Or, more to the point, why not have a mutually agreed on third party (e.g. Larry King, as you suggested on CNN) do the selection--and the testing? At a minimum, it would help rule out the possibility of collusion.

Show me where James Randi selects the sitters. You are strongly implying this, yet you have nothing in the protocol that suggests such a thing. My interpretaion of Randi's protocol is that Sylvia approves the list and the sitters are chosen off that list at random.

Furthermore, show me that Larry King would be less likely than Randi to try to influence the results. Larry King would have a lot to gain from a Sylvia victory. Can you explain why you think he is more trustworthy than Randi?

• 7. How Does Your SB Protocol Rule Out Subjectivity in Judging?

.

According to the Challenge rules, there is to be no “judging”, no subjectivity in determining the result. The rating process for the Sylvia Challenge that you is, sadly, rife with subjectivity and, in fact, does not yield “self evident” results at all.

There is no judging by parties not being tested. Obviously, dowser, mentalists, and of course, mediums and sitters must judge whether or not they think their own assesment is accurate, but not anyone else. You have not shown how you would eliminate this subjective judging by test participants, yet you rail at Randi because he can't. That is unfair on your part.

• 8. Finally, a Side Note: Is this the Preliminary test for Sylvia? Or is it, as it sounded on LKL, the Final, the test she takes “for the million”?

And, just to note again, I am no fan of Sylvia. This is not about her. It is, pure and simply, about whether or not the protocol you’ve proposed for testing her is (1) a good one, and (2) whether it is consistent with the Challenge principles. As far as I can tell, it is neither.

Thank you in advance for any clarity you can shed on these issues. I am more than willing to be shown I am mistaken on each of these key critical points.

The question as to whether this is preliminary or not I cannot answer. In any case, it will not matter, since Sylvia will not take this or any other test administered by Randi.

Your objections to the protocol are all moot or not addressed by a better protocol. In some cases, such as having Larry King pick sitters, it is worse. You have not made a single valid objection to the Randi protocol. The only valid one that exists (which I suggested) is that there are not enough repetitions of the test. That is one objection you have not made. I must ask, why not?

 

[Clancie]

Posted by Tricky
The other thread was not derailed. You simply balked at the question "how is your protocol better than Randi's"

No, I meant the original "Why are you at JREF?" thread. And, in yours I don't think you transferred over all of the above criticisms. So...that's why I started the new thread.

Re: how is "my" protocol better? First, the criticisms of the Challenge are mine. But this protocol I've adopted from Lurker's post at TVTalkshows. He modified Randi's challenge to be 10 people read...readings all transcribed....each identifies the one for himself. 4 correct out of 10 is 100:1 odds.

How is it "better than Randi's?" Easy.

First, unlike Randi's test, there's no judging needed (which is Randi's own stipulation for the Challenge--however, one he himself violates with his protocol for Sylvia).

Second, by having Sylvia read everyone...and have everyone choose "their" reading....you are simultaneously looking for mediumship -and- cold reading. Vastly preferable to Randi's method, in which he's really only looking for cold reading. Also, in Randi's test the results can be knowingly controlled by the participants themselves, giving a high or low rating to the reading in order to influence whether Sylvia wins or loses.

Obviously, this way is better since the participants can't "throw" the test one way or the other.

Third, if someone had genuine paranormal abilities, Randi's scoring might easily not detect it (since Sylvia could score the highest possible result and still not win.). In "my" (Lurker's) version, either the minimum number of people required (or more) correctly choose their readings or they don't. If they do...she wins. If they don't...she loses.

So, in this version, unlike Randi's, she either demonstrates her abilities...or not. No one can intentionally "throw" the results one way or the other.

Definitely, much improved. (I'll get to the other points next time ).

 

[davidhorman]

First, unlike Randi's test, there's no judging needed

Where is the judging in Randi's test? The sitters have to give their reading a score, sure, and that's subjective. It's no less subjective than having to pick out the reading that seems to match you best.

If the true sitter scores their reading higher than everyone else, Sylvia wins. That's objective and self-evident.

Your second point is good, but I'd replace "cold reading" with "guesswork". And it's not testing for mediumship either - the test doesn't make any distinction as to how the information was gained, only that it was gained anomalously and is pertinent to the true sitter.

Third, if someone had genuine paranormal abilities, Randi's scoring might easily not detect it

Can you remind us again what you mean by "highest possible result"? Is that where every sitter rates the reading a 10? I think most people would understand that 10 out of 10 means "completely accurate" - it's not just a high threshold. I don't find it any different from every sitter rating it a 7, in which case Sylvia's just listed enough generalities for it to apply equally to all sitters (lose).

(Randi) As for #2, we have a much more elegant, simpler, definitive protocol designed. We're in control, we know what we're doing.

If you do find yourself emailing Randi again, I think you should ask him to describe his protocol to you - no doubt he's had to time to think about it. If it's still "one true reading over the phone, 9 transcripts, and a score from each" that's decidedly dodgy as everyone seemed to agree in the other thread (although I think the point should be that the true sitter would know they are the true sitter, not that it's not testing for mediumship, or that it's testing for guesswork, which it isn't). Feel free to mention my name, and I'm sure others would like to hear it from Randi. I don't think he did himself any favours with that mysterious sentence above.

David

 

[Tricky]

How is it "better than Randi's?" Easy.

First, unlike Randi's test, there's no judging needed (which is Randi's own stipulation for the Challenge--however, one he himself violates with his protocol for Sylvia).

As I and davidhorman have pointed out to you many times, this is simply untrue. The sitters must "judge" which reading is theirs. What you seem to object to is that the sitters' judging involves numbers, rather than just picking one, but your proposal in no way eliminates judging by the participants.

First objection is incorrect.

Second, by having Sylvia read everyone...and have everyone choose "their" reading....you are simultaneously looking for mediumship -and- cold reading. Vastly preferable to Randi's method, in which he's really only looking for cold reading. Also, in Randi's test the results can be knowingly controlled by the participants themselves, giving a high or low rating to the reading in order to influence whether Sylvia wins or loses.
Obviously, this way is better since the participants can't "throw" the test one way or the other.

You are simply looking for whether or not Sylvia gets it right by whatever means. The test of "actual mediumship" is how correct you can be in your readings.

Now as to the "controlling" thing: If Sylvia's readings were really correct, the results could be knowingly controlled by a sitter simply choosing the reading most unlike their own. You still have not shown a way to be 100% certain that sitters play fair.

But as to the issue of a sabotaging sitter deliberately screwing with the rankings, suppose that a sitter DID deliberately rate a ranking too high. What if that sitter actually WAS the read sitter? Then he would be helping Sylvia. Your objection assumes that the sitters know whether or not they were read, and that is certainly not the case.

Second objection is incorrect

Third, if someone had genuine paranormal abilities, Randi's scoring might easily not detect it (since Sylvia could score the highest possible result and still not win.). In "my" (Lurker's) version, either the minimum number of people required (or more) correctly choose their readings or they don't. If they do...she wins. If they don't...she loses.

So, in this version, unlike Randi's, she either demonstrates her abilities...or not. No one can intentionally "throw" the results one way or the other.

If Sylvia had genuine paranormal abilities, she would never miss and the results would be so specific that no one would possibly mistake a reading for their own. Even your own protocol only requires that Sylvia be right four out of ten times. In suggesting such a protocol, you are acknowledging that her "power" only works occasionally, or more precisely, that there is only a statistical chance that she will be correct.

So if the evaluation is statistical anyway, you are merely arguing about the way the statistics are performed and you have not shown that the Randi way of doing statistics is worse than your own.

And I'm not sure how you came up with the "100:1 odds" either. It doesn't look right to me.
Third objection is incorrect.

I am not saying your protocol is invalid, only that your criticisms of Randi's are incorrect. Randi's protocol was probably proposed because Sylvia would balk at having to make too many readings. Especially if she wasn't getting paid for them!