Misleading Sceptic Report article

[TLN]

To get back to the contents of the article, one that purports to show the current state in 2000, to omit these results is something of a mystery.

Sure. Let me sum this up for everyone involved. You and others claim that there are serious parapsychological studies that show a reproducible effect. May I see one please, complete with testing protocol and raw data? Please don't throw out random names and places and tell me to "go look" or to "Google" such-and-such.

You say there are good studies. Show one.

 

[CFLarsen]

Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Misleading Sceptic Report article

Isn't it odd that a paper that hasn't been updated since Jan. 2000 would be put in the Jan. 2004 issue?

The paper omits the RNG experiment results, things which Sagan even said deserve study:

"At the time of writing, there are three claims in the ESP field which, in my opinion, deserve serious study: (1) that by thought alone humans can (barely) affect random number generators in computers;..."
(Sagan, The Demon-Haunted World, 1995, p. 302)

Yes, T'ai Chi, we've been through this before. Sorry, you don't have a case.

And you selecting only the part of Sagan's quote....you really thought you would get away with that?

BTW: How come you can take a neutral stance on paranormal matters (or at least claim that you do), when you demanded that people expressed either a pro-psi or con-psi stance, before they were allowed to sign up for your (botched) experiment?

 

[The Don]

If it is at all possible, studies cited should also have been successfully repeated by an independent, unbiased, set of researchers

 

[Lothian]

'All show no effect' ...... Yes if you dismiss every positive controlled trial result as fraud, error, lucky fluctuations, etc. .....

So far but I am sure you can point me in the direction of one that wasn't.

 

[TLN]

So far but I am sure you can point me in the direction of one that wasn't.

Yeah, asking to actually see one of these mythical studies has really shut this tread up quick, huh?

 

[Dancing David]

Re: Re: Re: Re: Re: Re: Re: Re: Re: Misleading Sceptic Report article

Just a few lines would do. Something along the lines of - "although this test fails miserable on methodological grounds, the majority of scientific research into telepathy employs adequate controls in its experimental design"

Perhaps a link to the excellent article further down in the Sceptic Report - "The Current State of Parapsychology Research" by Scott Teresi

What hoot! HOOT!

 

[Dancing David]

Re: Re: Re: Re: Re: Re: Re: Misleading Sceptic Report article

"


Pre-autoganzfeld experiments were reviewed by Hyman and Honorton. Hyman claimed he found a correlation between study quality and effect sizes while Honortan did not (1) This was probably down to their different criteria applied to the study quality (interestingly the independent reviewed agreed with Honorton). Because of this discrepency, Hyman and Honorton came up with an agreed set of improved protocols for the autoganzfeld studies. According to your claim this should have resulted in an immediate disappearance of the effect. This did not happen. The results were consistent with the pre-ganzfeld studies (2) More recently, a meta-analysis revealed that the overall effect disappeared with the addition of all most recent experiments at the time of writing (well almost all, Wiseman and Milton left out 10 experiments that, if they had been included, would have resulted in a significantly replicated effect size) (3). However this late disappearance was found to correlate with the degree of standardness of the ganzfeld protocol rather than the degree of study quality, since all the recent studies follow the same autoganzfeld procedures as recommended by Hyman and Honortan (4)

1. Honorton, C. (1985). Meta-analysis of psi ganzfeld re search: A response to Hyman. Journal of Para psychology, 49, 51–91.

2. Bem, D. J., & Honorton, C. (1994). Does psi exist? Replicable evidence for an anomalous process of information transfer. Psy chological Bulletin, 115, 4–18.

3. Milton, J. & Wiseman, R. (2001). Does Psi Exist? Reply to Storm and Ertel (2001). Psychological Bulletin, 127, 434-438.

4. Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218.

Blah, blah blah, what a waste of space. The ganzfeld effct does not exist and if it does exist it is the product of so many confounding factors that psi is the least likely candidate.

You are really up a tree if you want to suggest that the ganzfeld experiments have any meaning. Unless you want to act like Amhearst and just assert that they have meaning.

Ganfeld effect discussed endlessly

If you read through this thread, you will find that there is a reasonable discussion of what the confounding effects are. If you don't have controls then you don't have an effect.

 

[T'ai Chi]

Sure. Let me sum this up for everyone involved. You and others claim that there are serious parapsychological studies that show a reproducible effect.

I didn't realize that "Sagan, while personally thinking that they aren't real, realizes that they 'deserve serious study'." implied all of that.

Edited to add an example of RNG experiment work that could have been discussed, if the author chose to do so, in his "Current State" paper from 2000: http://www.boundaryinstitute.org/articles/rngma.pdf

 

[CFLarsen]

I didn't realize that "Sagan, while personally thinking that they aren't real, realizes that they 'deserve serious study'." implied all of that.

But you were aware of the remainder of the quote?

How come you can take a neutral stance on paranormal matters (or at least claim that you do), when you demanded that people expressed either a pro-psi or con-psi stance, before they were allowed to sign up for your (botched) experiment?

 

[davidsmith73]

Look at the history of paranormal research: The experiments generally are full of design flaws. Radin, Schwartz, Rhine, Soal, Targ, Puthoff...need I go on?

Yes, you need go on!

Radin - which experiments and which flaws?

Schwartz - ok you got one there!

Rhine - not really up to date, but go on, which experiments and which flaws?

Soal - experiments and flaws?

Targ and Puthoff - probably got another one there

So is that it? Out of all parapsychological experiments you have a handfull of examples of poor methodology (which you haven't yet given specifics about). Hardly an "abundance". I'm not denying that incompetent research exists but if you are seriously interested in the truth then you cannot claim that there is an abundance of it.

Lets take the ganzfeld experiments again. From Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218.

...and a hit rate of 32% (p = .0008) for 10 computer controlled (“autoganzfeld”) studies conducted between 1983 and 1989 that had been specifically designed to eliminate methodological flaws identified in some of the earlier studies....
...More recently, Milton and Wiseman (1999) published a follow-up
meta-analysis of 30 additional ganzfeld studies that had been conducted from 1987 through 1997....
...In addition to the 30 studies analyzed by Milton and Wiseman (1999),an additional 10 studies were located by examining the six major publication outlets for parapsychological research."

So we have 50 ganzfeld studies that have stringent methodological design in accordance with the guidelines set out bu Hyman and Honorton. Is this already many more than you can find that are incompetent? And that's just the ganzfeld studies.

Davidsmith73: Prove that there is a correlation between improved study quality and disappearing effect sizes.

Claus:Excuse me? You claim to be knowledgable about paranormal research, and you are not aware of this?

Answering a question with a question is not proving your claim. You claimed that there is a correlation between improved study quality and disappearing effect sizes. Now put your money where your mouth is and prove it.

I see you misunderstand me, and I can understand why: When I said "disappearing", I should have said "diminishing".

Ok, prove your claim that when research is done with sound methodology this has resulted in rapidly diminishing effect sizes.

Who, among skeptics, reviewed this?

R. Rosenthal

Let me get this straight, OK? Are you saying that we can, today, perform a paranormal experiment that produces the same effect size, and that is not due to experimental flaws?

No, I'm saying that to my knowledge there is no evidence that there is a correlation between diminishing effect sizes and study quality. Show me the evidence if you think otherwise. It's your claim

Your complaint is therefore dismissed.

(*creak* as Claus reclines into his well used armchair)

 

[CFLarsen]

Yes, you need go on!

Why? You asked for examples, I gave them.

Radin - which experiments and which flaws?

An evening with Dean Radin
Tell me that Radin doesn't select his data.

Book Review: Dean Radin, "The Conscious Universe"
Tell me that Radin doesn't select his data.

Schwartz - ok you got one there!

Oh, yes.

Rhine - not really up to date, but go on, which experiments and which flaws?

Gimmeabreak! There were virtually no controls! People could see through the cards, cards were reflected in glasses, pre-trials were allowed to be recorded as real ones...

Soal - experiments and flaws?

Proven cheater.

Targ and Puthoff - probably got another one there

Oh, yes.

So is that it? Out of all parapsychological experiments you have a handfull of examples of poor methodology (which you haven't yet given specifics about). Hardly an "abundance". I'm not denying that incompetent research exists but if you are seriously interested in the truth then you cannot claim that there is an abundance of it.

A "handful"? Are you crazy? These are the All-Stars of parapsychology research, and their work is heavily flawed.

So we have 50 ganzfeld studies that have stringent methodological design in accordance with the guidelines set out bu Hyman and Honorton. Is this already many more than you can find that are incompetent? And that's just the ganzfeld studies.

Do I understand you correctly? Can I - or anyone else - do an experiment that will produce evidence of a paranormal phenomenon? Just yes or no.

No, I am not interested in your explanations. All I want is a "Yes" or a "No".

Answering a question with a question is not proving your claim. You claimed that there is a correlation between improved study quality and disappearing effect sizes. Now put your money where your mouth is and prove it.

I did.

Ok, prove your claim that when research is done with sound methodology this has resulted in rapidly diminishing effect sizes.

I don't see any headlines. I don't see any extra news programs. I don't see any Nobel prizes. Where is this evidence of a replicable paranormal event?

R. Rosenthal

....and??

No, I'm saying that to my knowledge there is no evidence that there is a correlation between diminishing effect sizes and study quality. Show me the evidence if you think otherwise. It's your claim

No, no, no. You claim that there is no such diminishing effect. Show it, then.

The onus is on you.

(*creak* as Claus reclines into his well used armchair)

It would be funny, if true. Since it isn't....

 

[Ersby]

So we have 50 ganzfeld studies that have stringent methodological design in accordance with the guidelines set out bu Hyman and Honorton. Is this already many more than you can find that are incompetent? And that's just the ganzfeld studies.

Strictly speaking, one of those experiments isn’t a ganzfeld study at all (the one using drum beats instead of white noise). So that should be 49.

We do not need to find incompetent studies (although that would be nice, and some would say that the 10 PRL trials offered avenues for sensory cues (sound leakage, video degredation) and so could be included in that category), but instead we need to find those experiments whose results cancel out this result. If you want to draw a line in the sand and say that we begin at the PRL experiments and Milton and Wiseman’s meta-analysis, that still only gives an average hit rate of 28% (including, of course, ganzfled studies until 20004).

Next point to add is that Rhine (certainly later in life, maybe earlier, but we can’t say for sure) had a policy at the Rhine Institute or repressing negative results, and this is why he and Honorton fell out (btw, the more I learn about Honorton, the more I like him) and prompted Honorton to establish a policy within the Parapsychological Association of reporting all results, good and bad.

It may also be worth adding that Charles Akers’ review of Hyman and Honorton’s debate came down nearer Hyman’s point of view regarding flawed studies. The fact that people tend to mention Rosenthal, but not Akers bothers me somewhat.

And there is a diminishing effect in the ganzfeld studies. The ealiest ganzfeld experiments are where you’ll find the best results. Dalton’s 1997 result is something of a freak, being the highest since 1979.

I still haven’t had a reply from Westerlund about his claims regarding Dalton’s work. Perhaps my hotmail address meant I got sent straight to his junk mail folder. I’ll try again in a week or so.

 

[davidsmith73]

Why? You asked for examples, I gave them.

I asked for specific peer reviewed experimental papers, not just names.

An evening with Dean Radin
Tell me that Radin doesn't select his data.

Where can I see the data that he has selected and the data he left out? And to be fair, your article looks like it was written on the basis of your memory of events.

Book Review: Dean Radin, "The Conscious Universe"
Tell me that Radin doesn't select his data.

How many experiments has Radin contributed to the body of parapsychology that use selected data?

Gimmeabreak! There were virtually no controls! People could see through the cards, cards were reflected in glasses, pre-trials were allowed to be recorded as real ones...

I'll have to take your word for that. Also, experiments done in the 1930's are not representative of the methodology of modern parapsychology. Criticisms against these experiments were cruical in designing the well controlled experiments post-Rhine.

Proven cheater.

Which peer reviewed experiments has Soal published?

A "handful"? Are you crazy? These are the All-Stars of parapsychology research, and their work is heavily flawed.

Science does not function on status. You may think that these experiments constitute a relative "abundance" of research in parapsychology, but the truth is that many more experimenters exists that perform well conducted, well controlled experiments. You can examine the experiments for yourself if you were to divert your attention from the soft targets

Do I understand you correctly? Can I - or anyone else - do an experiment that will produce evidence of a paranormal phenomenon? Just yes or no.

My point is about methodology. You are trying to change the focus of this debate. Lets stay on topic.

No, I am not interested in your explanations. All I want is a "Yes" or a "No".

Claus, you have a habit of trying to change the subject. Methodology is my concern, not results.

I did.

You have not proven your claim that there is a correlation between disapp.. sorry, diminishing effect sizes and study quality. Try again.

I don't see any headlines. I don't see any extra news programs. I don't see any Nobel prizes. Where is this evidence of a replicable paranormal event?

Prove your claim that there is a correlation between disapp.. sorry, diminishing effect sizes and study quality.

....and??

You're the one who wanted to know who the indepedent reviewer of the Honorton and Hyman meta-analysis was!

No, no, no. You claim that there is no such diminishing effect. Show it, then.

The onus is on you.

Why is the onus on me when it was you who first claimed that:
"it would be most incorrect to give readers not familiar with parapsychological research the impression that the "majority" (as you call it) of research is being done with sound methodology without also stating that this has resulted in rapidly disappearing effect sizes"

So come on Claus, prove your assertion.

 

[davidsmith73]

We do not need to find incompetent studies (although that would be nice, and some would say that the 10 PRL trials offered avenues for sensory cues (sound leakage, video degredation) and so could be included in that category), but instead we need to find those experiments whose results cancel out this result. If you want to draw a line in the sand and say that we begin at the PRL experiments and Milton and Wiseman’s meta-analysis, that still only gives an average hit rate of 28% (including, of course, ganzfled studies until 20004).

Is there a published meta-analysis that gives a 28% hit rate or is that your own analysis?

Also, my point is that there is no "abundance" of incompetent experiments as Claus claims. The article that appears in the Sceptic Report does not mention anything of the standard of modern telepathy experiments and could mislead uninformed readers. This is the standard of the Edinburgh ganzfeld protocol today - http://moebius.psy.ed.ac.uk/Ganzfeld_H.html#tg

Next point to add is that Rhine (certainly later in life, maybe earlier, but we can’t say for sure) had a policy at the Rhine Institute or repressing negative results, and this is why he and Honorton fell out (btw, the more I learn about Honorton, the more I like him) and prompted Honorton to establish a policy within the Parapsychological Association of reporting all results, good and bad.

Ok, and I think we have established that the ganzfeld experiments are a different kettle of fish from the telepathy "test" described in the Sceptic Report article.

It may also be worth adding that Charles Akers’ review of Hyman and Honorton’s debate came down nearer Hyman’s point of view regarding flawed studies. The fact that people tend to mention Rosenthal, but not Akers bothers me somewhat.

Do you have a reference for the Akers' review?

And there is a diminishing effect in the ganzfeld studies. The ealiest ganzfeld experiments are where you’ll find the best results. Dalton’s 1997 result is something of a freak, being the highest since 1979.

Is the difference between the pre-PRL and PRL effect sizes significant? The original pre-PRL studies had an effect size of 0.62, with a 95% confidence interval from 0.55 to .069. The PRL studies effect size was 0.59, with a 95% confidence interval from 0.53 to 0.64. Since the confidence intervals overlap we can't conclude that the difference in effect size is significant.

The Milton and Wiseman paper gives an effect size of 0.013 (different type of effect size - the PRL equivalent is 0.162) When an additional 10 studies were added to this analysis the effect size was 0.051 - larger but still significantly smaller than the PRL studies . However, the Bem, Palmer and Broughton paper shows that the effect size is correlated with adherence to the degree of "standardness". The further the experiments deviate from standard protocol the less the effect size becomes.

So we only have one significant decrease in effect size which is comparing PRL studies with those following, and this is shown to be more likely due to deviation from standard protocol, which is expected given the amount of process oriented experiments included in the later studies.

If the assertion that decreasing effect size is due to tightening of controls is true, you would expect a significant decrease between the pre-PRL studies and the PRL studies as well as between the PRL and post-PRL studies. This was not the case.

 

[Ersby]

Is there a published meta-analysis that gives a 28% hit rate or is that your own analysis?

My own, though I wouldn't call it an analysis, since it's just a collection of results.

Also, my point is that there is no "abundance" of incompetent experiments as Claus claims. The article that appears in the Sceptic Report does not mention anything of the standard of modern telepathy experiments and could mislead uninformed readers. This is the standard of the Edinburgh ganzfeld protocol today - http://moebius.psy.ed.ac.uk/Ganzfeld_H.html#tg

Ok, and I think we have established that the ganzfeld experiments are a different kettle of fish from the telepathy "test" described in the Sceptic Report article.

I've not read the report. I don't think there's a great deal of incompetent experiment in the ganzfeld database, simply because not a great deal is needed to bring things back to chance (imo).

Do you have a reference for the Akers' review?

He writes about in the the Skeptics Handbook to Parapsychology. Prometheus Books, edited by Paul Kurtz.

Is the difference between the pre-PRL and PRL effect sizes significant? The original pre-PRL studies had an effect size of 0.62, with a 95% confidence interval from 0.55 to .069. The PRL studies effect size was 0.59, with a 95% confidence interval from 0.53 to 0.64. Since the confidence intervals overlap we can't conclude that the difference in effect size is significant.

The Milton and Wiseman paper gives an effect size of 0.013 (different type of effect size - the PRL equivalent is 0.162) When an additional 10 studies were added to this analysis the effect size was 0.051 - larger but still significantly smaller than the PRL studies . However, the Bem, Palmer and Broughton paper shows that the effect size is correlated with adherence to the degree of "standardness". The further the experiments deviate from standard protocol the less the effect size becomes.

So we only have one significant decrease in effect size which is comparing PRL studies with those following, and this is shown to be more likely due to deviation from standard protocol, which is expected given the amount of process oriented experiments included in the later studies.

If the assertion that decreasing effect size is due to tightening of controls is true, you would expect a significant decrease between the pre-PRL studies and the PRL studies as well as between the PRL and post-PRL studies. This was not the case. [/B]

This all depends on what effect size you're using, doesn't it?

Sorry for the brief responses. Have to run!

 

[CFLarsen]

davidsmith73,

I can see that no matter what I present, you will ignore it. I can understand that.

You also refuse to show just one experiment that will produce evidence of a paranormal phenomenon. I can understand that, too.

It should be easy, though, since you claim that these experiments show quite a different picture than previous research. But you won't. Or can't. You even refuse to write an article about it.

Same old manure. Same old smokescreen. Feh...

 

[davidsmith73]

davidsmith73,

I can see that no matter what I present, you will ignore it. I can understand that.

Like I said, I do not deny that there is incompetent research in parapsychology. I do deny that there is an "abundance" of it. You have not shown that there is an "abundance" as indicated by your examples. I have suggested that the ganzfeld experiments are an example of the fact that parapsychology is not littered with incompetant research. The greater numbers of experiments with sound methodology in the ganzfeld field alone compared with the examples you gave indicate you are wrong.

You also refuse to show just one experiment that will produce evidence of a paranormal phenomenon. I can understand that, too.

Like I said, this particular debate is not about the evidence. Its about the methodology.

It should be easy, though, since you claim that these experiments show quite a different picture than previous research. But you won't. Or can't. You even refuse to write an article about it.

Are the ganzfeld experiments comparable to the card experiments of Rhine in terms of methodological flaws? I think not.
I've given you references that show the overall effect size of the PRL ganzfeld studies is not significantly different from the pre-PRL studies, previously shown to have methodological flaws. Also, when the more recent studies are analysed for degree of standardness, from Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218:

"This same outcome can be observed by defining as standard the 29 replications whose ratings fell above the midpoint of the scale (4) and defining as nonstandard the 9 replications that fell below the mid point (2 replications fell at the mid point): The standard replications obtain an overall hit rate of 31.2%, ES = .096, Stouffer Z = 3.49, p = .0002, one-tailed. In contrast, the nonstandard replications obtain an over all hit rate of only 24.0%, ES = –.10, Stouffer Z = –1.30, ns. The difference between the
standard and non standard replications is itself significant, U = 190.5, p =.020, one-tailed. Most importantly, the mean effect size of the standard replications falls within the 95% confidence inter vals of both the 39 preautoganzfeld studies and the 10 autoganzfeld studies summarized by Bem and Honorton (1994). In other words, ganzfeld studies that adhere to the standard ganzfeld protocol continue to replicate with effect sizes comparable with those obtained in previous studies."

Same old manure. Same old smokescreen. Feh...

Same old armchair dismissal

 

[Interesting Ian]

Originally posted by davidsmith73
Rhine - not really up to date, but go on, which experiments and which flaws?


Larsen
Gimmeabreak! There were virtually no controls! People could see through the cards, cards were reflected in glasses, pre-trials were allowed to be recorded as real ones...

WOW! That's interesting Claus. Could see through the cards? Used poor quality cards did they? Can you substantiate any of this? Or are you simply repeating what skeptics said at the time? I suspect the latter; but it is my understanding that it was just wild unsubstantiated speculation by skeptics. But prove me wrong.

 

[Interesting Ian]

Originally posted by davidsmith73
No, I'm saying that to my knowledge there is no evidence that there is a correlation between diminishing effect sizes and study quality. Show me the evidence if you think otherwise. It's your claim



No, no, no. You claim that there is no such diminishing effect. Show it, then.

The onus is on you.

Er . .excuse me . .he did not. He said he wasn't aware of any evidence showing such a diminishing effect. But I'm sure he would be delighted to become acquainted with this evidence, as would I. Now you have claimed there is indeed evidence showing a diminishing effect. Do you have this evidence, or don't you have this evidence?

 

[CFLarsen]

Like I said, I do not deny that there is incompetent research in parapsychology. I do deny that there is an "abundance" of it. You have not shown that there is an "abundance" as indicated by your examples. I have suggested that the ganzfeld experiments are an example of the fact that parapsychology is not littered with incompetant research. The greater numbers of experiments with sound methodology in the ganzfeld field alone compared with the examples you gave indicate you are wrong.

Show me a replicable Ganzfeld experiment that shows evidence of a paranormal phenomenon.

Like I said, this particular debate is not about the evidence. Its about the methodology.

It is very much about the evidence.

Are the ganzfeld experiments comparable to the card experiments of Rhine in terms of methodological flaws? I think not.

They are a brilliant example of how the effect diminishes when better controls are put in place.

I've given you references that show the overall effect size of the PRL ganzfeld studies is not significantly different from the pre-PRL studies, previously shown to have methodological flaws. Also, when the more recent studies are analysed for degree of standardness, from Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218:

Yes, a lot of words. Show me a replicable Ganzfeld experiment that shows evidence of a paranormal phenomenon.

Same old armchair dismissal

Show me a replicable Ganzfeld experiment that shows evidence of a paranormal phenomenon.

Everything else is merely a smokescreen.