Psi in the Ganzfeld

[davidsmith73]

Louie Savva, in the thread that started this thing, said it was common for informal experiments to be carried out, with only positive results getting any publicity. How far this applies to a cumbersome protocol like the ganzfeld is not sure, but I certainly think that this should be taken into account.

As tempting as it might be for someone to accept Louie's assertion, I don't think it is fair debate. If you recall, he wrote that he remebers having had conversations with fellow researchers about this at conferences. Is that a reliable indicator of the truth? I don't think so. Strong views, and he cleary has a strong view about the non-existence of psi (just read his blog), can influence the way we remember things. Not that I am saying he has definately misremembered, just that its not scientific to equate his word on this to evidence of a file draw. An unlikely scenario but, would you accept a prominent parapsychologists word that they heard from a converstaion that informal experiments are often carried out that are successful but are not published?

 

[davidsmith73]

Once again, that meta-analysis I did was to prove a point - that anyone with enough time or patience can manipulate a database to their own ends. You can't keep thinking that they somehow prove something. I, using quite sensible criteria knocked the odds down to 1 in 297. Chopping of those experiments of under 30 trials was stretching things, I grant you, but not bad for an hours work.

I think that the best we may get out of the ganzfeld meta-analysis is to use it to look for evidence that effects decline with better methodology or similar variables of interest, instead of looking at a "grand final" p-value.

Ersby, I was wondering whether you had time to do that log scale adjustment to the funnel plot? Not that I'm qualified to interpret it properly, but I would be interested to see if there was more of an indication of a funnel shape tapering towards an effect size above zero.

 

[Rodney]

I don't understand. What indicates to you that he's not objective? That he no longer believes there's anything to parapsychology? Isn't he in a position to know, considering that he had been working in the field?

As I noted previously, not long ago Louie said in response to a question that he anticipated being a parapsychologist in 10 years. Suddenly, he is no longer a parapsychologist and has nothing good to say about parapsychology. This is completely different than a parapsychologist's position evolving over many years.

What does "objective" mean, anyway? Undecided? Or right?

I think it excludes someone who leaves parapsychology in a huff.

Huh? We're talking about the possibility of unsuccessful experiments going unpublished. What would you consider sufficient evidence? A published report saying, "We ran this experiment, here are the details, but it was unsuccessful, so we decided not to publish ... oops, too late." ?

At a minimum, we need at least one name. Who, exactly, has done a ganzfeld experiment and not published the result?

 

[Ersby]

Ersby, I was wondering whether you had time to do that log scale adjustment to the funnel plot? Not that I'm qualified to interpret it properly, but I would be interested to see if there was more of an indication of a funnel shape tapering towards an effect size above zero.

I'll take a look at this. I thought it'd be really difficult so I put it off, then I just noticed there's an option to do it automatically! I'll have to bring in my data tomorrow and do it then.

An unlikely scenario but, would you accept a prominent parapsychologists word that they heard from a converstaion that informal experiments are often carried out that are successful but are not published?

Well, not really. I don't think that parapsychology is so confident about its own standing that it can afford to let successful work go unnoted.

 

[Ersby]

Suddenly, he is no longer a parapsychologist and has nothing good to say about parapsychology. This is completely different than a parapsychologist's position evolving over many years.

"Suddenly" according to who? Where does Savva say this?

I think it excludes someone who leaves parapsychology in a huff.

Is that your objective opinion?

At a minimum, we need at least one name. Who, exactly, has done a ganzfeld experiment and not published the result?

Parker.

 

[Cuddles]

At a minimum, we need at least one name. Who, exactly, has done a ganzfeld experiment and not published the result?

Do you really believe that all research is published? Although the situation is getting better and there are now a few journals dedicated to negative results, the vast majority of negative studies are never published. This is true for every field of science, but especially so for those that rely on results for funding. If a biologist shows that a cell doesn't work the way they thought, they can simply move on to a different part of the cell, if a parapsychologist shows that psi doesn't exist they are out a job. So, in addition to the natural bias against negative results, there is an additional bias in fields like parapsychology where negative results don't just remove one area for study, they reomove the whole field.

In addition, yuor question is obviously a nonsensical one. So you get a name, as Ersby has provided. What now? You can't look up the study because it was never published, so there is no way to prove anything either way.

 

[Ersby]

I decided it was a bit glib of me to just give a name, and leave it at that. So here's a little more detail.

I think there is a tendancy (as, I guess, in all sciences) to write up successful results in more detail than unsuccessful results. In the half dozen or so papers I have where the results are not given in numerical form, all but one are positive and the one that may be positive is written in such vague terms that I have a sneaking suspicion it wasn't too successful either.

It is difficult to judge the impact of unpublished results because they are, by their nature, unknown. Parker and Wiklund completed five unsuccessful experiments early in the history of the ganzfeld, and Sargent mentions in passing work his students completed which got significantly negative scores. Often I find myself with much less information on an experiment with poor results (sometimes just a name, number of trials and a hit rate) than with experiments with good results.

Blackmore did an investigation into unpublished ganzfeld papers and concluded the hit rate was not so different from those being reported at the time. But she doesn't list which researchers replied to her questionnaire so we can't be sure that work she covered wasn't subsquently published. (I emailed her on this, but she replied she had long since thrown out her notes on this paper and couldn't remember.)

As for unpublished successful experiments, I have a couple of theses which I found on the internet. Would these people have made the effort to make their work known if they hadn't been successful? I guess that's a judgement call we'd make according to our own existing world views.

 

[davidsmith73]

Do you really believe that all research is published?

Of course not, but the question is, how much is not published?

Although the situation is getting better and there are now a few journals dedicated to negative results, the vast majority of negative studies are never published. This is true for every field of science, but especially so for those that rely on results for funding. If a biologist shows that a cell doesn't work the way they thought, they can simply move on to a different part of the cell, if a parapsychologist shows that psi doesn't exist they are out a job. So, in addition to the natural bias against negative results, there is an additional bias in fields like parapsychology where negative results don't just remove one area for study, they reomove the whole field.

That could be true. There's certainly more motivation for the results of parapsychology to be positive. But there are different areas of parapsychology, and if one particular experimental paradigm were shown to be on completely the wrong track, there are others that an experimenter could follow. Its naive to think that ideal experimental conditions will be found in the short term considering how little we know about the phenomena and how complex human psychology can be. Indeed, I think parapsychology has made progress in improving experimental conditions for success, for example, experiments that use unconscious physiological measures. I hope the people who fund this kind of research realise that.

 

[Dancing David]

Yes, but if you aggregate several small experiments, you obtain -- all things being equal -- the equivalent of one large experiment. You seem to justify excluding small experiments by speculating that only the ones that produced above the expected number of hits were published, but I fail to see any evidence supporting your speculation. Do you have any?

Just so you know that is standard practice in meta-analysis. Which is a very strange statistical creature and normally used with a lot of caveats and warnings about the protocols and methodology. In my readings meta-analysis is usualy taken as a way of determing things to be investigated further rather than being definitive.

 

[Rodney]

"Suddenly" according to who? Where does Savva say this?

On the initial thread, I posted the following link, which was an interview done of Louie Savva when he was 26 (he's now 28): http://www.darkenchantment.co.uk/ouisavva.html

Among other things, Savva says: "If you’re asking me if it’s possible to be attacked by a spirit, I guess my first reaction would be, 'it’s possible'."

And: "When the new series of MH starts soon, you can see me getting absolutely terrified at the locations I visit."

And: "Hmm 10 years from now, I’d be 36. I’ll hopefully still be involved in parapsychology and be an active researcher. I know it’s probably a boring answer but just continuing what I’ve started."

So, in the space of at most two years, Savva went from being a parapsychologist who thought it was possible to be attacked by a spirit, who was terrified by certain (presumably "haunted") locations, and who intended to stay in the field at least another 10 years to a complete non-believer. That seems like a sudden change to me.

By the way, when are we going to discuss some of the successful ganzfeld experiments in greater depth?

 

[Ersby]

Hmm, two years is hardly sudden.

As for discussing successful experiments, I gave details about one in post #42, near the top of the second page.

 

[Rodney]

Hmm, two years is hardly sudden.

Where is the evolution of Louie's thought? Can you cite any articles that he wrote indicating a gradual change in his outlook?

As for discussing successful experiments, I gave details about one in post #42, near the top of the second page.

That example is from the early days of ganzfeld experiments. Can you cite anything from the 1980s on?

By the way, on p. 121 of Entangled Minds Radin shows a graph. The caption reads: "Funnel plot for the ganzfeld studies. The symmetric shape shows there's no file drawer problem."*

* "The overall weighted effect size is e = 0.16+- 0.02."

 

[Ersby]

Where is the evolution of Louie's thought? Can you cite any articles that he wrote indicating a gradual change in his outlook?

Reading his "Why I left parapsychology" post on his blog he details his years of research from 2001-2004 in which he found no evidence for psi, followed by his disillusionment with the papers submitted for the 2005 PA Convention. It seems to me he gave it a fair crack of the whip.

That example is from the early days of ganzfeld experiments. Can you cite anything from the 1980s on?

You didn't specify a time when you asked for an example. This experiment is included in Radin's work and is a good place to start.

By the way, on p. 121 of Entangled Minds Radin shows a graph. The caption reads: "Funnel plot for the ganzfeld studies. The symmetric shape shows there's no file drawer problem."

What was Radin's inclusion criteria for compiling that graph?

 

[davidsmith73]

"Suddenly" according to who? Where does Savva say this?

There's an entry in his blog where he posts an email exchange with Susan Blackmore. I'll quote the first couple of paragraphs here (bold mine):

"Dear Susan,

I have been involved in the parapsychology community for a number of years.

However about March last year it suddenly dawned on me that not only were my peers conducting poor research, but that parapsychology itself is almost entirely delusion. My PhD had been inspired by Darwin and Dawkins alike (though I'm sure Dawkins' would not be pleased to know that) and I have always used evolutionary theory for the basis for all of my theorising about almost any subject.

So at a parapsychology conference I realised that there is no god. No afterlife. And that there is no point to anything. So, I quit parapsychology completely."

original blog - http://everythingispointless.blogspot.com/2006/11/conversation-with-susan-blackmore.html

To me, this indicates a sudden change of opinion.

 

[Ersby]

I stand corrected. So, March 2005 (when he was working as chair of the PA Convention?) is when he decided it's all bunk.

By the way, I did the log scale adjustment to my database. Whether this is what you wanted, I don't know, but here it is.

x=number of trials
y=effect size r

 

[davidsmith73]

I stand corrected. So, March 2005 (when he was working as chair of the PA Convention?) is when he decided it's all bunk.

By the way, I did the log scale adjustment to my database. Whether this is what you wanted, I don't know, but here it is.

x=number of trials
y=effect size r

Thanks Ersby. Yes, that's what I wanted, just to compare it with the graph you gave in your introduction section. Bit disappointing really. I thought the log scale might spread the data out so that a funnel shape might be easier to see but I can't see a funnel shape there at all. The effect size is not being influenced much by the number of trials, which to me indicates that the effect is not reliable. But there does seem to be slightly more positive effect sizes in the 100-1000 trial range. Then again, I don't have enough knowledge to comment on this with much certainty. But lets keep in mind that this is the entire database so we're looking at quite a range of different experimental designs. I haven't seen the funnel plot that Rodney quotes from "Entangled Minds" but I'd be interested to see it.

 

[psp02ls]

However about March last year it suddenly dawned on me that not only were my peers conducting poor research, but that parapsychology itself is almost entirely delusion.

So at a parapsychology conference I realised that there is no god. No afterlife. And that there is no point to anything. So, I quit parapsychology completely."

My ears were burning again. Yes my conversion was sudden. I would describe it as an epiphany. But so, what? I had always been sceptical of most claims, although I was very much open to the existence of psi. In fact, I used to often declare that 'all psi is precognition', because that was what I thought the evidence suggested.

I actually woke up the morning of the 4th June (the day of the conference) very excited about meeting all of my old friends and colleagues. I had been very involved in the initial stages of organising the conference and hadn't seen some of the people for quite a while.

And then, talk after talk had people that I respected, showing no real evidence for the paranormal and not even asking basic questions about alternative interpretations or explanations. I walked around and chatted to people about this, and they just ignored my feelings.

They just didn't understand evolution!

Read this article by Douglas Adams. Sudden realisations about the implications of evolution are common, for people that do understand!

The sceptics amongst you have come out with some good stuff. I suggest though, that discourse with believers is a complete waste of time. Let them live and die, believing in fairy stories.

Unless you really want to be having the same discussion with David and Rodney, in fifty years? Yawwwwwwn.

 

[davidsmith73]

My ears were burning again. Yes my conversion was sudden. I would describe it as an epiphany. But so, what?

I was just offering to help clarify a difference of opinion between Rodney and Ersby by posting that excerpt. For me, it doesn't make any difference to the scientific evidence for psi whether you quit suddenly or not.

I actually woke up the morning of the 4th June (the day of the conference) very excited about meeting all of my old friends and colleagues. I had been very involved in the initial stages of organising the conference and hadn't seen some of the people for quite a while.

And then, talk after talk had people that I respected, showing no real evidence for the paranormal and not even asking basic questions about alternative interpretations or explanations. I walked around and chatted to people about this, and they just ignored my feelings.

They just didn't understand evolution!

In order for me to sceptically assess what you are saying, I'd have to know what experiments you were refering to, why you think they were not evidence for psi, what alternative explanations you were talking about and in what way you think they were ignoring your feelings. I mean, when you say "they ignored my feelings", perhaps they just disagreed with you. Disagreements happen in science all the time as well you know. So unless you give substantial context, I can't agree or disagree with what you are saying, nor should anyone else.

I know that you claim to have no interest in writing about parapsychology, but you evidently do have an interest in stating your opinion about the non-existence of psi and disdain for parapsychology. So the most informative thing to do would be to refer to specific experiments, put forward your alternative explanations and perhaps explain why you think these explanations are more likely. And the reference to evolution is just as lacking in context.

Unless you really want to be having the same discussion with David and Rodney, in fifty years? Yawwwwwwn.

Hey, well at least they are engaging in discussion

 

[Ersby]

Louie's reply to davidsmith73's post can be found here:

http://www.internationalskeptics.com/forums/showthread.php?t=70319&page=6

I was just offering to help clarify a difference of opinion between Rodney and Ersby by posting that excerpt.

A difference of opinion only because I wasn't given the correct data! Rodney said Savva left parapsychology in a huff, but now we learn it was an epiphany.

 

[Rodney]

What was Radin's inclusion criteria for compiling that graph?

Presumably, Radin included the 88 ganzfeld experiments that used a hit or miss protocol conducted from 1974 through 2004 that he referenced on the previous page of Entangled Minds. Can you determine which 88 studies those are?