Chris Carter Exposes Richard Wiseman Misdeeds

[Rodney]

It may interest you to know that the only way it can be presumed that combining all the hits and misses into one big study and testing for significance using the binomial distribution is valid, is if you presume that psi does not have an effect. Otherwise, it is an invalid test.

1) How so?
2) Do you think what Wiseman and Milton did was valid?

 

[alex.tsakiris]

Chris Carter's paper is a bit strange...

Pls all ow me to re-post from the skeptiko forum:
http://forum.mind-energy.net/skeptiko-podcast/1766-jref-post-wisemans-misdeeds.html#post36805

Richard Wiseman writes ...

Quote:
Heads I Win, Tails You Lose. How Parapsychologists Nullify Null Results

To my knowledge, only one paper has revealed an insight into the potential scale of this problem. Watt (2006) summarized all of the psi-related final-year undergraduate projects that have been supervised by staff at Edinburgh University’s Koestler Parapsychology Unit between 1987 and 2007.....

Note: Parapsychologist Caroline Watt is the long-term partner (or wife?) of Richard Wiseman.

Quote:
...Interestingly, Watt’s paper also demonstrated a reporting bias. Only seven of the thirty-eight studies had made it into the public domain, presented as papers at conferences held by the Parapsychological Association. ...

- Richard Wiseman

Reporting bias against 'student' projects? Gosh, what a shocker. The PA might be biased against reporting student projects, in favor of qualified researchers? I'd be far more worried if it was publication bias against post graduates and staff studies.

Quote:
.....Watt’s analysis, although informative, underestimates the total number of psi-related studies undertaken at Edinburgh University because it did not include projects undertaken by students prior to their final year ....

- Richard Wiseman

Is Richard Wiseman psychic? How does he know the projects by students prior to their final year were non-significant? Why didn't his partner Watt include the others?

Quote:
it did not include projects ... experiments run by postgraduate students and staff,

- Richard Wiseman

What a shame because according to the Fortean Times article below ..... major studies by the postgraduate students and staff.... were successful.

Quote:
Even stage magicians were consulted to help guard against trickery. Between 1993 and 2003, six out of nine major experimental studies produced statistically significant results. When asked directly if he personally believed in telepathy, Morris replied that he just did the research – but added that there was accumulating evidence indicating that it does occur. “We are perhaps studying nature, but in its fuller form,” he suggested. And then, in the late summer of 2004, Bob Morris went and died. Suddenly. Unexpectedly.

- Fortean Times

It seems there has been no 'major' reported experimental projects at the university since Caroline Watt and magician Peter Lamont took charge after 2004? Must be low on funding, I guess.

Quote:
or any work conducted before 1987.

- Richard Wiseman

I think Wiseman must be referring to the early student work under Dr John Beloff?. Why is Wiseman assuming these were non-significant studies? Why were these not included?

Quote:
To help the field move forward and rapidly reach closure on the psi question , parapsychologists need to make four important changes in the way they view null findings.

- Richard Wiseman

Rapidly reach closure? Why does Wiseman want to rapidly reach closure? Wiseman repeats this in the Skeptiko interview ...

Quote:
'...So the article was really about how we might try to push that forward and try and get some closure, i.e., can we produce the level of evidence one would need, and if not, do we agree not to carry on with the endeavor? '

- Richard Wiseman

So Wiseman wants to reach closure on parapsychology. It seems Wiseman wants to 'rapidly reach closure' on his partner Watt's career as a parapsychologist?

 

[alex.tsakiris]

Wiseman's selection of criteria:

The initial observation was that Pam's parents could tell when Pam was returning because of the way her dog behaved. This observation will be complicated by the fact that this occurs in the presence of knowledge about when Pam is expected to return and knowledge of when she has returned...

This is silly. I've done about 5 hours of show/interviews with all the players in this experiment (Sheldrake and Wiseman twice each).

First off, this was Sheldrake's experiment. He had been working with Jay-tee and Pam for over a year when he invited Wiseman in (he even let him use his camera). Wiseman changed the rules/criteria of the experiment to his ridiculous (and arbitrary) 2 minute cut-off crtiera that he invented... he then hid from Shedlrake and published the data without consulting the lead investigator/collaborator... really bad form. 10 years later he admitted (on skeptiko) that his data matches Sheldrake's (in a very convincing way as the Carter paper shows).

The data speaks much louder and more clearly than anything else.

 

[fls]

1) How so?

To combine all the hits, you have to presume that the base rate is the same, that is, that the effect in each study is the same. This means that Bem has to be wrong. That the standardness of the study has no effect. It also means that any of the other measures which are associated with variations in hit rate are not measuring variations in psi.

2) Do you think what Wiseman and Milton did was valid?

Yes, reasonably so. It isn't clear to me how the various studies should be weighted. I'm not sure that sample size is adequate, since it doesn't look like at least one of the larger studies is considered better quality. Since the Dalton study hasn't ever been published, it suggests that it's not up to snuff. Variance may be a better choice, but of course, Wiseman can't suggest this, otherwise he will be criticized. It's best he just goes along with what everyone else is doing.

Linda

 

[Wowbagger]

This is ridiculous! Either your results get stronger with more controls or weaker.

If they only get weaker, either the phenomenon you are trying to observe is not real; or you need to refine a testable hypothesis to account for those results, and see if you can strengthen the signal around that. No Testability = "Explained Away"

If you are actually able to isolate and measure a phenomenon. And, the signal for that phenomenon actually gets stronger with the more controls you apply: THEN some scientists might sit up and take notice. IF they are reliably replicable, by independent parties, you win big!

Until that happens, your criticism of the critics is pointless.

 

[Beth]

It may interest you to know that the only way it can be presumed that combining all the hits and misses into one big study and testing for significance using the binomial distribution is valid, is if you presume that psi does not have an effect. Otherwise, it is an invalid test.
So which is it?
Linda

That doesn't make sense. The null hypothesis assumes that there is no psi affect. That is, the statistics are analyzed presuming that there is no effect and if the p-value is sufficiently low, we can then reject that null hypothesis. There may be other issues, but I don't understand your complaint here.

To combine all the hits, you have to presume that the base rate is the same, that is, that the effect in each study is the same. This means that Bem has to be wrong. That the standardness of the study has no effect. It also means that any of the other measures which are associated with variations in hit rate are not measuring variations in psi.

Again, I'm not clear on what the issue is with doing this. Why would he not presume the base rate was the same for the null hypothesis?

If someone could quote this post, I'd appreciate it as I think Linda has me on ignore.

Beth

 

[Garrette]

You better explain since your opinion seems to be in the minority... even among skepitcs.

I'm a skeptic who has concerns raised by Carter's article, but don't count me in that number who make fls a minority (for the record, I don't think she is a minority).

[I wrote a lengthy post but erased it after some thought because there are really only two relevant points]

1. If you prove absolutely that Wiseman's critique was wrong and he is a monstrous hypocrite (something which is a long way from being demonstrated, let alone proven), then you still have done absolutely nothing to demonstrate the validity of Sheldrake's findings.

2. If you prove that Sheldrake performed his experiments and analysis correctly and his conclusions follow from the experiment, then you still are a long way from demonstrating psi; if the experiment is so ground-breaking, then why haven't the parapsychologists replicated it?

 

[fls]

This is silly. I've done about 5 hours of show/interviews with all the players in this experiment (Sheldrake and Wiseman twice each).

First off, this was Sheldrake's experiment. He had been working with Jay-tee and Pam for over a year when he invited Wiseman in (he even let him use his camera). Wiseman changed the rules/criteria of the experiment to his ridiculous (and arbitrary) 2 minute cut-off crtiera that he invented... he then hid from Shedlrake and published the data without consulting the lead investigator/collaborator... really bad form. 10 years later he admitted (on skeptiko) that his data matches Sheldrake's (in a very convincing way as the Carter paper shows).

The data speaks much louder and more clearly than anything else.

I'm sorry. I don't understand what this has to do with what I said. Regardless of whether any of that is true, it doesn't change the issue of which methodologies can reliably be used to discover if the dog's behaviour predicts Pam's return.

Linda

 

[fls]

That doesn't make sense. The null hypothesis assumes that there is no psi affect. That is, the statistics are analyzed presuming that there is no effect and if the p-value is sufficiently low, we can then reject that null hypothesis. There may be other issues, but I don't understand your complaint here.

Again, I'm not clear on what the issue is with doing this. Why would he not presume the base rate was the same for the null hypothesis?

If someone could quote this post, I'd appreciate it as I think Linda has me on ignore.

Beth

I explain about the issues of combining studies here:

http://www.internationalskeptics.com/forums/showthread.php?postid=4636708#post4636708

Linda

 

[cj.23]

As I had already done this over on RatSkep, and it may be useful for those trying to follow the discussion a little reference for those who want to reach an educated opinion on the debate between Wiseman and Sheldrake on psychic dogs--


Latest article by Chris Carter from JSPR - http://www.sheldrake.org/D&C/controversies/Carter_Wiseman.pdf

The background

http://www.sheldrake.org/Articles&Papers/papers/animals/pdf/dog_video.pdf - Sheldrake's paper

http://www.richardwiseman.com/Jaytee.html is Wiseman's account

http://www.sheldrake.org/D&C/controversies/wiseman.html is Sheldrake's response


Listen to the participants speak

Skeptico podcasts on the issue featuring Wiseman & Sheldrake on the dog experiments
http://www.skeptiko.com/11-dr-richard-wiseman-on-rupert-sheldrakes-dogsthatknow/
http://www.skeptiko.com/35-dr-steven-novella-and-dr-richard-wiseman-on-dogs-that-know-research/
http://www.skeptiko.com/rupert-sheldrake-and-richard-wiseman-clash/

Read much more on the issue, and take part in designing replications - Open Source Science
http://www.opensourcescience.net/in...ts:_Can_Dogs_Anticipate_Their_Owner's_Return?


Short Bibliography

Sheldrake, R. (1998). A dog that seems to know when his owner is returning: Preliminary investigations. Journal of the Society for Psychical Research, 62, 220-232.
http://www.sheldrake.org/Articles&Papers/papers/animals/pdf/dogknows.pdf

Wiseman, R., Smith, M. & Milton, J. (1998) Can animals detect when their owners are returning home? An experimental test of the 'psychic pet' phenomenon. British Journal of Psychology 89, 453-462.
https://uhra.herts.ac.uk/dspace/bitstream/2299/2285/1/902380.pdf

Sheldrake, R. (1999a) Commentary on a paper by Wiseman, Smith and Milton on the 'psychic pet' phenomenon. JSPR 63, 306-311.
http://www.sheldrake.org/Articles&Papers/papers/animals/pdf/comment.pdf

Sheldrake, R. (1999b) Dogs that Know When Their Owners are Coming Home. London: Hutchinson.

Wiseman, R., Smith, M. & Milton, J. (2000) The 'psychic pet' phenomenon: A reply to Rupert Sheldrake. JSPR 64, 46-49.
http://www.psy.herts.ac.uk/wiseman/papers/psychicdogreply.pdf

Sheldrake, R, Smart, P (2000) A Dog That Seems To Know When His Owner is Coming Home:
Videotaped Experiments and Observations, Journal of Scientific Exploration 14, 233-255
http://www.sheldrake.org/Articles&Papers/papers/animals/pdf/dog_video.pdf

Sheldrake, R., and Smart, P. (2000b). Testing a return-anticipating dog, Kane. Anthrozoös, 13(4), 203-212.
http://www.sheldrake.org/Articles&Papers/papers/animals/pdf/dogkane.pdf

Carter C (2010) Heads I win, Tails you lose, or how Richard Wiseman nullifoes positive results in parapsychology, and what to do about it JSPR 74, 156-2007
http://www.sheldrake.org/D&C/controversies/Carter_Wiseman.pdf

Hope of interest
cj x

 

[fls]

Thanks cj x. That's handier than googling and combing through old threads.

Linda

 

[Beth]

I explain about the issues of combining studies here:

http://www.internationalskeptics.com/forums/showthread.php?postid=4636708#post4636708

Linda

Thanks for responding to my post. Yes, this old post of yours discusses some issues with combining studies. But this does not address the question that I asked. What problem do you perceive with the analysis as result of assuming the null hypothesis is true?

 

[Ersby]

It might be helpful to post a link to Wiseman's paper, too.
http://www.csicop.org/si/show/heads...r_how_parapsychologists_nullify_null_results/

Is Richard Wiseman psychic? How does he know the projects by students prior to their final year were non-significant? Why didn't his partner Watt include the others?

He doesn't say they were non-significant.

What a shame because according to the Fortean Times article below ..... major studies by the postgraduate students and staff.... were successful.

Now, usually I like to refer to first hand reports, so I'm acutely aware that I'm responding to someone who's quoting someone else who's quoting someone else who's reporting someone else's data (without a reference!). Nevertheless, I'll have a go.

The nine "major" experiments are probably the nine ganzfeld experiments. As such, I'd need to know where else these are called "major" and by what criteria.

There is an overlap in these "major" experiments and the experiments listed in Watt's paper, which is why I suspect this "major" label is fairly meaningless in this context. Also, it is important to bear in mind that both Watts and Wiseman are talking about all psi experiments, whereas these "major" experiments are all ganzfeld work.

It seems there has been no 'major' reported experimental projects at the university since Caroline Watt and magician Peter Lamont took charge after 2004? Must be low on funding, I guess.

Again, until I know what you mean by "major", I can't comment.

 

[GnaGnaMan]

(Hi from your friends at www.skeptiko.com )

JSPR has published a penetrating critique by Chris Carter of Richard Wiseman's latest attack on parapsychology - his 'Heads I win, tails you lose' paper published in Skeptical Inquirer. Chris Carter's reply:
http://www.skepticalinvestigations.org/Mediaskeptics/Carter_Wiseman.pdf

As has been been pointed out, Carter's reply is mainly a Tu Quoque fallacy. It does not address the meat of Wiseman's "attack".
BTW the criticisms that Wiseman expresses are by no means new. They have been variously expressed by other people in the past and in greater depth (and altogether better IMHO). For example Alcock's Give the null hypothesis a chance.

Publication Bias
Publication bias cannot be proven. A meta-analysis may detect certain tell-tale signs of it but that is far from conclusive. The skew of outcomes that is detected in the latest Storm et al. meta-analysis is certainly such a tell-tale sign.
The attempts to determine whether a file-drawer-effect may be responsible for the entirety of the observed effect do only that. They do not rule out availability bias being responsible for part of the effect. That is true for Blackmore's survey as well as statistical methods.
As an aside, Wiseman doesn't claim what Carter attempts to rebut. Quite the opposite.

Demkina case
It is not clear if Carter says that Wiseman should have changed the success criterion post hoc (a big NO, obviously) or if he thinks that Wiseman should have employed a different one from the start.
There are problems with the protocol, even though not of the kind that Carter claims.
Demkina was allowed to inspect the subjects will all normal senses. It is not surprising that she was able to assign the diseases better than would usually be expected if she had just acted randomly. She was simply not acting randomly and there is no reason to think she should have.
In favor of such an approach one could validly argue that even though she could use normal senses, a very great skill at diagnosing would be interesting even if not paranormal.
If Carter believes that what she displayed was really diagnosing skill great enough to be interesting he can go investigate it himself. I certainly don't think so.
http://www.csicop.org/si/show/natasha_demkina_the_girl_with_normal_eyes/
http://www.csicop.org/si/show/testing_natasha/
http://www.csicop.org/specialarticles/show/statistics_and_the_test_of_natasha/

The psychic dog
Again, Carter displays a a great unfamiliarity with the literature.
Wiseman's success criterion was not arbitrary but made in consultation with the owners of the dog. The claim that a psychic dog would respond telepathically to the owner's decision to return home had been made widely on television.
It has since been repeated. There has been an abortive, unpublished (and obviously unsuccesful) replication attempt by Alex Tsakiris. At the time he promoted 2 videos claiming that the dog reacted at the time the owner set out to go home.
In short there is no problem with Wiseman's criterion (even if there are some valid critcisms of the experiment).
Sheldrake's criterion is simply defunct. By that criterion we should judge an action made by the dog as influenced telepathically by the owner even if the owner has not made the decision yet that supposedly causes the action. Huh?

Ganzfeld Meta-analysis
The simple truth is that Wiseman did precisely the same thing that previous meta-analyses had done.
Again, there are actual problems with the study. It fails, for example, to make tests for availability bias and other such issues.


In summaryt, Carter's attack piece is an ill-informed and fallacious piece of fail.
It does little but making the idea of wide-spread incompetence and fraud within parapsychology more credible.

 

[Rodney]

To combine all the hits, you have to presume that the base rate is the same, that is, that the effect in each study is the same. This means that Bem has to be wrong. That the standardness of the study has no effect. It also means that any of the other measures which are associated with variations in hit rate are not measuring variations in psi.

I don't know if we're talking past one another, so let me ask a question: If ten parapsychology researchers simultaneously undertook ganzfeld experiments with identical protocols (including 25% probability of success) and each recorded, say, 28 hits in 100 trials, would it be valid or invalid to aggregate those results? If invalid, why?

Yes, reasonably so.

So why are all of Wiseman's and Milton's negative Stouffer Z figures wrong?

 

[Ersby]

The nine "major" experiments are probably the nine ganzfeld experiments. As such, I'd need to know where else these are called "major" and by what criteria.

Okay, someone on skeptiko has posted a link that he says was the reference for the Fortean Times article, which pretty much confirms that Morris was talking about the ganzfeld work.

http://www.scimednet.org/2003-smn-a...tival-of-science-at-the-university-of-salford

My source was a powerpoint written by Robert Morris in February 2004 which lists nine ganzfeld experiments, six of which were significant. I figured that the similarity between this and the results from these "major" experiments wasn't just coincidence, and so it turns out to be.

But I'm aware that we're drifting from the point. Chris Carter thought that this paper didn't count as a "shred of evidence." I think it does.

 

[fls]

I don't know if we're talking past one another, so let me ask a question: If ten parapsychology researchers simultaneously undertook ganzfeld experiments with identical protocols (including 25% probability of success) and each recorded, say, 28 hits in 100 trials, would it be valid or invalid to aggregate those results? If invalid, why?

If you set up something like a multicentre trial, then it is okay to pool the results. But why bring this up, since it doesn't describe the situation at hand or reflect what I said? Do you disagree with the parapsychologists who think that the standardness of the tests, or the creativity of the participants alters the hit rate which represents the effect of psi?

So why are all of Wiseman's and Milton's negative Stouffer Z figures wrong?

I may have misunderstood what you were asking. I thought you were asking whether they had chosen a reasonably valid effect size and weighting method, not whether they had performed their calculations carefully. I don't know if their calculations are correct. As I mentioned previously, I found discrepancies between my calculations and the listed numbers (in several metanalyses using those studies, not just Milton and Wiseman) and I haven't followed up on where I think the problem lies. I wouldn't presume any of the numbers in these studies are correct (but I tend to do that anyway). It's not "negative Stouffer Z figures", though. It's "negative z scores". The Stouffer Z is what you get when you combine the z scores.

Linda

 

[alex.tsakiris]

The psychic dog
Again, Carter displays a a great unfamiliarity with the literature.
Wiseman's success criterion was not arbitrary but made in consultation with the owners of the dog.

It may have been base on a conversation with the owner, but it's still arbitrary (why a two minute cut off)... and a silly way to get an answer to Sheldrake's research question. Not even Wiseman offers much defense for this "success criteria" when simply measuring time-at-window/time-away-from-window gives you all you need to know.

The claim that a psychic dog would respond telepathically to the owner's decision to return home had been made widely on television.
It has since been repeated. There has been an abortive, unpublished (and obviously unsuccesful) replication attempt by Alex Tsakiris. At the time he promoted 2 videos claiming that the dog reacted at the time the owner set out to go home.

I did publish them on Youtube (as you noted). I then funded research at U of Florida... they couldn't go very far with it. This experiemnt is hard to do... lotta people problems. In general, the experience greatly increased my confidence that the phenomena is real.

 

[Beth]

It may have been base on a conversation with the owner, but it's still arbitrary (why a two minute cut off)... and a silly way to get an answer to Sheldrake's research question. Not even Wiseman offers much defense for this "success criteria" when simply measuring time-at-window/time-away-from-window gives you all you need to know.

I agree. It's been a while since I looked at this, but my recollection is that Wiseman set up his criteria after the data had been collected rather than prior to setting up the experiment. Do you know if that is the case?

I did publish them on Youtube (as you noted). I then funded research at U of Florida... they couldn't go very far with it. This experiemnt is hard to do... lotta people problems. In general, the experience greatly increased my confidence that the phenomena is real.

I find the data presented in the various papers, including Wiseman's data with Sheldrake's analysis, to be evidence for the hypothesis that something is going on. What, exactly, I'm not sure.

 

[cj.23]

I agree. It's been a while since I looked at this, but my recollection is that Wiseman set up his criteria after the data had been collected rather than prior to setting up the experiment. Do you know if that is the case?
.

I'll email Matthew Smith and ask him.

cj x